I previously had Lizzie and Leela Zero working without a GPU under Debian 9, but the atttempt to replicate it six months ago under Debian 10 failed, in ways that are now lost in the mists of time. I decided to try again at Christmas when my son would be able to help out, and to buy and install a GPU to make things harder. This post is a detailed account of what happened, because... 1) I like war stories; and 2) The record may help with the inevitable repeat when Debian 11 becomes the next stable release.
It is likely that 80% of what we did was unnecessary, but I got a working setup in the end, I learnt about Docker and I get this chance to muse about my limitations in public. Extensive use is made of hidden sections to improve the flow.
To start with the conclusion: we bought an Nvidia GeForce GTX 1660. It manages 250 to 1000 visits per second, depending on the game state. Weights file 877d874de6dc08110158ac3b321f131cb6f276c5bd07eaa4bef9fdf12c9f3f68.gz was used. As an odd test of accuracy, Leela reproduced AlphaGo's mistake in answer to Lee Sedol's move 78 in game 4 of their match.
Why was the task so hard? 1. The GPU card was installed for use with an existing Linux installation. If the GPU had been present at the time of installation, it is possible that more support would have been built in to the O/S already. 2. The available online instructions for Lizzie and Leela under Linux were written by and for the Ubuntu distribution, and things are a bit different with Debian. This actually helped us during software installation because my son has a laptop with a small pre-installed Nvidia GPU running Ubuntu. He quickly got Lizzie and Leela up and running in that environment, which we could then try to replicate under Debian. 3. The task may be a bit harder under Debian 10 than it was under 9. The next hidden section describes my usual approach to Debian upgrades and how it failed to help in this case.
My 70-year-old eyes, shaking hands and inflexible brain would have been unequal to the task without help.
The hidden section describes my usual approach to Operating System upgrades.
The UK computer writer Jack Schofield offers the following maxim which I paraphrase. Whenever I find myself putting data into a new program or system, I ask myself "How can I get it out again?". Experience has indeed taught me that it may not be easy to decode, decompress or decrypt data which was encoded, compressed or encrypted by an older version of the same operating system. The upgrade from Debian 8 to 9 was particularly troublesome. All the problems could be fixed, but it took time to track them down. My policy now is to install each new version using dual boot and run both versions alternately for one or two months until it is clear that everything works before switching over entirely. In this way, it is often possible to execute an old program under the new operating system to recover the data.
An upgrade can also introduce other problems if a program has changed its behaviour in ways that I consider unacceptable. In that case, I can sometimes keep the old version or build the program from source and change the behaviour myself, but there will be no automatic security fixes afterwards.
The upcoming change from X windows to Wayland in Debian may cause other problems: I use the XFCE desktop environment because it is relatively simple and does not get in the way. It is not yet clear that XFCE will run under Wayland.
Another example is xpdf, which is my preferred viewer for .PDF files. (It is simple and does not get in the way.) It now has no Debian maintainer. If Debian abandons xpdf, I may be able to use an old binary for the rest of my life.
Despite all this caution and suspicion, the upgrade to Debian 10 caught me out. The old binaries for Lizzie and Leela would not run under the new O/S, and a newly compiled and updated Lizzie never managed to run the leelaz program. It was probably a simple configuration error, but not one I could find.
There are various online instructions for building Leela zero and Lizzie for Linux. We decided to compile from source. This worked easily under Ubuntu, so we decided to prototype the method for Debian by installing a Docker instance of Debian 10 on the Ubuntu laptop and then to repeat the process inside Docker. That worked, though it was not possible to run Lizzie inside Docker because the Debian instance that we chose is a small one that does not have a Graphical User Interface. Instead, the resulting binaries were copied out of Docker into the host Ubuntu system, where they did run. We decided to repeat the process on the target PC with Debian 10 as the host and the same Docker instance of Debain 10. That also worked, and the binaries from Docker also ran on the Debian 10 host. Our scripts appear below. This same method will be used when Lizzie, Leela or the Operating System are next upgraded, because new stuff can be tried out inside Docker without messing up the existing running system.
GPU Choice and Installation
We found the GTX 1660 online for about £180. Two weeks later, it was on sale for about £210 from the same supplier. Apparently, the price of GPUs varies with the price of cryptocurrencies.
The GTX requires at least a 400W power supply. The existing power supply has a big label on its side reading "450W", so that was not a problem.
We had concerns about cooling inside the PC's case. An "Air Duster" aerosol can removed a lot of dust from the cooling fins attached to the CPU heatsink, which helped a lot. The Air Duster was available from Poundland (a Dollar Store in US English) at the time. We also decided to replace one fan in the wall of the PC case that had a slightly noisy bearing. This was not easy, because the old fan took its power directly from the main power supply unit, and we had bought a new fan that expects to plug into a different kind of socket on the motherboard. The solution was a "VanDeSaiL 4 Way Black Sleeved Braided Fan Power Cable 15 Inch for 12V", which was quite cheap to buy. It connects to the Power Supply and provides several extra sockets like those on the motherboard. The result is bit more clutter inside the case, but the main problem for me would have been knowing that such cables even exist.
There was enough room for the GPU board: at least we researched that properly. However, it was supplied without a self-tapping screw to secure it to the PC's case. That was a problem, because the GPU is relatively heavy, and it needs to resist the force when the HDMI cable is plugged in to it. We borrowed a screw from a smaller and lighter board temporarily.
After improving the PC's cooling and installing the GPU board, we closed the case and moved the HDMI lead from the socket on the PC motherboard to the socket on the new GPU. Fortunately, the Linux kernel already contained the reverse-engineered Nouveau driver for Nvidia devices, so we could still see screen output after rebooting, even though the screen resolution was wrong and some desktop elements were therefore off the screen.
The following commands added the non-free driver and a detection utility. Non-free sources were already enabled in /etc/apt/sources.list. We ran everything as root. You might want to be more careful.
We then rebooted and adjusted the display settings using the GUI's Display Settings editor. We changed the default 1024x768 resolution to match the actual screen and then adjusted the dots per inch (DPI) until text looked right. The resulting display looked like it did before the GPU was added. We also disabled the default nouveau driver:
Code:
echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf
We then installed the following packages to support Nvidia and OpenCl. (The system probably had some of these already, but asking for them again causes no problems.)
The main script is go.sh (below). It calls the following three scripts in turn. The three sub-scripts were used to exploit cacheing in Docker during development and testing.
Script build-leela-and-lizzie-1.sh
Code:
######################################################### # Set up dependencies #########################################################
# To use an nvidia GPU with leela-zero we need to install proprietary nvidia drivers. # To install these we must enable the contrib and non-free parts of the default apt repos. sed -iE 's/main$/main contrib non-free/g' /etc/apt/sources.list &&
# Remove a warning in apt that CLI utilities are not installed. We don't need them. export DEBIAN_FRONTEND=noninteractive &&
# Now update repos and install apt-utils, which is necessary for apt (dont know # why but it moans without). Note that when installing apt-utils, debconf will # emit a warning that apt-utils is not installed, but the internet says this is ok. apt-get update && apt-get -y dist-upgrade && apt-get install -y apt-utils &&
# Install nvidia driver. nvidia-detect must be present before # nvidia-driver can be installed. apt-get install -y --install-suggests nvidia-detect && nvidia-detect && apt-get install -y nvidia-driver nvidia-cuda-toolkit &&
# Install a few packages to support the build process apt-get -y install git
# Remove a warning in apt that CLI utilities are not installed. We don't need them. export DEBIAN_FRONTEND=noninteractive
# Create a temporary directory to build leela mkdir /tmp/go-program-build && cd /tmp/go-program-build &&
# Clone the leela-go repository from github and follow the instructions given there to build it. git clone https://github.com/leela-zero/leela-zero.git && cd leela-zero && apt-get install -y clinfo && clinfo &&
# Remove a warning in apt that CLI utilities are not installed. We don't need them. export DEBIAN_FRONTEND=noninteractive
cd /tmp/go-program-build &&
# Install maven (which pulls in a jre) for lizzie apt-get install -y maven &&
# Now clone the lizzie repo from github and build it. git clone https://github.com/featurecat/lizzie.git && cd lizzie && mvn package
When the three sub-scripts are in place, it is finally time to run the script go.sh, which is as follows.
Code:
# go.sh
# First remove all pre-existing outputs rm -rf outputs/*
# Use docker to build leela-zero and lizzie in a debian buster environment. # The command uses the commands located in ./Dockerfile and builds a container # called leelaz-build-env which will contain leelaz and lizzie. The dockerfile # invokes the scripts called buildscripts/build-leela-and-lizzie{1,2,3} inside # the new debian environment. docker build -t leelaz-build-env .
# Now copy the binaries into ./outputs. mkdir -p outputs docker run -it --mount type=bind,source=$(pwd),target=/mnt leelaz-build-env sh copy-outputs.sh
If the above process succeeds, the remaining operations are performed on the host system. Copy lizzie-0.72-shaded.tar and leelaz from ./output and the weights file from ~/Download to a common directory. Rename the weights file as lznetwork.gz, then run this command in the common directory.
Code:
java -jar lizzie-0.7.2-shaded.jar &
GPU Initialisation took about 2 minutes, then lizzie and leela were ready.
To tidy up the docker installation, do something like this.
Code:
docker system prune -f --volumes docker container prune apt-get purge docker-ce-cli maven # There are probably many more that could go. apt-get autoremove rm -rf /var/lib/docker
Users browsing this forum: No registered users and 1 guest
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot post attachments in this forum
Login
Register
FAQ
Search
thirdfogie
