Some Lizzie Ai questions

[jlt]

15 blocks is better for slow computers and 40 for fast ones.I don't know when a computer is fast enough to use 40 blocks.

[JoeS1]

Does anyone else know any reason why there'd be such a difference in calculation of variations between Lizzie and KaTrain and what I asked in previous post? I know this community is kind of small. Is there anyone who knows how these programs work that has gone in depth on AI? I'm having a hard time Googling and finding specific info on AI Go.

[PelleasGo]

Does anyone else know any reason why there'd be such a difference in calculation of variations between Lizzie and KaTrain and what I asked in previous post? I know this community is kind of small. Is there anyone who knows how these programs work that has gone in depth on AI? I'm having a hard time Googling and finding specific info on AI Go.

First of all there's just inherent variance. I can run the same sgf through KaTrain two times under 25,000 playouts a move and the evaluations will differ.

Then there's all the differences caused by the different components. If you're using different neural networks through the same engine, that will cause differences. And if you're comparing different engines entirely, such as Lizzie and KataGo, then that will also be the cause of differences (on top of the fact that you're inherently using different neural networks and inherent variances between runs).

[lightvector]

15 blocks is better for slow computers and 40 for fast ones.I don't know when a computer is fast enough to use 40 blocks.

Probably 40 blocks is better no matter what speed your computer is at this point (specifically, the latest nets from https://katagotraining.org). It's trained so much more that it will just dominate. Also the 15 blocks net predates some specific joseki training (mi yuting's flying dagger, some 3-4 pincer joseki) that are known AI blindspots for bots that don't have training for such positions, so it will systematically make well-known mistakes on some joseki that new nets will get right even with far less compute.

[lightvector]

Thanks for the replies. I was trying to find more info on Lizzie AI and came across KaTrain and AlphaGo Zero/Master games which I didn't know existed, and have been replaying those. Surprising to see it still makes mistakes as well, but they are typically giving up a point or two endgame or answering a KO threat in a way that adds another unnecessary KO threat. Those are the easy things even a kyu player could see. It almost seems like there should be a second pass where the program takes the top moves and analyzes which ones lead to the most points gained locally or something. This seems feasible endgame when there aren't many possibilities left.

Heh. Keep in mind that AlphaZero/Master put literally zero value on winning with a larger score versus a smaller score. The're not trying to gain points locally, or trying to avoid giving up points, in the first place. They're only trying to win, and if they see that giving the opponent free points in some endgame situations wins just as much as not giving those points, then they're indifferent, a win is a win. You can see this clearly in one of the AlphaGo vs Ke Jie games, where AlphaGo won by 0.5 points, but was ahead by far, far more (10+ points, if I recall).

KataGo "fixes" this by putting a mild weight on maximizing points as well. Winning/losing is valued much more, so KataGo will still give up points for safety, roughly to the degree that a human pro sometimes yields and plays more peacefully when ahead instead of stretching for the maximum possible. But even KataGo will not care things that don't affect points (like the final ko threat count), and may due to noise or positional difficulty in the search sometimes waste threats (Sometimes bots will waste threats at very similar moments that a human would deliberately waste a threat for timesuji! Although the reasons are a bit different). And of course, bots can still make mistakes, as mentioned before.

When searching for a guide on Lizzie I found mention of KaTrain and have been messing with that. Lizzie doesn't seem to have an option to end the game and count the score. KaTrain does, but after downloading all the modules, I noticed it only seems to do a few hundred variations across a few moves total per minute. Is KaTrain using a more advanced version of KataGo? I'm on Lizzie version 0.7.4 which appears to be the latest version.

I'm using the Open CL, 1.11 windows 64 bit engine, because I think that's the only one for GPUs, but there are 6 models. There are 15 block, 20 block, 30 block, 40 block, latest model, strongest model. Does anyone know how these compare to one another? Is Lizzie using the 15 block? Most I could find was 40 block was better than 20 block, but I didn't notice much of a difference in calculations between 40 block and strongest version. Still seems incredibly slow.

I believe Lizzie by default comes with a pretty old 15-block Leela Zero, and a very outdated 20-block Katago. Neither of these nets is the best choice for analysis, you should just use KaTrain's strongest model. Which you can upgrade Lizzie to use if you like, by manually downloading all those updated versions and networks and replacing the existing versions in your Lizzie download with those.

Even the older nets will often provide similar answers in analysis situations, simply because they are still strong and in a position where there is a clear best move, or there are clear "normal" continuations, all strong bots, and often high dan or pro humans, might well agree on them. But at the times they disagree, likely the strongest 40b net will be right more often than it's wrong compared to older nets, even despite being able to compute about 4x less.

If you have more "basic" questions, you might find better answers asking your questions in discord chat: https://discord.gg/bqkZAz3 - it's a pretty quiet discord, but some of the people hanging out there know a lot more about the details of different bots and related programs than those elsewhere.

[johnsmith]

Lightvector, you're a real gift to the go community!

I'd just like to add that there is a software called LizzieYzy https://github.com/yzyray/lizzieyzy/releases, which is Lizzie on steroids in a way. It comes with the latest executables (both GPU and CPU), latest 40b net, strongest 15b net etc. There are so many exquisite features in there that every go player who wishes to study and improve in this beatiful game can utilize.

Big thanks to lightvector and yzyray

[Knotwilg]

Any idea why suddenly I get an error prompt when selecting the KataGo engine from Lizzie? Downloading the latest version didn't help.
I didn't change anything else lately on my laptop.
Installed latest Java too and the errors keep coming. LeelaZero works but when switching the engine to KataGo it fails.

[Gomoto]

@Knotwilg: Just for your information Katago is working with Lizzie on my system like always. What is the error message on your computer? Was there a recent driver update for your gpu?

[Knotwilg]

Thanks for asking - it works again on mine too. I can't reproduce the behavior.