I heard it was beating professional players. However, I just downloaded it, and I have good chances against it taking a 4 stone handicap. I play the regular leela at that same handicap. It doesn't use or select joseki. It seems to play terrible against handicap stones. It seems to usually enter at the 3x3 point, which is usually terrible, simplifying things and allowing me to create a huge moyo.

Leelazero doesn't play well against handicap.

Also, its strength depends a lot on the number of visits. How many visits per move did you use?

What's a visit, and how do you configure that?

To imagine how strong currently LeelaZero is, take this into consideration:

There is a bot called BensonDarr (aka PhoenixGo), plays on FoxGoServer.In last 300 games against Fox 9d (including many pros) he scored 299 wins.In last 20 games against LeelaZero he scored 1.

I suspect this particular LZ is run on stronger hardware than BD, but anyway, LZ can be damn strong.Handi games are different thing, you need to tweak Leela to play good on handi, but 3-3 invasions are not that rubbish as you think, even in middle handi games.

--gtp --playouts 2500 --noponder --threads 4 -w C:\Users\Joe\Documents\leelaz\logs\ww.txt
time_settings 0 5 1

Above is how I have it configured. Is there a way to configure it to play stronger? I am using a 2 year old standard high end laptop.

You are obviously a stronger player than me. However, I know 3-3 invasions are playable in handicap games, but starting out by invading one corner then another in a 4-stone game is obviously not optimal.

There is specially tweaked LeelaZero bot on KGS named Geomancer (or geomancer2, not sure) - he is able to cope with high handi (including 9 stones handi if i am correct) - you can try it, i guess it will be much harder task to beat it that Leela on your laptop.Might be hard to get game with it, however i think it is at least worth to try.

FineArt and Golaxy are approx. two stones above top humans. dunno about LZ.
In a recent exhibition, a bot gave four women pros 2 stones and scored 4-0.
Male top pros have yet to play the top bots at 2-stones in public,
for various reasons (*cough*).
And yes, high handicaps (above 2 stones) are another story, as the bots haven't been sufficiently trained for those.

Even though it didn't know how to play a 4-stone handicap game, I didn't think leelaz on my computer played anything like the level of a top pro. I was surprised. Is there a way to configure it to play stronger? I was hoping to learn a lot playing 9-stone games against a top pro level program, but that doesn't seem possible.

If I understand correctly, LeelaZero thinks for 5 seconds before playing a move. If you replace "time_settings 0 5 1" with "time_settings 0 15 1" it will take 15 seconds so will be stronger. You may also increase the number of playouts but this won't have any effect if your computer is too slow to reach 2500 playouts within 15 seconds. Whaterver, don't expect superhuman level while playing with more than 2 handicap stones.

Can you just remove the time settings parameters so it does 2500 playouts. That might then take 342 seconds a move and you will have learnt you have a slow computer and why it was weak before.

Also what network file are you using?

I can assure you it's very strong. Try playing even games or up to 2 handicap stones and see if you feel ahead at any point in the game.

The problem with higher handicap is: since leela expects the opponent to play good moves, it thinks all its own moves are losing moves (since the opponent is so far ahead with the handicap). So it doesn't differentiate between "good" and "bad" moves anymore and starts playing this weird stuff. It doesn't expect us to make that many mistakes
I think there is some work going on trying to correct this but I'm not up-to-date with the progress.

Other ways to increase strength (might have already been said):
1) use a dedicated graphics card !!!
2) give more time/playouts (much easier with (1))

But yeah, reduce the handicap (2 max) and there should be no more crazy stuff (except the occasional ladder hiccup).

Make sure your GPU can indeed do 2500 play out in 5 seconds. Usually you don't use time settings when payouts is specified, or just use time settings without payouts. You can also take out the --noponder setting.

I watched In-Seong, Seong-Jin and Young-Sam team up against Leela Zero towards the end of last year. Leela Zero was being run on a gaming laptop but nothing particularly special as far as hardware is concerned. Nevertheless, the trio lost.

In the opening stages, the audience were laughing at Leela Zero for playing a move that looked like a ladder-reading mistake. In the aftermath, that very move turned out to be a good one -- just one that no human would have judged to be so.

How can we even question whether Leela Zero is "that strong" when there is concrete evidence to the contrary?



Leela Zero has defeated professional players. Even very old versions & networks.



None of these AlphaGo-Zero-like bots play well with handicap stones on the board. They were trained to play even games with a fixed rule-set and komi value -- any other scenario is not what they were trained for and they will perform sub-optimally.



This is patently not true. Leela Zero *does* indeed play some human-invented Joseki in even games, at least. There are also new Joseki that these new algorithms have popularized and that are now played by professionals.



This generalisation is now considered obsolete theory. AlphaGo showed that invading at 3-3 is often a very good move, indeed. Most of the new bots and modern professionals play it, today.



Again, obsolete. Modern thinking says that concrete territory is more valuable than large moyos and that sabaki is always possible. There are some exquisite examples of this in the AlphaGo Zero games. Whatever the case, these algorithms play as if they can live, anywhere, if necessary.

I would be very interested to hear if you have a similar experience with KataGo, a bot that I've been working on.

You can actually play it right now on OGS, it's one of the many bots available there, and seems to be pretty popular as a handicap game opponent, judging by its game records: https://online-go.com/player/592684/

For running on a local machine, it's a more difficult to compile and use, requiring CUDA+CUDNN and a modern NVIDIA GPU (I am steadily working to remove that requirement), but if you have that, then it should be straightforward to compile on Linux from the GitHub source using the stated instructions, or on Windows 'alreadydone' has generously worked to produce some precompiled binaries.

Like Leela Zero, most of its training data is from even games, and possibly as a result it still likes doing lots of 3-3 invasions in handicap games, which is a very strong move in even games, but which, like you, I am slightly skeptical of for very high handicap. But unlike Leela Zero, it explicitly tries to maximize score in addition to just winning, so in a handicap game the search still has a reasonable idea of what to shoot for instead of just seeing all moves as almost 100% losing and indistinguishable, so it's able to often play well even in high-handicap: example 13x13 example 19x19. It is NOT as strong as Leela Zero due to having far less training, so at least in even games it is "merely" pro-ish level or slightly superhuman depending on hardware, rather than vastly superhuman. (Also a thing I'm steadily working on).

But there's a chance that it would provide the opponent that you're looking for, if you're interested in trying it, and as it's still under active development, feedback is of course welcome.

As a separate note, on OGS handicap stones in Chinese rules are free placement, so you can try putting some 3-4 or 3-3 stones if for learning purposes you want to get a greater variety of corner patterns than just 3-3 invasions, and presumably whatever GUI program you're using to play against bots locally should have a way to do this too?

Edit: Oh, and I just remembered there's also this issue from the LZ github where ihavnoid explores adding territory prediction and an explicit score maximization term to LZ as well, which might be of interest here too: https://github.com/leela-zero/leela-zero/issues/2331

Is there a way to configure it not to resign? When I play leelaz at 5 stones, it will resign when I think it still has good chances due to its skill advantage.

It also doesn't run if I take out the --noponder option.

As far as invading at the 3x3 point, it does it in situations where it clearly isn't good, like first move of a handicap game. This also simplifies the game and is not good handicap strategy.

running leeaz --help gives



so "-r 0"

I changed the configuration to not resign. I then played a game where it edged me out at the end by 3 points at 6 stones and I won easily at 7 stones. I couldn't often beat a weak professional player at 7 stones and I play regular leela at 4 stones. I might be able to beat leelaz at 5 stones with practice.

I noticed that it generally did not launch invasions enough in a high handicap game. Also, it appeared to make clear errors in capturing stones that were pretty dead. I find it hard to believe it could beat top professionals, but it could be my computer and configuration also make it weaker.

If you want to know if LZ can beat top professionals in an even game then look at the result of it playing top professional in an even game. Trying to extrapolate from a high handicap against a 1k and how many stones pros can give a 1k is answering a different question. You are using the assumption if A beats C at x handis and B loses to C at x handis that implies A will beat B in an even game. Whilst that is often (but not always) true for humans, it is often not true for bots which are trained to play even games and bad at handicap.

Here is LZ on my £200 GPU easily beating a Japanese 8p (goratings ~#500) in a one sided win by over 20 points. That was with a few thousand playouts a move, strong hardware is in the hundreds of thousands so will be stronger. https://lifein19x19.com/viewtopic.php?p=240789#p240789

I'm not aware of many recent even games between top pros and LZ. Here is a review from over a year ago of LZ (a 20 blocks network, much weaker than latest 40 block ones) on 4 beefy Tesla V100 GPUs beating top 10 pro Fan Tingyu 9p.
https://youtu.be/QZY2erUtJ_0

The "AlphaGo Style" bots do not play a good handicap game. Their whole design is to learn by themselves on how to play even games on 19x19.
They dont train handicap, and since their judgement has been so honed and perfected, they constantly think that they are loosing in handicap, making them sometimes play silly moves.
When every move is a loosing move, it doesn't really matter what you play.. they're incapable of understanding that you will play sub-optimally. They just think the opponent will play as good as they can.

Is Leela Zero stronger than any human professional alive (and probably dead) on a mid range GPU? Yes, has been for ages, and can probable give 0 komi or 2-3 handicap.

A small handicap is easier for the bot to claw back, as the judgement of winning percentages is still in a sane range. When you go to high handicap every move is a loosing move in the eyes of the bot.


A F1 car doesn't really work well as a rally car, and a rally car doesn't really work well as a F1 car. Both are racing cars built to win
Just because a bot can destroy any professional in an even game, doesn't mean it can play handicap (sadly)