Leela has a new version

[lodestone]

I'm curious what other people's experiences are with the rated game system on the newest version of Leela.

When I play rated games on a 9x9, I wind up settling at 6kyu or 7kyu most of the time, which is slightly below my actual strength.

On a 19x19, I wind up at 1kyu or 2kyu usually, which is well above my actual strength.

Are others having similar experiences, or does this just tell me I'm weak at fighting but stronger at whole-board thinking?

[DrStraw]

I have it on my computer but rarely use it. I am all over the place. Anything from 1d to 4d. I think I tend to play it too quickly and overlook silly things so that I lose though silly mistakes. I need to get over the attitude of it's only a computer and so no big deal.

[gowan]

I'm curious what other people's experiences are with the rated game system on the newest version of Leela.

When I play rated games on a 9x9, I wind up settling at 6kyu or 7kyu most of the time, which is slightly below my actual strength.

On a 19x19, I wind up at 1kyu or 2kyu usually, which is well above my actual strength.

Are others having similar experiences, or does this just tell me I'm weak at fighting but stronger at whole-board thinking?

Playing at 9x9 is generally easier than playing 19x19 so I would expect Leela to be weaker at 19x19. Also, most of the programs' strength depends on the hardware being used. A monster computer might close that gap between 9x9 and 19x19.

[Arty]

haha i don't know if i'm just really bad at playing against it or something but i'm a KGS 3-4D, Tygem 5-6D and i have yet to beat it a single game out of the past 4 that I played. my experience totally doesn't line up with what some of you guys are saying who (i think) are around my level...

[LovelyLull]

I played it against it and it was quite weak. It couldn't tell that it was dead in 3 different places. Sometimes, it plays well but it's easy to beat once you realize it can't read tactical sequences at all

[denizen]

I think a lot of the difference comes down to how many simulations Leela is getting to run. I had an email exchange with the developer where he estimated a doubling of thinking time as adding one stone to the strength of the program. Obviously, that depends on where you start from, and there must be diminishing returns at some point. But computer hardware could be making a multiple-stone difference in strength for the same amount of thinking time.

[splee99]

Every now and then bots tend to make simple low level mistakes in particular situation, which let people jump to the conclusion that they are weak.
When running on my computer, two core AMD with no GPU, it is certainly stronger than 4d. I lose most of the time when it gives me two handi's. Like other bots, Leela seems to have a good feeling of thick and thin shapes, aji's and even ko fight, but just bad at capture race.

[Drew]

I think a lot of the difference comes down to how many simulations Leela is getting to run.

When I use Leela to analyze the best next move, I find that usually Leela "decides" what is best within 100k simulations. Very rarely does a variation sequence change or get bumped from first rank after 300k simulations on that tree. None of this takes more than 20s on my desktop.

Are some of you seeing changes in Leela's best move after more simulations? or the same type of settling?

[denizen]

I think a lot of the difference comes down to how many simulations Leela is getting to run.

When I use Leela to analyze the best next move, I find that usually Leela "decides" what is best within 100k simulations. Very rarely does a variation sequence change or get bumped from first rank after 300k simulations on that tree. None of this takes more than 20s on my desktop.

Are some of you seeing changes in Leela's best move after more simulations? or the same type of settling?

I have the same observation. But I think there are probably some people with slower computers that don't get as many simulations in to get to that point. There's also the question of how many moves you'd play differently if you were 6d rather than 5d, for example. It might be that a few moves is the difference; I don't know.

[goame]

I think a lot of the difference comes down to how many simulations Leela is getting to run.

When I use Leela to analyze the best next move, I find that usually Leela "decides" what is best within 100k simulations. Very rarely does a variation sequence change or get bumped from first rank after 300k simulations on that tree. None of this takes more than 20s on my desktop.

Are some of you seeing changes in Leela's best move after more simulations? or the same type of settling?

I have the same observation. But I think there are probably some people with slower computers that don't get as many simulations in to get to that point. There's also the question of how many moves you'd play differently if you were 6d rather than 5d, for example. It might be that a few moves is the difference; I don't know.

On my desktop I need only ~2 seconds to get 100k simulations. Of course with the gpu version of Leela 0.8.0.
Often there is only a forced move on the board because every other move is far away from this best move.
But if you look at other positions with no forced moves you will see that the first move will change very often depends on simulations.
Sometimes when the opponent is thinking I get with Leela 12000000 simulations.

Also Leela is much stronger than Zenith GO 6 because it can use GPU and not only CPU.
Worstest result for Leela was a win with + 1.5 points.
Best result for Leela was a win with more than 100 points, I don't know the exact number it was to big to remember and happened to often xD

[pookpooi]

Also Leela is much stronger than Zenith GO 6 because it can use GPU and not only CPU.
Worstest result for Leela was a win with + 1.5 points.
Best result for Leela was a win with more than 100 points, I don't know the exact number it was to big to remember and happened to often xD

According to computer go server, Leela 0.8.0 on 4 cores 1 gpu is weaker than Zenith Go 6 (Zen 11.4) with only one core.
http://www.yss-aya.com/cgos/19x19/bayes.html

[Uberdude]

goame, are you sure you have Zenith Go playing at maximum strength. I recall its configuration is confusing, setting it to max strength by dan rank is actually weaker than giving it a decent amount of fixed time on a decent computer.

[goame]

goame, are you sure you have Zenith Go playing at maximum strength. I recall its configuration is confusing, setting it to max strength by dan rank is actually weaker than giving it a decent amount of fixed time on a decent computer.

I know that fact

[Gomoto]

Also Leela is much stronger than Zenith GO 6

Not true



q.e.d

Zenit Go 6 [setting: 20 seconds per move] (actual total time: 27:38)
Leela GPU [setting: 30 minutes for the game] (actual total time: 38:56)

CPU Intel 2500K GPU Nvidia GTX 970

(Leela is a good program and quite strong indeed. But it is not stronger than Zenith Go 6.)

[Gomoto]

Leela 0.9.0 is able to win against Zen 6 (Leela using 51 minutes versus Zen using 40 minutes)

Zenit Go 6 [setting: 20 seconds per move] (actual total time: 39:42)
Leela 0.9.0 GPU [setting: 30 minutes for the game] (actual total time: 51:25)

CPU Intel 2500K GPU Nvidia GTX 970

Leela is using Chinese Rules and Zenith is using Japanese Rules. Leela and Zenith do not agree on the status of the lower left corner. (Leela: Black wins with 21.5 points)

Please note settings were very unfair this time. (Leela was setup with pondering, calculating during Zeniths time as well. Leela not only used much more time, it also hindered Zenith by stealing CPU time )