Leela Master GX88 plays good 4h games against Leela 0.11

[splee99]

While the latest Leela Zero struggles to win any three handicap games with Leela 0.11 in time parity, Leela Master GX88 has no trouble at all to win four handicap games. I guess one of the advantage of GX88 is the knowledge of many openings other than 3-3 invasion. This game I stopped manually because for GX88 the winrate was already 98% at the last move.

lm88-l-20s-4h.sgf

(2.25 KiB) Downloaded 1920 times

[sorin]

While the latest Leela Zero struggles to win any three handicap games with Leela 0.11 in time parity, Leela Master GX88 has no trouble at all to win four handicap games. I guess one of the advantage of GX88 is the knowledge of many openings other than 3-3 invasion. This game I stopped manually because for GX88 the winrate was already 98% at the last move.

lm88-l-20s-4h.sgf

Very interesting, thanks for sharing!

Judging from this game, I wouldn't say that it knows many openings other than the 3x3 invasion
But beating Leela 0.11 at 4 stones is very impressive.

Do you know how does GX88 compare to pure LZ in direct matches?

Also, how does GX88 compare to more recent GX.. versions? I notices many versions that are more recent than 88 when I looked at https://github.com/pangafu/LeelaMasterW ... er/weights.

[splee99]

In even games Leela master is slightly weaker than LZ. I tested it with #195 and LM lost. So I believe LM is weaker than all the later LZ weights. However, it seems to me that learning better opening moves is the way to go for winning handicap games. If the LZ current weight plays a 4 handicap game it almost certainly invades all four 3x3s and then lose the game.

I don't see any newer weight than GX88 on Google drive.

[sorin]

In even games Leela master is slightly weaker than LZ. I tested it with #195 and LM lost. So I believe LM is weaker than all the later LZ weights. However, it seems to me that learning better opening moves is the way to go for winning handicap games. If the LZ current weight plays a 4 handicap game it almost certainly invades all four 3x3s and then lose the game.

I see, with all 4 early invasions in 3x3 it seems hard to white to win at 4h, since there are too few chances to complicate the game.

I don't see any newer weight than GX88 on Google drive.

That is strange, I wonder if we are looking in different places.
I am looking in the LeelaMaster_G_15-192 folder from my link above.
I see GX88 has a "Nov 20, 2018" timestamp, and I see many other networks with newer timestamps: GX89 (Nov 27), etc... newest seems to be GXAA (Jan 2, 2019).

[splee99]

OK, I just see it. I'll try GXAA. Thanks.

[Uberdude]

That loose net and capture of the h6 stones was brutal! But Leela 0.11 made quite a few mistakes to manage to lose that semeai.

[splee99]

Well it seems the above game was just a lucky one for lm GX88. It lost many other 4h games with Leela 0.11 mainly due to not being able to read ladder correctly. I have also tried GXAA and it is not good at 4 handicap games. In the beginning of the game the winrate is already lower than 1% and it is easy to fall down a ladder.

[sorin]

I have also tried GXAA and it is not good at 4 handicap games. In the beginning of the game the winrate is already lower than 1% and it is easy to fall down a ladder.

I also tried GXAA, and it seems to be closer to LZ in terms of style, it favors 3x3 much more than GX88.

[Vargo]

I've tried two 10 game matches :
LZ0.16 with weights LM_GXAA v. Leela 0.11 OpenCL

GPU: 1x1080, twogtp 1.5.0, komi 0.5

Complete command line for H4 :

Code: Select all

gogui-twogtp -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\LM_GXAA.txt --noponder -v 4000 -t 4 -r 1" -black "C:\Users\jm\gogui150\Leela0110GTP\Leela0110GTP_OpenCL.exe --gtp --noponder --thread=4 -p 9000" -games 10 -sgffile l011h4aa -handicap 4 -auto -komi 0.5

Complete command line for H3 :

Code: Select all

gogui-twogtp -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\LM_GXAA.txt --noponder -v 4000 -t 4 -r 1" -black "C:\Users\jm\gogui150\Leela0110GTP\Leela0110GTP_OpenCL.exe --gtp --noponder --thread=4 -p 9000" -games 10 -sgffile l011h3aa -handicap 3 -auto -komi 0.5

1) at H4, Leela 0.11 wins 9-1
2) at H3, Leela 0.11 loses 9-1

At H4, average game length is only 130 moves, because LZ usually resigns soon. Average time used per game, B : 104s, W: 138s
At H3 , average game length is 280 moves. Average time used per game, B : 272s, W: 285s

All the H3 games (only game lost by LZ is l011h3aa-1.sgf)

H3.zip

(10.77 KiB) Downloaded 613 times

All the H4 games (only game won by LZ is l011h4aa-4.sgf)

H4.zip

(7.51 KiB) Downloaded 600 times

[splee99]

Thanks for the test. The GXAA is not good at 4h games. Could you please test again with GX88 for 4h games?

[Vargo]

Two 10 game matches Leela0.11OpenCL (B) v. LZ0.16 LM_GX88 (W)
Time parity, H3 and H4, same parameters as before.

at H3, LM_GX88 wins 10-0
at H4, Leela011 wins 8-2

Detailed stats :

H3

H3.gif (45.17 KiB) Viewed 11066 times

H4

H4.gif (44.15 KiB) Viewed 11066 times

[Uberdude]

I tried GX88 against a human 9d on Fox. It was interesting: messed up big time in the opening by playing a move which was only good if it had a ladder to counter opponent's resistance (which it barely considered so didn't read) but didn't. But impressively it managed to come back from 15% to win.

viewtopic.php?f=18&t=16198&p=240996#p240996

[splee99]

That was an interesting game indeed. I guess all GX series are bad at reading ladders, including GXAA. The recent 40b weights are much better in that aspects. By the way GX89 seems to be little stronger than GX88 from my handicap game tests.