LZ's progression

[And]

does anyone know where to download LZ ZQ elf-2, LZ ZQ elf-5 ?
https://github.com/breakwa11/GoAIRatings

[Vargo]

20 game match at time parity between
LZ0.16 #204 and LZ0.16 Elfv2
1x1080, twogtp 1.5.0, 5min per side and per game.

Elfv2 wins 13-7
All games by resignation, no error, no duplicate game.
Stats :

204 v elfv2.gif (52.85 KiB) Viewed 16657 times

[Uberdude]

Vargo, about how many playouts per move is this? The official LZ test was 1600 each and LZ won 65%.

[Vargo]

how many playouts per move is this?

5 min per side and per game is, in fact, ~3.5 min/game effectively used, and is ~2s/move. It's similar to -v 1600 for #204 and -v 3000 for Elfv2 , all this with 1x1080.

[Vargo]

Another 10 game match between LZ0.16_#204 and LZ0.16_Elfv2 at time parity
2x1080Ti, 5 minutes per side per game (probably similar to -v 5000 for #204 and to -v 9000 for Elfv2)
twogtp 1.5.0, no pondering, komi 7.5, no duplicate game, no error.
Result : Elfv2 wins 7-3.

The games :

204_Elfv2.zip

(9.67 KiB) Downloaded 743 times

I've used "-alternate", so, #204 is B in the even numbered games, and #204 is W for the odd numbers.
(#204 only won the games numbered 1, 3, and 6)


The command lines and the stats :

elfv2v204.gif (50.97 KiB) Viewed 16597 times

[nbc44]

LZ0.16_#204 vs LZ0.16_Elfv2 2x1080Ti, 3s per move:

gogui-twogtp -black "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\05d10f27.gz -t 12 --gpu 0 --gpu 1 --noponder --precision single" -white "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\05dbca15.gz -t 12 --gpu 0 --gpu 1 --noponder --precision single" -games 100 -sgffile 204-elfv2 -auto -time 1s+4s/1 -komi 7.5 -verbose
gogui-twogtp -white "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\05d10f27.gz -t 12 --gpu 0 --gpu 1 --noponder --precision single" -black "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\05dbca15.gz -t 12 --gpu 0 --gpu 1 --noponder --precision single" -games 100 -sgffile elfv2-204 -auto -time 1s+4s/1 -komi 7.5 -verbose

Nothing interesting:

Code: Select all

+28-72=0 (as black)
+34-66=0 (as white)

Total: +62-138=0

[Vargo]

Nothing interesting:

Why do you say that ? I find it very interesting, particularly considering it's 200 games !
________________________________________________________________________________________

New network #205

40 game match #205 v. Elfv2
1x1080, 5min per side and per game, no pondering, komi 7.5

Elfv2 wins 25-15 (62.5 %)

40 games :

elfV2_205.zip

(34.9 KiB) Downloaded 756 times

Command lines and stats (205 is B) :

Elfv2_205B.gif (96.14 KiB) Viewed 16295 times

Command lines and stats (205 is W) :

Elfv2_205W.gif (99.14 KiB) Viewed 16295 times

[nbc44]

Why do you say that ? I find it very interesting, particularly considering it's 200 games !

I suppose the test result is predetermined.

Long test now:
LZ0.16_#205 vs LZ0.16_Elfv2 - 2x1080Ti, 120s (wow!) per move, (it will be 10 games):

+1-4=0 (#205 is black)
+1-4=0 (#205 is white)

Elfv2 wins 8-2 (80 %)

P.S.
Dragon tail loss :

[Vargo]

In another thread, @jlt wrote an interesting comment :

... I would be surprised if, for some n, LeelaZero(n) didn't beat LeelaZero(n-10) more than 50% of the time.

The last 40b network is #207, it's now 50 networks away from the last 15b, and 30+ networks from the last 20b.

20 game matches LZ(n) v. LZ(n-10) at time parity, 3 min/game and /side, 1x1080, komi 7.5, no pondering, LZ0.16, twogtp 1.5.0.

#207 v. #197 --> 12-8 (40b v. 40b)
#197 v. #187 --> 12-8 (40b v. 40b)
#187 v. #177 --> 15-5 (40b v. 40b)
#177 v. #167 --> 13-7 (40b v. 20b)
#167 v. #157 --> 5-15 (20b v. 15b)

And one more match : LZ(n) v. LZ(n-50)

#207 v. #157 --> 15-5 (40b v. 15b)

All games by resignation, no error, no duplicate game.

Average time was around 1.3 sec/move.

Below, the little hands point the networks #157,167,177, etc.

elo2.gif (27.31 KiB) Viewed 15961 times

If someone wants the games or the stats, I'll upload them.

[jlt]

Yes, I should have added the condition "if LZ(n) and LZ(n-10) are networks of the same size". Changing the network size introduces some discontinuity. When 20-block networks were introduced, results were disappointing, that's why the LeelaZero project shifted to 40 blocks rather quickly.

[Vargo]

You're right, 15b #157 was a turning point, and 20b #158 was weaker.

Another 20 game match (just finished, with the same parameters) :
LZ(n) v. LZ(n-49)

#207 v. #158 --> 19-1 (40b v. 20b)

Not very surprising, but still... it's hard to pretend that LZ doesn't progress anymore

[Vargo]

100 game match : LZ(today) v. LZ(1 year ago)

One year ago, the best LZ network was #90 (6x128)
2 minutes per game and side, LZ0.16, twogtp 1.5.0 no pondering, komi 7.5, gpu : 1x1080

Try to guess the result
NB. Because of the "-alternate" command, #207 is always named B, even though it was W 50 times.

90.jpg (331.19 KiB) Viewed 15827 times

[Vargo]

What's the effect of the number of visits on a given network ?
For example, what would be the score of LZ#207 --visits=801 v. LZ#207 --visits=1601 ?

I ran such a match yesterday (#207 with --visits=1, --visits=401, --visits=801, --visits=1601, --visits=3201) but... the results were inconclusive, more than half the games were duplicates

dup.gif (46.78 KiB) Viewed 15532 times

probably because #207 knows all the tricks of #207

______________________________________________________________________________

Anyway, there's a new network, #208.
20 game matches : #208 with various visits counts, and -m 40
-m 40 is used to have a bit more randomness in the first 40 moves, and so, avoid duplicate games.

Code: Select all

 gogui-twogtp -black "C:\PATH TO LZ\leelaz.exe --gtp --weights=C:\PATH TO NETWORKS\208.gz --noponder -m 40 -v yyyy" -white "C:\PATH TO LZ\leelaz.exe --gtp --weights=C:\PATH TO NETWORKS\208.gz --noponder -m 40 -v zzzz" -games 20 -sgffile XXX -auto -komi 7.5 -alternate

twogtp 1.5.0, LZ0.16, gpu:1x1080
no duplicate game, no error.

time/move seems to scale linearly :
-v 1 : ~0 sec/move
-v 401 : ~0.8
-v 801 : ~1.5
-v 1601 : ~3
-v 3201 : ~5 to 6


Results :

208.gif (9.4 KiB) Viewed 15532 times

If someone wants all the stats (times, lengthes, etc) , I'll upload them.


All the games :
The smallest number of visits is always B in the even numbered games (and W in the odd ones)
for example, 208_401_801-17 is game number 17 between #208 with 400 visits and # 208 with 800 visits. 400 visits is W

games.zip

(143.93 KiB) Downloaded 699 times

[maf]

Did a quick test using LZ207, p100 vs p1000, got 0:20. Nothing surprising, just fyi.

[And]

several matches 25x25, nets received by the program https://drive.google.com/open?id=1bgkVB ... oXHUdDuqt7,
https://github.com/leela-zero/leela-zero/issues/2240, 10sec/move, cpuonly, gogui-twogtp:
LM 192x15 GX89(25x25) - LZ 40x256 #205(25x25) 25:15
LZ 192x15 f438268e(25x25) - LZ 40x256 #205(25x25) 18:22
elf v2 256x20(25x25) - LZ 40x256 #205(25x25) 17:23, black elf all parties (11) won because of the ladder
converted minigo(25x25) 000930-goliath and 000990-cormorant do not work in gogui and sabaki.
Can someone with a powerful gpu make a couple of matches?