Life In 19x19
http://www.lifein19x19.com/

LZ's progression
http://www.lifein19x19.com/viewtopic.php?f=18&t=15718
Page 4 of 21

Author:  hydrogenpi7 [ Sun Jul 08, 2018 3:13 pm ]
Post subject:  Re: LZ's progression

[admin]
Off-topic link removed.
-JB
[/admin]

Author:  johnsmith [ Mon Jul 09, 2018 2:48 pm ]
Post subject:  Re: LZ's progression

jokkebk wrote:
https://www.gokgs.com/gameArchives.jsp?user=petgo3

Hint: KGS archives with color highlight of wins/losses of given user with my Tampermonkey (Chrome plugin) script: http://joonaspihlajamaa.com/data/kgs_graphs.user.js
This script is just excellent, thank you!

Author:  Vargo [ Wed Jul 11, 2018 12:29 am ]
Post subject:  Re: LZ's progression

I've run a 100 games twogtp match between regular LZ 0.15, with network #155 and LZ Variable Komi V0.3, with network #155.
Even games, with komi 7.5 , --noponder --visits=3201 -r 10 for both.

Regular LZ wins 52-48 (50 games as W, 50 as B)

It looks like LZVK is as strong as Regular LZ on even games (I should run more games to confirm this...)

Author:  Vargo [ Wed Jul 18, 2018 7:24 am ]
Post subject:  Re: LZ's progression

Four 10-games matches between LZ variable komi V0.31 (network#156) and gnugo 3.8 at various handicaps (I used twogtp V1.4.10)

komi 7.5, playouts=1000 for LZvk
H6 : LZvk wins 10-0

komi 7.5, playouts=2000 for LZvk
H8 : draw 5-5

komi 7.5, playouts=2000 for LZvk
H9 : gnugo wins 8-2

komi 7.5, playouts=4000 for LZvk
H9 : gnugo wins 9-1 (a bit curious, compared to 8-2 with less playouts)


Two 10-games matches between LZ variable komi(#156) and Leela 0.11.0 OpenCL

komi 7.5, playouts = 1000 for both
H5 : Leela OpenCl wins 6-4

komi 7.5, playouts = 3000 for both
H5 : Leela OpenCl wins 7-3

I didn't use -r 0 , so, in some games, LZvk resigns rather early.
If someone wants to see the games, I can upload them.

Author:  Vargo [ Wed Jul 18, 2018 9:35 pm ]
Post subject:  Re: LZ's progression

A while ago, I ran H6, H7 and H8 mini matches between LZvk(#153) and LZ#80.
LZvk won at H6 and H7, but jokkebk's remark is true...

Quote:
Vargo wrote:
Thanks to alreadydone, LZ can now handle high handicap (HERE)
Playing (and winning) H7 and H8 games against 4 Dan... Wow !
Quote:
jokkebk wrote :
Winning against "4d" LZ #80 might not be such an achievement, considering the LZ network is probably very bad playing as black in high handicap.

So, what about a rematch, but with LZvk for both #157 and #80 (they're supposed to be 10.6D and 4D)

visits=3201, komi 7.5, noponder for both, 10 games match with twogtp 1.4.10, at a more reasonable H5.

LZvk with network #157 wins (only) 6-4
If someone wants the games, I'll upload them.

twogtp v1.4.10 with handicap seems to work well with LZvk , with regular Leela OpenCl and with gnugo, but not with gtp4zen, I don't know why.

Author:  Vargo [ Tue Jul 24, 2018 2:05 am ]
Post subject:  Re: LZ's progression

In his last witty post (the "#xxx Ten Game Match Report" series at reddit, cbaduk), HERE, Dontbteme writes :
Quote:
Actually, maybe LZ#157 could already give LeelaElf a run for its money with Time Parity... Too bad I didn’t think of setting a match between them before, and REALLY meant it when I said I’d go to bed early.)

He's right, maybe it would show LZ's progress.

So, here are three 20 games matches at 5 minutes (per game and per side), at 10 minutes, and at 15 minutes, no pondering, komi 7.5.
The matches are run with twogtp 1.4.10 (win10, i7, 1x1080)

LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)

So, we're not there yet, but it's coming nearer ;-)


Edit : I just saw that Dontbteme has posted an Elf vs #157 match , but with very different result... HERE

Attachments:
File comment: the 60 games
157vElf.rar [47.27 KiB]
Downloaded 464 times

Author:  jokkebk [ Tue Jul 24, 2018 3:14 pm ]
Post subject:  Re: LZ's progression

Vargo wrote:
LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)


Very nice! It means one can likely get already quite on par analysis with Lizzie on Leela Zero #157 as LZElf. If there's still 2-3 promotions with 192x15b net, it might be 50-50 already on time parity, but my bets would be that it'll take too long and Leela will switch to larger net before that.

Author:  Vargo [ Mon Jul 30, 2018 11:12 pm ]
Post subject:  Re: LZ's progression

There's been some controversy about the passage to 20 block networks.

Three 10 games matches between LZ15 with #157 (192x15) and LZ15 #159 (256x20)

twogtp V 1.4.10 , komi 7.5, no pondering, 5 minutes per side and per game , 5 games as B, 5 games as W. (id with 10 min, and with 15 min.)

All games won or lost by resignation, all 30 games between 125 and 347 moves. Time used is consistently around two thirds of the alloted time.

LZ#157 vs LZ#159 :
at 05 minutes , LZ157 wins 6-4 (4 wins as B, and 2 as W)
at 10 minutes , LZ157 wins 6-4 (3 wins as B, and 3 as W)
at 15 minutes , LZ157 wins 7-3 (5 wins as B, and 2 as W)


Seems like #157 is still stronger at equal time setting...

Edit : hardware : 1xGTX1080, i7, win 10

Author:  Uberdude [ Mon Jul 30, 2018 11:44 pm ]
Post subject:  Re: LZ's progression

Roughly how many playouts per move are the 15- and 20-block networks getting at these settings?

Author:  Vargo [ Tue Jul 31, 2018 1:20 am ]
Post subject:  Re: LZ's progression

Same kind of results with #160 at 5' per side and per game

LZ15#157 vs LZ15#160 :
at 5 minutes , LZ157 wins 7-3 (4 wins as B, and 3 as W)

@Uberdude
I don't know. At the end of an automatically played n-game match, twogtp gives reports like this
Attachment:
rep.jpg
rep.jpg [ 32.76 KiB | Viewed 11700 times ]

with : winner, number of duplicate games (hopefully none) , length of game, time for B and for W, number of errors, but nothing about playouts.
I'll run some tests, and I'll let you know.

Author:  Vargo [ Tue Jul 31, 2018 2:47 am ]
Post subject:  Re: LZ's progression

@Uberdude :
#160 at 1600 visits takes a little more time than #157 at 3200 visits, and the matches take about the same time as the 5' games.
cf this :
Attachment:
rep2.jpg
rep2.jpg [ 179.06 KiB | Viewed 11713 times ]


@Marcel Grünauer
I've just downloaded the new ELF network, looking forward to trying it, thanks !

Author:  Uberdude [ Tue Jul 31, 2018 2:53 am ]
Post subject:  Re: LZ's progression

Marcel Grünauer wrote:
I'm not very knowledgable about how to interpret the results, but did note that with the 20-block networks there is a wild fluctuation of win rates. For example, yesterday and today some 20-block candidates only had a 30% win rate against the current #160.

So maybe getting a high enough percentage to be promoted is more of a fluke than a steady progression. With the 15-block networks and earlier, there were lots of candidates just below the promotion threshold of 55%.


Yes, I noticed this too. There was a test match of #159 (the 2nd 20-block against #157 (best 15-block) and it got 50% win vs 45% for #158. I've not yet seen a test match for the new #160 vs #157. Given all these 20-30% candidates there is more danger you are not progressing but just jumping around about the same level networks that randomly sometimes do well; so these test matches against #157 as a more external benchmark are important. In fact during normal 15-block training I think doing some occasionally e.g. #157 vs #147 would be a good idea to see how inflated the incremental self-improvement Elo is (based on the incremental Elo differences from the promotions 147->148->149 etc it went from 11401 -> 11806 which predicts #157 would beat #147 91% of the time, but I would bet it would be quite a lot lower than that in reality).

Author:  Vargo [ Tue Jul 31, 2018 5:45 am ]
Post subject:  Re: LZ's progression

20 games match between ELF1 and ELF2 at 1600 visits.
(twogtp 1.4.10, komi 7.5, noponder, all games by resignation)

ELF2 wins 14:6 (70% , 9 wins as W, 5 wins as B)


If someone wants the games and/or the reports...

Author:  dfan [ Tue Jul 31, 2018 6:12 am ]
Post subject:  Re: LZ's progression

There's a test match going on right now between the ELF1 and ELF2 weights, imported into Leela Zero. As I write this ELF2 is up 102 to 51.

Edit: the final result was 265 to 145 (64.6%).

Author:  Uberdude [ Tue Jul 31, 2018 6:42 am ]
Post subject:  Re: LZ's progression

Instead of win ratios amongst all these super-strong bots, I'd kind of like "average number of moves a strong/weak Tygem/Fox 9d lasts until only 10% winrate" :D

Author:  Vargo [ Tue Jul 31, 2018 8:16 am ]
Post subject:  Re: LZ's progression

Last one for today ;-) :

20-game match LZ15#160 v LZ15_ELF2 at time parity.

5 minutes per game (1xGTX1080), noponder, komi 7.5

ELF2 wins 18:2 (90% , 10 wins as W, 8 wins as B)

Author:  MikeKyle [ Tue Jul 31, 2018 10:51 am ]
Post subject:  Re: LZ's progression

Great stuff. Thanks for sharing.

Almost off topic, but it seems like you have a method that could be useful for generating large numbers of Bot vs Bot games. I'd be interested to play with that. Are you using a freely available tool? Or something you've developed yourself?

Author:  sorin [ Tue Jul 31, 2018 9:05 pm ]
Post subject:  Re: LZ's progression

I saw today two games on WBaduk server, between Ichiriki Ryo 8p (one of Japan's top pros) and LeelaZero. Both were played on even.

LeelaZero won both (one playing black, one white).

Does anyone know what was the event?

Author:  Vargo [ Tue Jul 31, 2018 9:24 pm ]
Post subject:  Re: LZ's progression

@MikeKyle
It's a freely available tool : gogui-twogtp (there's V1.4.9 and V1.4.10)
It automatically plays n games between two programs, then saves the games and gives you a detailed report for each game (winner, score, number of moves, time spent, etc)
It works well with LZ, LZ variable komi, gtp4zen, Ray, and gnugo (and probably with others I haven't tried) but NOT with AQ 2.1.1

V1.4.9 is easy to install under Windows (there's an installer) , it works well for even games, but I think there's a scoring problem with handicap games.
You can download the installer here : https://sourceforge.net/projects/gogui/ ... gui/1.4.9/

V1.4.10 is less easy to use at first (because there's no win installer), but it handles well handicap games.

Once installed, you write a .txt file containing the parameters of the match you want to set up (which programs, nb of playouts, time, pondering, nb of games to play, etc.) and you rename your .txt file as .bat

And you just launch it.

For example,

A file named xxx.bat, and containing :


gogui-twogtp -black "C:\.....path to LZ.....\leelaz.exe --gtp --weights=C:\...path to your weight...\160.txt --visits=1000 --noponder" -white "C:\.....path to LZ.....\leelaz.exe --gtp --weights=C:\...path to your weight...\159.gz --playouts=100 --noponder" -games 30 -sgffile filenameyouwant -auto


would launch a 30 games match between
B : LZ with weights 160.txt (assuming it's the name these weights have on your computer) at 1000 visits and without pondering
and
W : LZ with weights 159.gz (assuming...) at 100 playouts and without pondering

the final report and the games would be named :

filenameyouwant.dat
filenameyouwant01.sgf
filenameyouwant02.sgf
etc...


EDIT :

For gogui-twogtp V1.4.9, The xxx.bat file must be in the same (install) directory as the file "gogui-twogtp.exe"

For both V1.4.9 and V1.4.10, instead of --playouts=xxx or --visits=yyy , you can set the time
-time 5 is the same as -time 5m and means 5min. for the whole game (per side)
-time 1200s means 1200sec
-time 120m+30s/1 means 2hours, plus one 30sec. byoyomi
-time 1s+5s/1 means 5sec. per move (total time must be >0)

For V1.4.10, as there's no installer, the .bat file must be in the installation directory, and must begin with

java.exe -jar lib\gogui-twogtp.jar -black "C:\etc.etc

for example :

If you want to set up an H7 10-games match between LZ and gnugo, you must use twogtp V1.4.10, and you must have one of the lZ variable komi versions , choose for example the one called "dynamic komi for handicap games"

and your .bat file should be something like :

java.exe -jar lib\gogui-twogtp.jar -black "C:\...path to gnugo ...\gnugo.exe --mode gtp" -white "C:\...path to LZvariablekomi ...\leelazVK.exe --gtp --weights=C:\...path to your weights...\40b_155_328k.gz --visits=2000 --noponder" -handicap 7 -games 10 -sgffile whatuwant -auto

for an H7 game between gnugo and LZ_variable_komi

Author:  Uberdude [ Fri Aug 03, 2018 7:36 am ]
Post subject:  Re: LZ's progression

sorin wrote:
I saw today two games on WBaduk server, between Ichiriki Ryo 8p (one of Japan's top pros) and LeelaZero. Both were played on even.

LeelaZero won both (one playing black, one white).

Does anyone know what was the event?


I've seen a bunch more, and some by PhoenixGo too. Mostly against young low dan pros. So my guess is it is similar to the online training series against DeepZen from last year. I've not seen a human win. Also I believe the Japan server on WBaduk is a mirror/clone/relay/something of the Japanese Yugen No Ma server. Currently playing is username "aikiller" with 1p Japanese flag, let's see if he or she can live up to that!

Update: Well, aikiller is recorded as winning by 0.5 points, but PhoenixGo played 1 move inside its territory unnecessarily as the last move, presumably because it assumes 7.5 komi but the server rules were 6.5. So not a moral victory even if it technically was one.

Page 4 of 21 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/