It is currently Thu Mar 28, 2024 4:29 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7 ... 21  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #61 Posted: Sun Jul 08, 2018 3:13 pm 
Dies in gote

Posts: 63
Liked others: 0
Was liked: 3
[admin]
Off-topic link removed.
-JB
[/admin]

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #62 Posted: Mon Jul 09, 2018 2:48 pm 
Lives with ko

Posts: 136
Liked others: 47
Was liked: 21
Rank: KGS 6 dan
jokkebk wrote:
https://www.gokgs.com/gameArchives.jsp?user=petgo3

Hint: KGS archives with color highlight of wins/losses of given user with my Tampermonkey (Chrome plugin) script: http://joonaspihlajamaa.com/data/kgs_graphs.user.js
This script is just excellent, thank you!

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #63 Posted: Wed Jul 11, 2018 12:29 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
I've run a 100 games twogtp match between regular LZ 0.15, with network #155 and LZ Variable Komi V0.3, with network #155.
Even games, with komi 7.5 , --noponder --visits=3201 -r 10 for both.

Regular LZ wins 52-48 (50 games as W, 50 as B)

It looks like LZVK is as strong as Regular LZ on even games (I should run more games to confirm this...)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #64 Posted: Wed Jul 18, 2018 7:24 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Four 10-games matches between LZ variable komi V0.31 (network#156) and gnugo 3.8 at various handicaps (I used twogtp V1.4.10)

komi 7.5, playouts=1000 for LZvk
H6 : LZvk wins 10-0

komi 7.5, playouts=2000 for LZvk
H8 : draw 5-5

komi 7.5, playouts=2000 for LZvk
H9 : gnugo wins 8-2

komi 7.5, playouts=4000 for LZvk
H9 : gnugo wins 9-1 (a bit curious, compared to 8-2 with less playouts)


Two 10-games matches between LZ variable komi(#156) and Leela 0.11.0 OpenCL

komi 7.5, playouts = 1000 for both
H5 : Leela OpenCl wins 6-4

komi 7.5, playouts = 3000 for both
H5 : Leela OpenCl wins 7-3

I didn't use -r 0 , so, in some games, LZvk resigns rather early.
If someone wants to see the games, I can upload them.


This post by Vargo was liked by: Friday9i
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #65 Posted: Wed Jul 18, 2018 9:35 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
A while ago, I ran H6, H7 and H8 mini matches between LZvk(#153) and LZ#80.
LZvk won at H6 and H7, but jokkebk's remark is true...

Quote:
Vargo wrote:
Thanks to alreadydone, LZ can now handle high handicap (HERE)
Playing (and winning) H7 and H8 games against 4 Dan... Wow !
Quote:
jokkebk wrote :
Winning against "4d" LZ #80 might not be such an achievement, considering the LZ network is probably very bad playing as black in high handicap.

So, what about a rematch, but with LZvk for both #157 and #80 (they're supposed to be 10.6D and 4D)

visits=3201, komi 7.5, noponder for both, 10 games match with twogtp 1.4.10, at a more reasonable H5.

LZvk with network #157 wins (only) 6-4
If someone wants the games, I'll upload them.

twogtp v1.4.10 with handicap seems to work well with LZvk , with regular Leela OpenCl and with gnugo, but not with gtp4zen, I don't know why.


This post by Vargo was liked by: Friday9i
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #66 Posted: Tue Jul 24, 2018 2:05 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
In his last witty post (the "#xxx Ten Game Match Report" series at reddit, cbaduk), HERE, Dontbteme writes :
Quote:
Actually, maybe LZ#157 could already give LeelaElf a run for its money with Time Parity... Too bad I didn’t think of setting a match between them before, and REALLY meant it when I said I’d go to bed early.)

He's right, maybe it would show LZ's progress.

So, here are three 20 games matches at 5 minutes (per game and per side), at 10 minutes, and at 15 minutes, no pondering, komi 7.5.
The matches are run with twogtp 1.4.10 (win10, i7, 1x1080)

LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)

So, we're not there yet, but it's coming nearer ;-)


Edit : I just saw that Dontbteme has posted an Elf vs #157 match , but with very different result... HERE


Attachments:
File comment: the 60 games
157vElf.rar [47.27 KiB]
Downloaded 447 times

This post by Vargo was liked by 2 people: Uberdude, Waylon
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #67 Posted: Tue Jul 24, 2018 3:14 pm 
Dies in gote

Posts: 44
Liked others: 2
Was liked: 14
Rank: EGF 1 kyu
KGS: finity
Vargo wrote:
LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)


Very nice! It means one can likely get already quite on par analysis with Lizzie on Leela Zero #157 as LZElf. If there's still 2-3 promotions with 192x15b net, it might be 50-50 already on time parity, but my bets would be that it'll take too long and Leela will switch to larger net before that.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #68 Posted: Mon Jul 30, 2018 11:12 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
There's been some controversy about the passage to 20 block networks.

Three 10 games matches between LZ15 with #157 (192x15) and LZ15 #159 (256x20)

twogtp V 1.4.10 , komi 7.5, no pondering, 5 minutes per side and per game , 5 games as B, 5 games as W. (id with 10 min, and with 15 min.)

All games won or lost by resignation, all 30 games between 125 and 347 moves. Time used is consistently around two thirds of the alloted time.

LZ#157 vs LZ#159 :
at 05 minutes , LZ157 wins 6-4 (4 wins as B, and 2 as W)
at 10 minutes , LZ157 wins 6-4 (3 wins as B, and 3 as W)
at 15 minutes , LZ157 wins 7-3 (5 wins as B, and 2 as W)


Seems like #157 is still stronger at equal time setting...

Edit : hardware : 1xGTX1080, i7, win 10

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #69 Posted: Mon Jul 30, 2018 11:44 pm 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Roughly how many playouts per move are the 15- and 20-block networks getting at these settings?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #70 Posted: Tue Jul 31, 2018 1:20 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Same kind of results with #160 at 5' per side and per game

LZ15#157 vs LZ15#160 :
at 5 minutes , LZ157 wins 7-3 (4 wins as B, and 3 as W)

@Uberdude
I don't know. At the end of an automatically played n-game match, twogtp gives reports like this
Attachment:
rep.jpg
rep.jpg [ 32.76 KiB | Viewed 11354 times ]

with : winner, number of duplicate games (hopefully none) , length of game, time for B and for W, number of errors, but nothing about playouts.
I'll run some tests, and I'll let you know.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #71 Posted: Tue Jul 31, 2018 2:47 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
@Uberdude :
#160 at 1600 visits takes a little more time than #157 at 3200 visits, and the matches take about the same time as the 5' games.
cf this :
Attachment:
rep2.jpg
rep2.jpg [ 179.06 KiB | Viewed 11367 times ]


@Marcel Grünauer
I've just downloaded the new ELF network, looking forward to trying it, thanks !


This post by Vargo was liked by: Uberdude
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #72 Posted: Tue Jul 31, 2018 2:53 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Marcel Grünauer wrote:
I'm not very knowledgable about how to interpret the results, but did note that with the 20-block networks there is a wild fluctuation of win rates. For example, yesterday and today some 20-block candidates only had a 30% win rate against the current #160.

So maybe getting a high enough percentage to be promoted is more of a fluke than a steady progression. With the 15-block networks and earlier, there were lots of candidates just below the promotion threshold of 55%.


Yes, I noticed this too. There was a test match of #159 (the 2nd 20-block against #157 (best 15-block) and it got 50% win vs 45% for #158. I've not yet seen a test match for the new #160 vs #157. Given all these 20-30% candidates there is more danger you are not progressing but just jumping around about the same level networks that randomly sometimes do well; so these test matches against #157 as a more external benchmark are important. In fact during normal 15-block training I think doing some occasionally e.g. #157 vs #147 would be a good idea to see how inflated the incremental self-improvement Elo is (based on the incremental Elo differences from the promotions 147->148->149 etc it went from 11401 -> 11806 which predicts #157 would beat #147 91% of the time, but I would bet it would be quite a lot lower than that in reality).


This post by Uberdude was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #73 Posted: Tue Jul 31, 2018 5:45 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
20 games match between ELF1 and ELF2 at 1600 visits.
(twogtp 1.4.10, komi 7.5, noponder, all games by resignation)

ELF2 wins 14:6 (70% , 9 wins as W, 5 wins as B)


If someone wants the games and/or the reports...

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #74 Posted: Tue Jul 31, 2018 6:12 am 
Gosei

Posts: 1590
Liked others: 886
Was liked: 527
Rank: AGA 3k Fox 3d
GD Posts: 61
KGS: dfan
There's a test match going on right now between the ELF1 and ELF2 weights, imported into Leela Zero. As I write this ELF2 is up 102 to 51.

Edit: the final result was 265 to 145 (64.6%).


Last edited by dfan on Tue Jul 31, 2018 9:08 am, edited 1 time in total.

This post by dfan was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #75 Posted: Tue Jul 31, 2018 6:42 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Instead of win ratios amongst all these super-strong bots, I'd kind of like "average number of moves a strong/weak Tygem/Fox 9d lasts until only 10% winrate" :D

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #76 Posted: Tue Jul 31, 2018 8:16 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Last one for today ;-) :

20-game match LZ15#160 v LZ15_ELF2 at time parity.

5 minutes per game (1xGTX1080), noponder, komi 7.5

ELF2 wins 18:2 (90% , 10 wins as W, 8 wins as B)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #77 Posted: Tue Jul 31, 2018 10:51 am 
Lives with ko
User avatar

Posts: 205
Liked others: 49
Was liked: 36
Rank: EGF 2k
KGS: MKyle
Great stuff. Thanks for sharing.

Almost off topic, but it seems like you have a method that could be useful for generating large numbers of Bot vs Bot games. I'd be interested to play with that. Are you using a freely available tool? Or something you've developed yourself?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #78 Posted: Tue Jul 31, 2018 9:05 pm 
Lives in gote

Posts: 388
Liked others: 416
Was liked: 198
I saw today two games on WBaduk server, between Ichiriki Ryo 8p (one of Japan's top pros) and LeelaZero. Both were played on even.

LeelaZero won both (one playing black, one white).

Does anyone know what was the event?

_________________
Sorin - 361points.com

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #79 Posted: Tue Jul 31, 2018 9:24 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
@MikeKyle
It's a freely available tool : gogui-twogtp (there's V1.4.9 and V1.4.10)
It automatically plays n games between two programs, then saves the games and gives you a detailed report for each game (winner, score, number of moves, time spent, etc)
It works well with LZ, LZ variable komi, gtp4zen, Ray, and gnugo (and probably with others I haven't tried) but NOT with AQ 2.1.1

V1.4.9 is easy to install under Windows (there's an installer) , it works well for even games, but I think there's a scoring problem with handicap games.
You can download the installer here : https://sourceforge.net/projects/gogui/ ... gui/1.4.9/

V1.4.10 is less easy to use at first (because there's no win installer), but it handles well handicap games.

Once installed, you write a .txt file containing the parameters of the match you want to set up (which programs, nb of playouts, time, pondering, nb of games to play, etc.) and you rename your .txt file as .bat

And you just launch it.

For example,

A file named xxx.bat, and containing :


gogui-twogtp -black "C:\.....path to LZ.....\leelaz.exe --gtp --weights=C:\...path to your weight...\160.txt --visits=1000 --noponder" -white "C:\.....path to LZ.....\leelaz.exe --gtp --weights=C:\...path to your weight...\159.gz --playouts=100 --noponder" -games 30 -sgffile filenameyouwant -auto


would launch a 30 games match between
B : LZ with weights 160.txt (assuming it's the name these weights have on your computer) at 1000 visits and without pondering
and
W : LZ with weights 159.gz (assuming...) at 100 playouts and without pondering

the final report and the games would be named :

filenameyouwant.dat
filenameyouwant01.sgf
filenameyouwant02.sgf
etc...


EDIT :

For gogui-twogtp V1.4.9, The xxx.bat file must be in the same (install) directory as the file "gogui-twogtp.exe"

For both V1.4.9 and V1.4.10, instead of --playouts=xxx or --visits=yyy , you can set the time
-time 5 is the same as -time 5m and means 5min. for the whole game (per side)
-time 1200s means 1200sec
-time 120m+30s/1 means 2hours, plus one 30sec. byoyomi
-time 1s+5s/1 means 5sec. per move (total time must be >0)

For V1.4.10, as there's no installer, the .bat file must be in the installation directory, and must begin with

java.exe -jar lib\gogui-twogtp.jar -black "C:\etc.etc

for example :

If you want to set up an H7 10-games match between LZ and gnugo, you must use twogtp V1.4.10, and you must have one of the lZ variable komi versions , choose for example the one called "dynamic komi for handicap games"

and your .bat file should be something like :

java.exe -jar lib\gogui-twogtp.jar -black "C:\...path to gnugo ...\gnugo.exe --mode gtp" -white "C:\...path to LZvariablekomi ...\leelazVK.exe --gtp --weights=C:\...path to your weights...\40b_155_328k.gz --visits=2000 --noponder" -handicap 7 -games 10 -sgffile whatuwant -auto

for an H7 game between gnugo and LZ_variable_komi


Last edited by Vargo on Mon Aug 06, 2018 12:18 pm, edited 3 times in total.

This post by Vargo was liked by 3 people: ez4u, Uberdude, Waylon
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #80 Posted: Fri Aug 03, 2018 7:36 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
sorin wrote:
I saw today two games on WBaduk server, between Ichiriki Ryo 8p (one of Japan's top pros) and LeelaZero. Both were played on even.

LeelaZero won both (one playing black, one white).

Does anyone know what was the event?


I've seen a bunch more, and some by PhoenixGo too. Mostly against young low dan pros. So my guess is it is similar to the online training series against DeepZen from last year. I've not seen a human win. Also I believe the Japan server on WBaduk is a mirror/clone/relay/something of the Japanese Yugen No Ma server. Currently playing is username "aikiller" with 1p Japanese flag, let's see if he or she can live up to that!

Update: Well, aikiller is recorded as winning by 0.5 points, but PhoenixGo played 1 move inside its territory unnecessarily as the last move, presumably because it assumes 7.5 komi but the server rules were 6.5. So not a moral victory even if it technically was one.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7 ... 21  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group