It is currently Fri Apr 19, 2024 6:01 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 14 posts ] 
Author Message
Offline
 Post subject: Elf v2 @ 800 playouts
Post #1 Posted: Thu Jan 30, 2020 6:02 am 
Dies in gote

Posts: 62
Liked others: 8
Was liked: 4
What would the strength of the Elf-v2 weights in Leela-Zero-0.17 with 800 playouts be?

If I would estimate it to be about a 10 dan amateur would that be to high or to low?

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #2 Posted: Thu Jan 30, 2020 6:16 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
Perhaps you could look at CGOS and find a bot/rating that you found to be equivalent to a 7d/8d/9d amateur and then compare to the ELFv2 p800 bot
http://www.yss-aya.com/cgos/19x19/cross ... _p800.html

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #3 Posted: Thu Jan 30, 2020 6:38 am 
Dies in gote

Posts: 62
Liked others: 8
Was liked: 4
Yakago wrote:
Perhaps you could look at CGOS and find a bot/rating that you found to be equivalent to a 7d/8d/9d amateur and then compare to the ELFv2 p800 bot
http://www.yss-aya.com/cgos/19x19/cross ... _p800.html

Thanks, but the problem is to estimate the strength of any bot :-)
As the ELFv2 p800 bot is fairly well known I found that to be a good starting point.
I will be happy to hear what every one think of the ELFv2 p800 strength.

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #4 Posted: Thu Jan 30, 2020 6:00 pm 
Dies with sente

Posts: 101
Liked others: 2
Was liked: 16
Rank: KGS 2 D
Well I never heard any amateur 10 dan rank. We think we can safely put it into the professional rank range. However, it may be hard to invite any professionals to do any match with this bot. Just a wild guess, 1600p maybe around 9p.

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #5 Posted: Fri Jan 31, 2020 12:58 am 
Dies in gote

Posts: 62
Liked others: 8
Was liked: 4
splee99 wrote:
Well I never heard any amateur 10 dan rank. We think we can safely put it into the professional rank range. However, it may be hard to invite any professionals to do any match with this bot. Just a wild guess, 1600p maybe around 9p.

Thanks very much for your answer!
And how much weaker would you place the 800p instance?

And no, amateur usually turn pro before they reach 10 dan :-)
But the pro scale is a whole different thing.

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #6 Posted: Fri Jan 31, 2020 3:17 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
I've had the opportunity to play KataGo 1.2 20 block with around 1600playouts (I manually placed stones on the board) against 2 strong amateurs at the level of, or slightly above european pros. The games were played without time constraints on the humans, although it was a casual setting so they didn't sit and think for hours.

Those games were very one-sided and both players lost by +30 pts.

Elfv2 is about the same as KataGo 1.2 at equal playouts, so Elf v2 at 800 playouts is probably a bit weaker.

Some tests suggests that doubling/halving the playouts is about 100 elo difference.

I don't know on which scale your '10d amateur' is to be placed. Fox/Tygem 10d? European 10d? Aga 10d? Korean 10d? There are many possibilities

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #7 Posted: Fri Jan 31, 2020 3:57 am 
Dies in gote

Posts: 62
Liked others: 8
Was liked: 4
Yakago wrote:
I don't know on which scale your '10d amateur' is to be placed. Fox/Tygem 10d? European 10d? Aga 10d? Korean 10d? There are many possibilities
Lets say European 10 dan.

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #8 Posted: Fri Jan 31, 2020 7:34 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
10d European would imply a player that wins 90%+ against the players I mentioned (I think)

Yeah that's probably not too far off for Elfv2 p800. maybe stronger than 10d.

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #9 Posted: Fri Jan 31, 2020 8:30 am 
Dies with sente

Posts: 113
Liked others: 11
Was liked: 27
Rank: 1d
Universal go server handle: iopq
Yakago wrote:
10d European would imply a player that wins 90%+ against the players I mentioned (I think)

Yeah that's probably not too far off for Elfv2 p800. maybe stronger than 10d.

It means wins vs. 8d at 2 stones handicap.

Which ELF easily does even on p800. But it would probably deteriorate at 3+ handicap

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #10 Posted: Fri Jan 31, 2020 12:10 pm 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
Yes - and because ELF is not particularly good at handicap compared to its strength, I wanted to avoid that comparison here.

I would like to compare in terms of ELO rather than amount of handicap stones I suppose, even though that becomes weird if someone wants to compare to perfect play ;)

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #11 Posted: Fri Jan 31, 2020 12:50 pm 
Lives in gote

Posts: 445
Liked others: 0
Was liked: 37
iopq wrote:
It means wins vs. 8d at 2 stones handicap.

Wins half at 2 EXTRA stones handicap (H3 giving komi).

Quote:
Which ELF easily does even on p800.

I doubt these older bots can do meaningful H3 games. And you cannot reliably measure stone scale without handi games (Elo is not enough for this).

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #12 Posted: Fri Jan 31, 2020 1:03 pm 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
Quote:
And you cannot reliably measure stone scale without handi games (Elo is not enough for this).


Agreed. But it is not a given that we are measuring 'dan levels' in that way.

Might as well be Elo, since that's for instance how EGF rating works, and European amateur dan level was specifically mentioned

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #13 Posted: Fri Jan 31, 2020 3:37 pm 
Lives in gote

Posts: 445
Liked others: 0
Was liked: 37
I think EGF Elo based ranks just try to approximate the actual stone differences, also anchored by handi games. Those magic constants would not be determinable otherwise (at least I cannot imagine "dan levels" without stone scale).

Top
 Profile  
 
Offline
 Post subject: Re: Elf v2 @ 800 playouts
Post #14 Posted: Fri Jan 31, 2020 9:43 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
40 game test at time parity : Elfv2 @p800 v. Katago 1.3.1 (20b s1420_d3509)
Katago wins 34-6 (85% , no error, no duplicate game, both program always agree on the score, Katago always appear as B, because of the command -alternate)
According to this site, winning 15% would put Elfv2@p800 ~300Elo behind Katago (or not ? :) )
Attachment:
k1.jpg
k1.jpg [ 147.08 KiB | Viewed 5906 times ]
Attachment:
k2.jpg
k2.jpg [ 349.28 KiB | Viewed 5906 times ]

PS. I had forgotten to change back the very high resign threshold for Katago, that's why its lost games are 300+ moves.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group