Life In 19x19
http://www.lifein19x19.com/

KataGo : How many stones ?
http://www.lifein19x19.com/viewtopic.php?f=18&t=17328		 Page 1 of 1
Author:  Vargo [ Wed Mar 18, 2020 10:17 am ]
Post subject:  KataGo : How many stones ?
How many stones could Katago give to a pro ?

There is a short but very interesting thread about KG vs top pros here : HERE
The first (?) LZ network with a pro level was LZ#125 (8000 visits) HERE

I've tried handicap games between LZ#125 and KG. KG wins with many stones, but it may be unfair.
At the beginning of these handicap games, I think LZ sees itself so far ahead, that it plays too conservatively.
Maybe, it would be better to have KG-KG handicap games. But which settings would give KG the same strength as LZ#125 (8000 visits) ?
Kata s297 9d resign threshold 90% 1 thread 16 playouts is 9d , according to @go4thewin.

After some tests :
KG 16p wins ~32% of its games against #125 (8k visits)
KG 50p wins ~75% of its games against #125 (8k visits)
KG 35p wins ~48% of its games against #125 (8k visits)


so, KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 may be around the same level as LZ#125,8kv


KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=1600 (W)
v.
KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H3, komi 0.5
W wins 24-11


KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=8000 (W)
v.
KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H5, komi 0.5 (2 gpus)
W wins 6-4

Unscientific, but fun !

Details (H5 games) :
Attachment:
k1.jpg
k1.jpg [ 464.72 KiB | Viewed 5360 times ]
H5 games :
Attachment:
KG35p_KG8000v.zip [9.08 KiB]
Downloaded 444 times
Author:  lightvector [ Wed Mar 18, 2020 6:19 pm ]
Post subject:  Re: KataGo : How many stones ?
Upcoming in next release is a change that may improve how hard it is to beat KataGo on low playouts when it *receives* handicap stones. The gist of the change is that when receiving handicap stones, Kata will assume that the opponent must be stronger, and use what the neural net has already internally learned from its handicap game training to play more conservatively given that the opponent is believed stronger.

This is exactly analogous to how when giving handicap stones, Kata will play more aggressively to try to keep the game complex in a way that a weaker opponent might find more challenging. The release will simply enable the other side of this behavior for when receiving stones.

Given that Kata is not yet doing the conservative-play side of this behavior, but that human pros are likely to do so to some degree when receiving stones (assuming they are playing to win), probably tests like this overestimate the plausible handicap. And they still might, even after this change - because as I've mentioned before, Kata's modeling of conservativeness/aggression in handicap games is still pretty basic.
Author:  Jæja [ Thu Mar 19, 2020 3:34 am ]
Post subject:  Re: KataGo : How many stones ?
@lightvector: how well is KataGo able to deal with large reverse komi in the current and new situation? For example, 25.5 reverse komi to replace a 5-stone handicap as described in this table.

Reverse komi is a really nice alternative for handicap games and games turn out way different. If this conservative style were an option, instead of inferred from the fact that a handicap games is being played, then this would be applicable to reverse komi also, I'd reckon. What do you think about this?
Author:  Vargo [ Thu Mar 19, 2020 5:05 am ]
Post subject:  Re: KataGo : How many stones ?
lightvector wrote:
Upcoming in next release is a change that may improve how hard it is to beat KataGo on low playouts when it *receives* handicap stones.
On Discord, you uploaded a "katago.exe" that should play better when receiving stones, if I understood correctly. And you wrote :
" people should feel free to play around more. kata one-playout or low playouts may become a little harder to defeat when receiving stones now.
I'd also be curious about exactly how the the play is different - i.e. what sorts of moves actually change, that are considered "safer" against a strong opponent. I haven't had time to take a look at that in much detail.
(enjoy!)"

I do enjoy ! Thank you very much !

I reran the exact same handicap tests, with the same commands, only with the "new" katago.exe as black.

The difference is stunning

KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=1600 (W)
v.
KG 1.3.3 (discord version, 2020.03.19) g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H3, komi 0.5
B wins 28-7


KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=8000 (W)
v.
KG 1.3.3 (discord version) g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H5, komi 0.5 (2 gpus)
B wins 10-0


So, I tried a more reasonable H3 game
KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=8000 (W)
v.
KG 1.3.3 (discord version) g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H3, komi 0.5 (2 gpus)
B wins 6-4

PS. In the last test, two games were B+0.5, and in the config file, I have whiteHandicapBonus = 0 , score may have changed if it was whiteHandicapBonus = N (?)

Details for the H3 games
Attachment:
k1.jpg
k1.jpg [ 461.74 KiB | Viewed 5208 times ]
The games :
Attachment:
KGhandi35p_KG8000vh3.zip [10.33 KiB]
Downloaded 443 times
Author:  Bill Spight [ Thu Mar 19, 2020 6:35 am ]
Post subject:  Re: KataGo : How many stones ?
Jæja wrote:
@lightvector: how well is KataGo able to deal with large reverse komi in the current and new situation? For example, 25.5 reverse komi to replace a 5-stone handicap as described in this table.


The table is way off. Even in the old days professionals thought that a handicap stone was worth 10 pts. So for the equivalent of a five stone game they would give 50 pts. reverse komi. We now know that that is too low.

In 1976 I did some statistics on pro-pro handicap games and found a value of about 13¾ pts. per stone. OC, that value cannot hold for reverse komi for high handicaps, but the relationship is approximately linear through 9 stones, which was the limit for pro-pro handicap games. The first stone should be worth half that, or approximately 7 pts. And, as far as we can now tell, that seems to be the case. So proper reverse komi instead of giving 5 stones should be around 60 pts., not 25.
Page 1 of 1 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/