Life In 19x19

BadukAI (KG 10b, 10s/move) - CSDL 5d 8:2 (time parity)

akigo wrote:I made a new version (0.9) with the following changes:

1) When AI passes, a popup message is displayed shortly, which informs about the pass
2) KataGo now uses chinese rules

many many gratitude !!!

lightvector, akigo
you make the world a better place and people happier! very pleasant program!

And wrote:BadukAI (KG 10b, 10s/move) - CSDL 5d 8:2 (time parity)

I don't understand... With my smartphone (Exynos 990) , BadukAI KG 20b, 10s/move usually wins (as W) at H6 against CSDL 5d, H6 seems a fair handicap. KG 20b, 10s/move is several ranks stronger. CSDL 5d can't win 20% of its games in even games.
I just tried 12 such even games, and predictably they all went rapidly the same way : after 50-70 moves, CSDL's winrate was always around 0. (For example, BadukAI is B in the 4 screenshots) How is it possible ? Is the 10b network so much weaker then the 20b ?

I have the 6b net w/4 playouts (1 thread) stronger than gnugo (ogs 13k), the 10b policy network at around 3k, the 15b policy at 2-3 dan, the 20b policy at 4 dan, and the 40*256 bot policy at around 5 dan. The 10b at 9 payouts (1 thread) can beat the 15 bot policy. So it is a high kyu/low dan bot. The 20b is a high dan bot trained on 40b data. At 9 playouts, it is maybe 1 stone behind gtp4zen7 9dan. And what surprised me the most was that the 40b is the strongest at 5 sec/move on modern phones!

attached is komi 9 game with 10b policy (black) winning against pachi at default settings on my pentium 4405u. I cannot get pachi to play as black because of some bug, but the result of this game is typical between the two.

Vargo screenshots of two games, BadukAI (KG 10b, 10s/move) - CSDL 5d. sgf did not save

At equal playouts the 20b network is about 1350 Elo stronger than the 10b network (see KataGo's README). At time parity we have to consider that the 10b network is about 8 times faster (at least on my device) due to the smaller size, so that the 10b network will have 8 times the number of playouts. Since doubling the playouts gains around 200 Elo (see https://github.com/leela-zero/leela-zero/issues/2236), this will reduce the gap by around 600 Elo. But we still have a whopping 750 Elo advantage for 20b at time parity. And the difference between @Vargo's and @And's configuration will be even larger, since @Vargo uses a faster device. So the difference in scores against CS is very plausible.

BadukAI (KG 15b, 10 playouts) - CS Zero 6d 1:0 (BadukAI W +18.5)
BadukAI (KG 15b, 10 playouts) - CS Zero 7d 1:0 (BadukAI W +0.5)

BadukAI KG 15b, 10 playouts ~ 15 sec/move
Mobile Linpack 1.4 (https://play.google.com/store/apps/deta ... kbenchmark)
threads:2 max:9.33 mflop/sec, threads:1 max:7.78 mflop/sec
emulator nox

Vargo can you see the speed of your device mflop/sec?

CSDL 5d usually thinks for 10 seconds regardless of the device. launched on the emulator CS 5d (1 core) - CS 5d (4 cores) 3:7

@And

Hi, I use CSDL on my laptop (i7-9750H, 6 cores, 12 threads, Win 10)
At 5dan, one move takes around 0.1 or 0.2 sec. For example, below is a screenshot of a complete 205 moves game (CSDL5d v. CSDL5d).
Total time for the 205 moves was 29 sec (~0.14 sec/move) Curiously, on the Windows version of CSDL, the time shown is the total cumulated time, but in the Ipad version of CSDL, the time shown is the time for the current move only.
On my Ipad pro, at 5d, each move takes 5-10 sec, but the strength is the same as in the Win version. For the specifications of the Exynos 990 , see this site
I'll make some tests with KG10b and CSDL.

Two games : BadukAI_KG_10b_10s/move v. CSDL 5d

At H6, BadukAI_KG10b loses, H6 seems too much. At H4, BadukAI_KG10b wins The winrate graph for the H4 game (Mmmmmmmh...the board is rotated 180° )

@And

just for comparison I ran the Mobile Linpack 1.4 on my device (Snapdragon 855+):

1 Thread: max. 999 mflops/s
2 Threads: max. 1030 mflops/s

And from geekbench 5 scores I know that @Vargo's Exynos 990 is still a little faster.

snapdragon 625 1 thread >150 (1st reading), <174 when device first turns on.
on this device 40*256 net is strongest at any time setting. Just to add another data point.

2nd device is Likebook mars e ink ereader processor=?. 1 thread <68.5 . Here, 20b takes maybe 4 seconds for 1 thread 1 playout policy move.

akigo wrote:@And

just for comparison I ran the Mobile Linpack 1.4 on my device (Snapdragon 855+):

1 Thread: max. 999 mflops/s
2 Threads: max. 1030 mflops/s

And from geekbench 5 scores I know that @Vargo's Exynos 990 is still a little faster.

Vargo

1 performance difference is more than 100!
2 engines work differently ("Crazy Stone has made a huge step forward by combining Deep Neural Networks with Monte Carlo Tree Search." https://play.google.com/store/apps/deta ... oid.gocsdl)
how does the strength of their game change with performance?
CSDL settings - 5d, and if its strength does not depend much on the device (it should not depend!) then everything is in order!

Vargo wrote: ...
On my Ipad pro, at 5d, each move takes 5-10 sec, but the strength is the same as in the Win version...

3 just a few games
given this, no results will surprise me! are you still surprised?

the program hangs at the end of the game. I could not get out of this state. saved sgf, does not react to anything else. if necessary, there is a log. this happened for the first time, does not bother, I inform you just in case