Life In 19x19

Life in 19x19. Go, Weiqi, Baduk... Thats the life.
https://www.lifein19x19.com/

Engine Tournament

https://www.lifein19x19.com/viewtopic.php?t=13322

Page 15 of 27

Re: Engine Tournament

Posted: Wed May 23, 2018 7:17 am
by as0770
abcd_z wrote:
as0770 wrote:I didn't mix different weights or different versions of one engine
I... what? I never said you did.
as0770 wrote:But I don't see a reason to explain why I don't test all versions and networks.
I didn't say that, either. I feel like you're reading something different from what I'm writing.
I know what you said. Everything is in order. From the discussion one could get the impression I would use different weights with one participant. I just stated this clear.
abcd_z wrote:Wait, you mean that thing about Leela Zero? The reason I brought LZ up specifically was because, unlike all the other engines, LZ changes frequently, and it's easy for a reader to assume that the tests you're running on it are indicative of LZ's current strength.
No, the size of the network is specified. It is much easier to understand than the network names. Beside that when I publish the results the network will be quite up to date and one can also take a look at the date of a post to see how old it is.

Re: Engine Tournament

Posted: Thu May 24, 2018 2:37 am
by as0770
For a change some results :-)

Updates:
-Leela Zero 0.15 with the 192x15 network 11057dd from 2018.05.20
-ELF OpenGo v0 now also with Leela Zero 0.15.

So the results in the Champions League are all new, other Leagues are not affected.

ELF performed better with LZ 0.15, maybe there was a problem with the first LZ version that was able to use the ELF network.

LZ 192x15 now clearly outperformes Zen 7. Beside the new network there were changes in the time management.

Zen won it's only game because of a ladder against ELF. There were no results affected by the komi of 6.5.

Champions League:

Code: Select all

    1. ELF OpenGo v0 LZ 0.15           14/16
    2. Leela Zero 0.15 11057dd          9/16
    2. Zen 7                            1/16
League A:

Code: Select all

    1. Zen 7                           11/16
    2. ELF OpenGo v0 LZ 0.13x (CPU)    11/16
    3. AQ 2.1.1                         8/16
    4. Leela 0.11                       6/16
    5. Leela Zero 0.13 128x10 (CPU)     4/16
League B:

Code: Select all

    1. Leela Zero 0.13 128x10 (CPU)    22/24
    2. Leela 0.11.0                    19/24
    3. Rayon 4.6.0                     13/24
    4. Oakfoam 0.2.1 NG-06             12/24
    5. Leela Zero 128x6 (CPU)          12/24
    6. Hiratuka 10.37B (CPU)            4/24
    7. DreamGo 0.5.0                    2/24
League C:

Code: Select all

    1. DreamGo 0.5.0                   18/24
    2. DarkForrest MCTS 1.0            16/24
    3. Pachi 12                        16/24
    4. Ray 9.0.1                       11/24
    5. DarkGo 1.0                      10/24
    6. Leela Zero 64x5 (CPU)           10/24
    7. Mogo 4.86                        3/24
League D:

Code: Select all

    1. MoGo 4.86                       18/20
    2. deltaGo 1.0.0                   14/20
    3. Fuego 1.1                       13/20
    4. Michi C-2 1.4.2                  8/20
    5. Orego 7.08                       5/20
    6. GNU Go 3.8                       2/20
League E:

Code: Select all

    1. GNU Go 3.8                      18/20
    2. Hara 0.9                        11/20
    3. Dariush 3.1.5.7                  9/20
    4. Indigo 2009                      9/20
    5. Matilda 1.25                     9/20
    6. OnePunchGo 1.0                   4/20
League F:

Code: Select all

    1. OnePunchGo 1.0                  17/20
    2. Aya 6.34                        16/20  
    3. JrefBot 081016-2022             12/20  
    4. Fudo Go 3.0                     10/20
    5. Iomrascálaí 0.3.2                4/20
    6. SimpleGo 0.4.3                   1/20
League G:

Code: Select all

    1. SimpleGo 0.4.3                  17/20
    2. AmiGo 1.8                       15/20
    3. Crazy Patterns 0008-13          13/20
    4. Marcos Go 1.0                   11/20
    5. Beancounter 0.1                  4/20
    6. Stop 0.9-005                     0/20
League H:

Code: Select all

    1. Stop 0.9-005                    18/20
    2. GoTraxx 1.4.2                   14/20
    3. V-Run 1                          9/20
    4. CopyBot 0.1                      8/20
    5. Leela Zero 32x4 (CPU)            7/20
    6. Brown 1.0                        4/20

Configuration:
Please note this is a continuous tournament, new versions will replace old versions, new AI's will be added. Matches in existence will not be replayed as long as the conditions remain the same.

Champions League: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 120 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League A-B: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League C-H: 1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

Amigo: amigogtp
AQ: AQ

Code: Select all

aq_config.txt:
-main　time[sec] =3600
-time controll =true
-japanese rule =true
Aya: Aya.exe –mode gtp –level max
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
Dariush: DarGTP.exe –level 10
DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 750 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
deltaGo: deltaGo.exe
Dream Go: export NUM_ITER=1375 and dream_go
ELF OpenGo: leelaz_015 --gtp --threads 4 --weights /path/to/network_elf --noponder
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
Fuego: fuego.exe –config fuego.cfg

Code: Select all

fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
GoTraxx: GoTraxx.exe
Hara: hara
Hiratuka: Hiratuka-19×19.exe -po 50000
IndiGo: Indigo.exe -gtp
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 50000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Leela Zero: leelaz --gtp --threads 4 --w /path/to/Leelaz_best-network_xxxxxx --noponder
Matilda: matilda

Code: Select all

matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Michi C-2: michi gtp

Code: Select all

ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.
MoGo: mogo
Oakfoam: oakfoam -c nicego-cnn-06.gtp

Code: Select all

nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4
OnePunchGo: taskset -c 0 OPG

Code: Select all

TimeInfo.h (Modified to play 15sec/move because OPG requires time_left command which is not supported by twogtp and Sabaki)
timeAvailable = 15;
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachidcnn -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
PhoenixGo: leelaz_for_phoenix_go --gtp --threads 4 --weights /path/to/network_phoenix_go --noponder
Ray: ray --time 3600 --thread 1 --no-debug
Rayon: rayon --thread 4 --no-debug
Simple Go: python /path/to/play_gtp.py --node_limit=100
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp
V-Run: python3 /path/to/V-Run
Zen: gtp4zen.exe -z 7 -t 4 -T 30 -s 10000 -i 1000
Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Beancounter: Private
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Dream Go: https://github.com/Chicoryn/dream-go
ELF OpenGo: http://zero.sjeng.org/ Find the network 62b5417b and use it with the latest Leela Zero.
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/w ... 73259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/w ... 88349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: http://zero.sjeng.org/
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
OnePunchGo https://github.com/AlexKent3141/OnePunchGo
Orego: https://sites.google.com/a/lclark.edu/d ... arch/orego
Pachi: https://github.com/pasky/pachi
PhoenixGo: https://github.com/yenw/LeelaZero_PhoenixGo
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Simple Go: https://sourceforge.net/projects/londerings/
Stop: https://www.vanheusden.com/stop/
V-Run: https://github.com/Fioelkais/MCTS
Zen via gtp4zen (info): https://www.lifein19x19.com/forum/viewt ... 18&t=15599
Best,
Alex

Re: Engine Tournament

Posted: Sat May 26, 2018 12:13 am
by as0770
Update: PhoenixGo, in fact LZ 0.14 with the PhoenixGo 256x19 network, added to the Champions League.
Edith: CPU version of PhoenixGo added to League A.

Champions League:

Code: Select all

    1. ELF OpenGo v0 LZ 0.15           20/24
    2. Leela Zero 0.15 11057dd         14/24
    3. PhoenixGo v1 LZ 0.14x           12/24
    4. Zen 7                            2/24
League A:

Code: Select all

    1. ELF OpenGo v0 LZ 0.13x (CPU)    15/20
    2. Zen 7                           14/20
    3. AQ 2.1.1                        11/20
    4. PhoenixGo v1 LZ 0.14x (CPU)      7/20
    5. Leela 0.11                       7/20
    6. Leela Zero 0.13 128x10 (CPU)     6/20
League B:

Code: Select all

    1. Leela Zero 0.13 128x10 (CPU)    22/24
    2. Leela 0.11.0                    19/24
    3. Rayon 4.6.0                     13/24
    4. Oakfoam 0.2.1 NG-06             12/24
    4. Leela Zero 128x6 (CPU)          12/24
    6. Hiratuka 10.37B (CPU)            4/24
    7. DreamGo 0.5.0                    2/24
League C:

Code: Select all

    1. DreamGo 0.5.0                   18/24
    2. DarkForrest MCTS 1.0            16/24
    2. Pachi 12                        16/24
    4. Ray 9.0.1                       11/24
    5. DarkGo 1.0                      10/24
    5. Leela Zero 64x5 (CPU)           10/24
    7. Mogo 4.86                        3/24
League D:

Code: Select all

    1. MoGo 4.86                       18/20
    2. deltaGo 1.0.0                   14/20
    3. Fuego 1.1                       13/20
    4. Michi C-2 1.4.2                  8/20
    5. Orego 7.08                       5/20
    6. GNU Go 3.8                       2/20
League E:

Code: Select all

    1. GNU Go 3.8                      18/20
    2. Hara 0.9                        11/20
    3. Dariush 3.1.5.7                  9/20
    3. Indigo 2009                      9/20
    5. Matilda 1.25                     9/20
    6. OnePunchGo 1.0                   4/20
League F:

Code: Select all

    1. OnePunchGo 1.0                  17/20
    2. Aya 6.34                        16/20  
    3. JrefBot 081016-2022             12/20  
    4. Fudo Go 3.0                     10/20
    5. Iomrascálaí 0.3.2                4/20
    6. SimpleGo 0.4.3                   1/20
League G:

Code: Select all

    1. SimpleGo 0.4.3                  17/20
    2. AmiGo 1.8                       15/20
    3. Crazy Patterns 0008-13          13/20
    4. Marcos Go 1.0                   11/20
    5. Beancounter 0.1                  4/20
    6. Stop 0.9-005                     0/20
League H:

Code: Select all

    1. Stop 0.9-005                    18/20
    2. GoTraxx 1.4.2                   14/20
    3. V-Run 1                          9/20
    4. CopyBot 0.1                      8/20
    5. Leela Zero 32x4 (CPU)            7/20
    6. Brown 1.0                        4/20

Configuration:
Please note this is a continuous tournament, new versions will replace old versions, new bots will be added. Matches in existence will not be replayed as long as the conditions remain the same.

Champions League: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 4 -komi 7.5 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League A-B: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League C-H: 1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

Amigo: amigogtp
AQ: AQ

Code: Select all

aq_config.txt:
-main　time[sec] =3600
-time controll =true
-japanese rule =true
Aya: Aya.exe –mode gtp –level max
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
Dariush: DarGTP.exe –level 10
DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 750 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
deltaGo: deltaGo.exe
Dream Go: export NUM_ITER=1375 and dream_go
ELF OpenGo: leelaz_015 --gtp --threads 4 --weights /path/to/network_elf --noponder
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
Fuego: fuego.exe –config fuego.cfg

Code: Select all

fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
GoTraxx: GoTraxx.exe
Hara: hara
Hiratuka: Hiratuka-19×19.exe -po 50000
IndiGo: Indigo.exe -gtp
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 50000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Leela Zero: leelaz --gtp --threads 4 --w /path/to/Leelaz_best-network_xxxxxx --noponder
Matilda: matilda

Code: Select all

matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Michi C-2: michi gtp

Code: Select all

ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.
MoGo: mogo
Oakfoam: oakfoam -c nicego-cnn-06.gtp

Code: Select all

nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4
OnePunchGo: taskset -c 0 OPG

Code: Select all

TimeInfo.h (Modified to play 15sec/move because OPG requires time_left command which is not supported by twogtp and Sabaki)
timeAvailable = 15;
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachidcnn -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
PhoenixGo: leelaz_for_phoenix_go --gtp --threads 4 --weights /path/to/network_phoenix_go --noponder
Ray: ray --time 3600 --thread 1 --no-debug
Rayon: rayon --thread 4 --no-debug
Simple Go: python /path/to/play_gtp.py --node_limit=100
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp
V-Run: python3 /path/to/V-Run
Zen: gtp4zen.exe -z 7 -t 4 -T 30 -s 10000 -i 1000
Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Beancounter: Private
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Dream Go: https://github.com/Chicoryn/dream-go
ELF OpenGo: http://zero.sjeng.org/ Find the network 62b5417b and use it with the latest Leela Zero.
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/w ... 73259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/w ... 88349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: http://zero.sjeng.org/
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
OnePunchGo https://github.com/AlexKent3141/OnePunchGo
Orego: https://sites.google.com/a/lclark.edu/d ... arch/orego
Pachi: https://github.com/pasky/pachi
PhoenixGo: https://github.com/yenw/LeelaZero_PhoenixGo
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Simple Go: https://sourceforge.net/projects/londerings/
Stop: https://www.vanheusden.com/stop/
V-Run: https://github.com/Fioelkais/MCTS
Zen via gtp4zen (info): https://www.lifein19x19.com/forum/viewt ... 18&t=15599
Best,
Alex

Re: Engine Tournament

Posted: Sun May 27, 2018 4:16 am
by luigi
as0770 wrote:There were no results affected by the komi of 6.5.
It doesn't make sense to use 6.5 komi with area scoring.

Re: Engine Tournament

Posted: Sun May 27, 2018 8:02 am
by as0770
luigi wrote:
as0770 wrote:There were no results affected by the komi of 6.5.
It doesn't make sense to use 6.5 komi with area scoring.
It happend to be a komi of 6.5 and I won't replay all games.

Re: Engine Tournament

Posted: Tue May 29, 2018 11:23 am
by luigi
as0770 wrote:
luigi wrote:
as0770 wrote:There were no results affected by the komi of 6.5.
It doesn't make sense to use 6.5 komi with area scoring.
It happend to be a komi of 6.5 and I won't replay all games.
You can still make it 7.5 hereafter.

Re: Engine Tournament

Posted: Tue May 29, 2018 1:37 pm
by as0770
luigi wrote:You can still make it 7.5 hereafter.
I don't want to have different settings in one tourney. One day I'll get a new computer and will start from scratch :)

The statistical fluctuation has much more influence on the result than the komi.

Re: Engine Tournament

Posted: Wed May 30, 2018 8:40 pm
by pnprog
luigi wrote:You can still make it 7.5 hereafter.
Curious about that: how to do?
as0770 wrote:One day I'll get a new computer and will start from scratch
You may have to implement a 6.5komi league, and a 7.5komi league. Some bots are just not supposed to play at 7.5komi (AQ is the strongest of those) and other bots are not supposed to play at 6.5komi (Leela, LeelaZero, Ray... the vast majority of value network bots). It's probably that Japanese bots are more suitable for 6.5komi, so maybe Zen also plays better on 6.5komi, but it a case by case checking.

Bots without value network can probably play in both leagues, but those are expected to be much weaker bots. So maybe 2 leagues A (for 6.5komi and 7.5komi) and others leagues (B, C, D...) of weaker bots that can stay at 6.5komi so you don't have to re-run those matches.

Re: Engine Tournament

Posted: Thu May 31, 2018 5:53 am
by as0770
pnprog wrote:
luigi wrote:You can still make it 7.5 hereafter.
Curious about that: how to do?
as0770 wrote:One day I'll get a new computer and will start from scratch
You may have to implement a 6.5komi league, and a 7.5komi league. Some bots are just not supposed to play at 7.5komi (AQ is the strongest of those) and other bots are not supposed to play at 6.5komi (Leela, LeelaZero, Ray... the vast majority of value network bots). It's probably that Japanese bots are more suitable for 6.5komi, so maybe Zen also plays better on 6.5komi, but it a case by case checking.

Bots without value network can probably play in both leagues, but those are expected to be much weaker bots. So maybe 2 leagues A (for 6.5komi and 7.5komi) and others leagues (B, C, D...) of weaker bots that can stay at 6.5komi so you don't have to re-run those matches.
What can we expect by a change to komi 7.5?

In the Champions League there are 3 Leela networks. They don't care about the komi, all results of them are due to resignation. So playing with a komi of 7.5 would have lead to the same results, right? 4th bot is Zen, it won two games with white by resignation of the opponent in completely lost positions. The other games of Zen where lost, so also no influence by the komi. Even though I doubt it will make any difference I'll change the komi for the Champions League to 7.5 from now on since it would have lead to the same results in the previous games.

In League A it doesn't look very different. I am very sure the inaccuracy in the results is many times smaller than the statistical fluctuation of a 20 games tournament.

Re: Engine Tournament

Posted: Thu May 31, 2018 7:06 am
by pnprog
as0770 wrote:What can we expect by a change to komi 7.5?
I think Zen may drop out of the champion league and A league if komi is 7.5pt. But we won't know until we try.
In the Champions League there are 3 Leela networks. They don't care about the komi, all results of them are due to resignation. So playing with a komi of 7.5 would have lead to the same results, right?
Not necessarily. In fact, Leela Zero should only lose by resignation, if she lose 0.5pt, then there is definitively something wrong. Same goes for AQ, Leela and Ray.

But let's take and example, LZ (black) vs Zen (white), komi=6.5pt
20 moves before the end of the game, black (LZ) is about to win by 0.5pt, seems good for LZ. But because LZ is trained for 7.5pt komi, she believes she will loose by 0.5 (her value network gives a 23% win rate), so she starts desperate attempts at complicating the game, play weird end game, lose points here and there, then her win rate drops bellow 20% and finally she resigns.

Zen was suppose to loose, but he ends up winning, Leela Zero resigns from a completely lost position.
4th bot is Zen, it won two games with white by resignation of the opponent in completely lost positions.
Difficult to say, maybe Zen was losing by 0.5pt 20 moves before then end, and the opponent, believing it was losing, throw the game away?
The other games of Zen where lost, so also no influence by the komi.
For those ones, I agree!
Even though I doubt it will make any difference I'll change the komi for the Champions League to 7.5 from now on since it would have lead to the same results in the previous games.
If you do so, I encourage you to replay all the matches of the league, but, sure, that's a lot of time, energy (and money).
In League A it doesn't look very different. I am very sure the inaccuracy in the results is many times smaller than the statistical fluctuation of a 20 games tournament.
Well, I will be surprised if Zen can score 14 wins.

You know, even between bots that are trained at 7.5pt komi, it could make a difference. Maybe some of them are stronger "as black", and they loose that "edge" because of the 6.5pt komi? Who know?

Re: Engine Tournament

Posted: Thu May 31, 2018 8:30 am
by as0770
pnprog wrote:I will be surprised if Zen can score 14 wins.
Against the CPU version of Leela Zero with different networks?

I think I'll keep my results private in the future. This discussions take more time than to play the games.

Bye

Re: Engine Tournament

Posted: Sat Jun 02, 2018 12:08 am
by q30
Last version of Pachi became weaker (details).

Re: Engine Tournament

Posted: Mon Jun 11, 2018 12:51 am
by q30
Last version of Ray became a bit stronger (details).
The rating of available for usual users Go engines without GPU support is next:

Highest level
1. Leela;

High level
2. Hiratuka;

Middle level
3. Pachi_DCNN;
4. Ray;
5. Pachi;
6. MoGo.

Re: Engine Tournament

Posted: Sat Jun 23, 2018 2:01 am
by q30
I had sparred last version of Ray with other this level engines (details).
The rating of available for usual users Go engines without GPU support is next:

Highest level
1. Leela;

High level
2. Hiratuka;

Middle level
3. Pachi_DCNN;
4&5. Ray & Pachi;
6. MoGo.

Re: Engine Tournament

Posted: Sat Jun 30, 2018 2:07 am
by q30
I had tested LeelaZero (details).
The rating of Go engines without using GPU is next:

Super level
1. LeelaZero;

Highest level
2. Leela;

High level
3. Hiratuka;

Middle level
4. Pachi_DCNN;
5&6. Ray & Pachi;
7. MoGo.
All times are UTC-07:00
Page 15 of 27

Powered by phpBB® Forum Software © phpBB Limited