Life In 19x19

as0770 wrote: The first step will be to find the differences between all engines, (no matter where to start ) and then calibrate it with the reference engines of known strength. The 2nd step could be to do the same in faster games, then one could create a matrix of strength on different hardware.

Revilo wrote:

as0770 wrote: The first step will be to find the differences between all engines, (no matter where to start ) and then calibrate it with the reference engines of known strength. The 2nd step could be to do the same in faster games, then one could create a matrix of strength on different hardware.

Hello Alex,

you could have a look into BayesElo here: http://www.remi-coulom.fr/Bayesian-Elo/. It is a widely used tool in chess programming to determine strength differences between engines. I haven't worked with it much, but basically, you feed it game results in PGN format and receive ELO ratings calibrated around an initial ELO rating. It will also calculate a confidence interval (called the error bar), which will become narrower the more games are available.

Because BayesELO doesn't care about the chess moves but only for the metadata, you could write a script to convert your SGFs into PGNs without moves.

I don't know if GoGUI has good support for engine tournaments. In chess, there are tools that can run tournaments automatically - so you could set up a dedicated machine, configure the tournament, press "Start" and come back a week later to view at the BayesElo results...

Hane wrote:New engine Matilda by Gonçalo

https://github.com/gonmf/matilda

    1. Leela 0.6.2                  22:2
    2. Hiratuka 10.37               21:3
    3. PachiUCT 11.99               15:9
    4. Ray 8.0.1                    13:11
    5. Fuego 1.1                     7:17
    6. MoGo 3                        5:19
    7. GNU Go 3.8                    1:23

    1. GNU Go 3.8                   25:3
    2. Hara 0.9                     17:11
    3. Dariush 3.1.5.7              16:12
    4. Indigo 2009                  16:12
    5. Matilda 1.20                 14:14
    6. Aya 6.34                     13:15
    7. Fudo Go 3.0                  11:17
    8. AmiGoGtp 1.8                  0:28

Hane wrote:Leela 0.7.0

http://www.sjeng.org/leela.html

    1. Leela 0.7.0                  18/24
    2. Hiratuka 10.37B               6/24

    1. Leela 0.7.0                  22/24
    2. Hiratuka 10.37B              21/24
    3. PachiUCT 11.99               15/24
    4. Ray 8.0.1                    13/24
    5. Fuego 1.1                     7/24
    6. MoGo 3                        5/24
    7. GNU Go 3.8                    1/24

    1. GNU Go 3.8                   25/28
    2. Hara 0.9                     17/28
    3. Dariush 3.1.5.7              16/28
    4. Indigo 2009                  16/28
    5. Matilda 1.20                 14/28
    6. Aya 6.34                     13/28
    7. Fudo Go 3.0                  11/28
    8. AmiGoGtp 1.8                  0/28

gonmf wrote:Hello, I am the author of the Matilda bot, this is my first post.

I noticed you are launching matilda without any options, and this will by default use as many threads as available cores. I noticed that the other bots are playing on one thread and as such they may be playing in unfair ground.

Best wishes,
Gonçalo

gonmf wrote:Hello, I am the author of the Matilda bot, this is my first post.

I noticed you are launching matilda without any options, and this will by default use as many threads as available cores. I noticed that the other bots are playing on one thread and as such they may be playing in unfair ground.

Best wishes,
Gonçalo

q30 wrote:May be it will be another rate, if someone will let know, how to change Leela's thinking time...

as0770 wrote: You can set Leela's (and most other gtp engines) thinking time via gtp.

I use twogtp with this options TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

The gtp command looks like this: time_settings 3600 0 0

Best,
Alex

• League A
• 1. Leela

• League B
• 2. Ray
• 3. Pachi
• 4. MoGo

gonmf wrote:Hello, I am the author of the Matilda bot, this is my first post.

I noticed you are launching matilda without any options, and this will by default use as many threads as available cores. I noticed that the other bots are playing on one thread and as such they may be playing in unfair ground.

Best wishes,
Gonçalo

 -t 0+2m/1 --think_in_opt_time --memory 8000 --threads 2 -l ewi --log_dest o

#define BOARD_SIZ 19
#define DEFAULT_KOMI 15
#define DEFAULT_UCT_MEMORY 7000
#define LATENCY_COMPENSATION 270
#define DEFAULT_DATA_PATH "./data/"
#define MATILDA_RELEASE_MODE 1
#define DEFAULT_NUM_THREADS 0
#define MAXIMUM_NUM_THREADS 64
#define CAN_RESIGN 1
#define UCT_EXPANSION_DELAY 1
#define DEFAULT_TIME_PER_TURN 100000

#define UCT_RESIGN_WINRATE 0.25