Life In 19x19 http://www.lifein19x19.com/ |
|
Engine Tournament http://www.lifein19x19.com/viewtopic.php?f=18&t=13322 |
Page 6 of 21 |
Author: | aTan [ Mon Oct 02, 2017 3:53 pm ] |
Post subject: | Re: Engine Tournament |
When you are planning to include latest versions of leela and Ray? |
Author: | as0770 [ Tue Oct 03, 2017 5:09 am ] |
Post subject: | Re: Engine Tournament |
aTan wrote: When you are planning to include latest versions of leela and Ray? I'll add Leela with the official release, unfortunately I am still not able to compile the latest version Rayon... |
Author: | q30 [ Sat Oct 07, 2017 1:44 am ] |
Post subject: | Re: Engine Tournament |
I had downloaded CUDA older version and NVIDIA driver v.340. But while executing command "./darknet go test cfg/go.test.cfg go.weights" I get error: "CUDA Error: invalid device function"... And what about how is it possible to run darknet in gtp mode with maximum strong (including pondering, neural network), using 2 CPU cores, up to 8 GB memory and up to 2 min / move time? Thanks in advance. |
Author: | Hane [ Sun Oct 15, 2017 3:34 am ] |
Post subject: | Re: Engine Tournament |
New version of AQ released !! https://github.com/ymgaq/AQ/releases |
Author: | as0770 [ Fri Oct 20, 2017 11:21 am ] |
Post subject: | Re: Engine Tournament |
AQ 2.0.3 can now play with ponder off, so I set up a match against Leela 0.11.0 Beta 11. Leela vs. AQ Code: 1. AQ 2.0.3 12/16 2. Leela 0.11.0 Beta 11 4/16 Configuration: League A: Code: 1. Leela 0.10.0 22/24 2. Rayon 4.6.0 19/24 3. Oakfoam 0.2.1 NG-06 18/24 4. Hiratuka 10.37B (CPU) 9/24 5. DarkForest v2 MCTS 1.0 7/24 6 DarkGo 1.0 5/24 7. Pachi DCNN 11.99 4/24 Configuration: League B: Code: 1. Pachi DCNN 11.99 25/28 2. Ray 9.0.1 25/28 3. MoGo 4.86 18/28 4. deltaGo 1.0.0 16/28 5. Fuego 1.1 13/28 6. Michi C-2 1.4.2 8/28 7. Orego 7.08 5/28 8. GNU Go 3.8 2/28 Configuration: League C: Code: 1. GNU Go 3.8 24/28 2. Hara 0.9 18/28 3. Dariush 3.1.5.7 16/28 4. Indigo 2009 15/28 5. Matilda 1.24 15/28 6. Aya 6.34 11/28 7. Fudo Go 3.0 11/28 8. JrefBot 081016-2022 2/28 Configuration: League D: Code: 1. JrefBot 081016-2022 25/28 2. Iomrascálaí 0.3.2 21/28 3. Crazy Patterns 0008-13 19/28 4. Marcos Go 1.0 18/28 5. AmiGo 1.8 16/28 6. Stop 0.9-005 6/28 7. CopyBot 0.1 4/28 8. Brown 1.0 3/28 Configuration: Links: Best, Alex[/quote] |
Author: | q30 [ Sat Oct 21, 2017 2:53 am ] |
Post subject: | Re: Engine Tournament |
DarkNet-MoGo4: 3-1 DarkNet-Pachi: 0-4 DarkNet start up parameters: Code: go engine cfg/engine.cfg go.weights Details So DarkNet isn't ready for use (weak engine, no documentation, doesn't know nor time control commands, nor resigning, nor final score)... The rating of strongest programs with GTP support without using GPU by real sparring is next: League A 1. Leela League B 2. Hiratuka 3. Ray 4. Pachi 5. DarkNet 6. MoGo Go engines start up parameters. |
Author: | Vargo [ Sat Oct 28, 2017 6:08 am ] | ||||
Post subject: | Re: Engine Tournament | ||||
I'm not sure if it's the right place to post this. If not, please tell me. I've used Sabaki for matches (GTP) between AQ 2.0.3, Leela0110GTP_OpenCL, Leela0110GTP, and GnuGo 3.8 My computer : Windows 10 i7-6700K (4gHz) 32GB GTX1080 Leela benchmark : 200000 games in 9.27s. 21574 g/s (2696 g/s per thread) GnuGo level 10, AQ and Leela 30" per move (time_settings 0 30 1) no pondering Results : 9 stones handicap : Leela0110GTP_OpenCL (W) beats GnuGo (B) 5-0 (3 times by resignation, and twice by 40-50 points) Even games : AQ beats Leela0110GTP_OpenCL 5-0 2 stones handicap : AQ (W) beats Leela0110GTP_OpenCL (B) 4-1 (always by resignation) AQ is very strong... It seems to confirm these results : Quote: 1. AQ 2.0.3 12/16 2. Leela 0.11.0 Beta 11 4/16 2 stones handicap : Leela0110GTP_OpenCL (W) beats Leela0110GTP (B) 4-1 (always by resignation) Leela "GPU accelerated" seems to be 2 stones stronger than the standard version... I'm very grateful to the authors of these programs, I enjoy them immensely. I have lots of questions about the various commands (time settings, number of threads, etc. etc) but I don't think it's the right place to ask them. I'll keep experimenting, with other handicaps and time settings
|
Author: | pookpooi [ Sat Oct 28, 2017 7:34 am ] |
Post subject: | Re: Engine Tournament |
Who is stronger between AQ, Rayon, and Leela? Can we rank them in 1-2-3 order? |
Author: | q30 [ Sat Oct 28, 2017 10:02 am ] |
Post subject: | Re: Engine Tournament |
Vargo wrote: I'll keep experimenting, with other handicaps and time settings You can try with pondering, half of Your PC CPU (I don't know, how it may be with GPU) cores to each engine and 2' for move without handicap (it will be close to real pro games)... |
Author: | q30 [ Sat Oct 28, 2017 10:05 am ] |
Post subject: | Re: Engine Tournament |
pookpooi wrote: Who is stronger between AQ, Rayon, and Leela? Can we rank them in 1-2-3 order? If Leela without OpenCL - I think, it will be 3. But it beats Ray... |
Author: | as0770 [ Sun Oct 29, 2017 12:33 am ] |
Post subject: | Re: Engine Tournament |
q30 wrote: You can try with pondering, half of Your PC CPU (I don't know, how it may be with GPU) cores to each engine and 2' for move without handicap (it will be close to real pro games)... You can't play fair ponder on matches on one GPU. |
Author: | Shoreline [ Sun Oct 29, 2017 10:48 am ] |
Post subject: | Re: Engine Tournament |
pookpooi wrote: Who is stronger between AQ, Rayon, and Leela? Can we rank them in 1-2-3 order? Don't know anything about AQ. The DGS ranking of Leela and Rayon bots have been consistently showing Rayon with 2 stone stronger. Leela v0.10.o , 4 Dan http://www.dragongoserver.net/userinfo. ... LeelaBot19 Rayon version 8.0.1 (RN-3.9), 6 Dan http://www.dragongoserver.net/userinfo. ... r=rayonbot |
Author: | Vargo [ Sun Oct 29, 2017 10:22 pm ] |
Post subject: | Re: Engine Tournament |
Quote: Leela v0.10.o , 4 Dan Leela 0.10 is weaker than Leela 0.11, which is weaker than Leela 0.11 openCL, which is weaker than AQ.On my computer, AQ 2.0.3.1 (W) wins each time against Rn-4.20.4 (B) with 2 stones handicap In CGOS (computer go server) http://www.yss-aya.com/cgos/19x19/bayes.html AQ is 3877 Elo, it's very high. On the same site, for example, Gnugo-level 11 is ranked 517, with 1825 Elo, and Ray-4.20, running on 8 CPU, is 3743 Elo Still in the same site, Leela is around 3700 Elo with 16 threads and GPU. But it's heavily dependent on the hardware, it's hard to make a fair comparison, because we usually don't know the exact specifications of the computers used for these games, hence the interest of the results obtained by as0770 in his engine tournament, same conditions and same hardware used for every engine (I suppose) |
Author: | splee99 [ Mon Oct 30, 2017 8:36 pm ] | ||
Post subject: | Re: Engine Tournament | ||
Leela 0.11 gave Hiratuka 10.37B 4 handicaps and still won.
|
Author: | pookpooi [ Mon Oct 30, 2017 8:43 pm ] |
Post subject: | Re: Engine Tournament |
So in conclusion Latest AQ >> latest Rayon Latest Rayon > Latest Leela Latest Leela >>>> Latest Hiratuka |
Author: | as0770 [ Tue Oct 31, 2017 5:29 am ] |
Post subject: | Re: Engine Tournament |
League B updated with Leela Zero and the weights file provided by GCP. It will be interesting to see how it can improve with better weigths files. Interesting that it lost 4 of 4 vs. Fuego and even 2 of 4 vs. Orego. On the other hand it was able to win 2 of 4 against Ray. New entry in League D is GoTraxx. Leela vs. AQ Code: 1. AQ 2.0.3 12/16 2. Leela 0.11.0 Beta 11 4/16 Configuration: League A: Code: 1. Leela 0.10.0 22/24 2. Rayon 4.6.0 19/24 3. Oakfoam 0.2.1 NG-06 18/24 4. Hiratuka 10.37B (CPU) 9/24 5. DarkForest v2 MCTS 1.0 7/24 6 DarkGo 1.0 5/24 7. Pachi DCNN 11.99 4/24 Configuration: League B: Code: 1. Pachi DCNN 11.99 29/32 2. Ray 9.0.1 27/32 3. MoGo 4.86 21/32 4. deltaGo 1.0.0 17/32 5. Fuego 1.1 17/32 6. Leela Zero 0.1 15/17 7. Michi C-2 1.4.2 9/32 8. Orego 7.08 7/32 9. GNU Go 3.8 2/32 Configuration: League C: Code: 1. GNU Go 3.8 24/28 2. Hara 0.9 18/28 3. Dariush 3.1.5.7 16/28 4. Indigo 2009 15/28 5. Matilda 1.24 15/28 6. Aya 6.34 11/28 7. Fudo Go 3.0 11/28 8. JrefBot 081016-2022 2/28 Configuration: League D: Code: 1. JrefBot 081016-2022 29/32 2. Iomrascálaí 0.3.2 25/32 3. Crazy Patterns 0008-13 23/32 4. Marcos Go 1.0 22/32 5. AmiGo 1.8 20/32 6. Stop 0.9-005 10/32 7. CopyBot 0.1 6/32 8. GoTraxx 1.4.2 6/32 9. Brown 1.0 3/32 Configuration: Links: Best, Alex |
Author: | q30 [ Sat Nov 11, 2017 12:52 am ] |
Post subject: | Re: Engine Tournament |
Vargo wrote: But it's heavily dependent on the hardware, it's hard to make a fair comparison, because we usually don't know the exact specifications of the computers used for these games, hence the interest of the results obtained by as0770 in his engine tournament, same conditions and same hardware used for every engine (I suppose) Quite right! Except the problems of same conditions while using GPU and of obtained with this synthetic conditions results equivalenty to results, obtained with real games conditions... |
Author: | q30 [ Sat Nov 11, 2017 1:00 am ] |
Post subject: | Re: Engine Tournament |
I have tested Leela 0.11.0 (details). The rating of engines with GTP support and without using GPU by sparrings in real Go games conditions remains the same: League A 1. Leela League B 2. Hiratuka 3. Ray 4. Pachi 5. DarkNet 6. MoGo Go engines start up parameters. |
Author: | q30 [ Sat Dec 02, 2017 3:09 am ] |
Post subject: | Re: Engine Tournament |
Darknet from 08.11.17 doesn't play - passes. So this engine will not be in new ratings... Oakfoam remains very weak engine (weaker, then MoGo: details). The rating of Go engines without using GPU is next: League A 1. Leela League B 2. Hiratuka 3. Ray 4. Pachi 5. MoGo Go engines start up parameters. |
Author: | as0770 [ Thu Dec 07, 2017 5:38 am ] |
Post subject: | Re: Engine Tournament |
q30 wrote: Quite right! Except the problems of same conditions while using GPU and of obtained with this synthetic conditions results equivalenty to results, obtained with real games conditions... Playing engine matches with ponder=off on one GPU is no problem. There might be a slight difference because of the GPU memory usage but this is not measurable for me. Playing matches on different machines can always lead to different results. We can rate engines just for specific conditions. Budget- or High-End-PC. With one GPU, with several GPUs or without GPU, With one CPU or several CPUs. With timecontrol, or fixed number of playouts. The difference between ponder on and ponder off matches is nearly irrelevant compared to all the other influences and for sure interfered by the statistical fluctuation when rating the engines with less than 100 games. |
Page 6 of 21 | All times are UTC - 8 hours [ DST ] |
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |