Page 1 of 2
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Wed Sep 13, 2017 3:13 pm
by johnsmith
I 've been having the exact same issue on my average laptop.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Wed Sep 13, 2017 3:49 pm
by jeromie
I haven't tried running it, but I wonder if it's this item from the change log:
New, much stronger Monte Carlo evaluation by combining Policy Gradient Reinforcement Learning and Simulation Balancing.
That seems consistent with extra time being in the evaluation process.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Thu Sep 14, 2017 4:45 am
by jin2983
is twogtp.py available opensource? I can post results for comparison from a machine with a large gpu and linux.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Thu Sep 14, 2017 9:25 am
by Satorian
Given the release notes, it sounds like the playouts are a lot heavier than before, with a lot more attached to them. Playout count may go down, but overall level and playing quality should increase.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Fri Sep 15, 2017 12:46 pm
by Garf
Satorian wrote:Given the release notes, it sounds like the playouts are a lot heavier than before, with a lot more attached to them. Playout count may go down, but overall level and playing quality should increase.
From the same release notes:
"More accurate, larger value network for 19x19."
Larger network would also take longer to compute.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Sat Sep 16, 2017 7:48 pm
by pookpooi
This person said that Leela 11 beta is 2 stones stronger than Leela 10
https://www.reddit.com/r/baduk/comments ... nger_than/
By the way, Beta 5 is out, the changelog is
Windows version is now compiled with Clang/LLVM 5.0 instead of MSVC2017. This makes the Monte Carlo evaluations about 15% faster.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Tue Sep 19, 2017 8:29 am
by jin2983
I started on a 40 game series but 0.11 won the first 12 of them so I'll restart with 0.5 komi to get into 0-100% range.
Different from above, I used 7.5 komi as leela plays by chinese rules.
command:
Code: Select all
~/pachi/tools/twogtp.py --black '0.10.0/leela_0100_linux_x64_opencl --gtp --quiet --noponder' --white '0.11.0b5/leela_0110_linux_x64_opencl --gtp --quiet --noponder' --komi 7.5 --games 40
0.10 openCL benchmarks:
Code: Select all
Leela: benchmark
200000 games in 7.55 seconds -> 26490 g/s (3311 g/s per thread)
Avg Len: 451.69 Score: -1.961805
Leela: netbench
2000 predictions in 1.45 seconds -> 1379 p/s
10000 evaluations in 3.44 seconds -> 2906 p/s
0.11b5 openCL benchmarks:
Code: Select all
Leela: benchmark
200000 games in 10.76 seconds -> 18587 g/s (2323 g/s per thread)
Avg Len: 418.58 Score: -2.588590
Leela: netbench
2000 predictions in 3.11 seconds -> 643 p/s
10000 evaluations in 3.40 seconds -> 2941 p/s
Looking at benchmark results predictions are slower(explained by larger value net?) while evaluations are roughly equal(same sized policy net?).
As for playing strength / game quality, having a GPU seems almost a must.
First 5 games attached below:
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Wed Sep 20, 2017 8:41 am
by jin2983
so far 0.11b5 as white has won 7 out of 8 games on 0.5 komi
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Wed Sep 20, 2017 9:57 am
by pookpooi
beta 9 is out and changelog didn't mention any strength-related improvement
Judging by the number, if nothing wrong happen the completed version 11 will release soon.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Mon Sep 25, 2017 1:14 pm
by Krama
I am just a mid kyu level player but holy detritus is Leela strong.
Bot easily gives me 6 handicap and I struggle.
I can't imagine what AlphaGo level is.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Mon Sep 25, 2017 3:38 pm
by johnsmith
Krama wrote:I am just a mid kyu level player but holy detritus is Leela strong.
Bot easily gives me 6 handicap and I struggle.
I can't imagine what AlphaGo level is.
I feel you. Too strong... even now (Leela 0.11 beta 10)...
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Tue Sep 26, 2017 5:52 am
by toannguyenthanh
I think that Leela is stronger is the most important, slower is not a problem.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Tue Sep 26, 2017 11:51 am
by Krama
johnsmith wrote:Krama wrote:I am just a mid kyu level player but holy detritus is Leela strong.
Bot easily gives me 6 handicap and I struggle.
I can't imagine what AlphaGo level is.
I feel you. Too strong... even now (Leela 0.11 beta 10)...
You are a 6 dan on kgs? So that is like 4 dan EGF right?
How well can you fare against gpu version on unlimited simulations (let's say 30 min)
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Wed Sep 27, 2017 1:43 am
by Uberdude
Krama wrote:
You are a 6 dan on kgs? So that is like 4 dan EGF right?
As an aside, I don't think many 4d EGF are 6d KGS, I'm certainly not. There are 6d EGF who are 6d KGS like Lukas Kraemer, or Cornel Burzo was 6d for a long time and just got back to 7d. KGS dan ranks are pretty tough these days. Ilya and Pavol (7d/1p) are only 7d KGS now.
Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Posted: Wed Sep 27, 2017 3:59 am
by johnsmith
Krama wrote:
You are a 6 dan on kgs? So that is like 4 dan EGF right?
How well can you fare against gpu version on unlimited simulations (let's say 30 min)
Well, I don't play on KGS, only on Tygem. And I'm 7d over there. I guess that's something like solid 5d EGF
Anyways, LEELA(B) on Tygem has 50:50 score against pros. That surely tells something about its strength. And it only plays against pros.
Personally, I like to play with RoyalLeela on OGS (thx to roy777 who has Ryzen 1700 CPU and 1080 Ti GPU), and I have 50:50 against Leela on 3 handicap stones; winrate on 4 handi is 100% though I struggle a little bit
In even games RoyalLeela just smashes. Like Hulk.
On my personal average laptop, in even games I don't stand a chance either, though I have 100% score against 3 handi. And so far I won 2 games in Sunjung Baduk.