LZ's progression

nbc44 · **#381**

The main question is what is more important to us - victory or honesty? :salute:

iopq · **#382**

Did you set the batch number to half of the threads? You can get better perf.

Vargo · **#383**

Three last benchmarks :

not specifying -t XXX seems to give slightly less n/s

iopq wrote:

Did you set the batch number to half of the threads?

I'm not sure it's better... but again, I find the effect of -t XXX rather bizarre, and these benchmarks are maybe flawed, one way or another...

--precision half seems to be about the same as not specifying the precision

iopq · **#384**

Benchmark with batching, it would be faster than just threading

Uberdude · **#385**

LZ just beat Golaxy in the Fuzhou AI tournament :tmbup:

https://home.yikeweiqi.com/#/live/board/17523

Amtiskaw · **#386**

Is there a way to download SGF from that site, and if yes, can someone post it here? :study:

Alright I think I found them...

Uberdude · **#387**

I've previously hacked out the sgf from yike using browser dev tools, don't know if there's an easier way.

Inline sgf players:

splee99 · **#388**

iopq wrote:

Benchmark with batching, it would be faster than just threading

Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different.

Amtiskaw · **#389**

Leela lost both its semi-final games. I enjoyed watching the second one live, it had a rather drastic semeai, which sadly became 1-eye vs 0-eye...

hoa803 · **#390**

nbc44 wrote:

hoa803 wrote:

NBC, if you're using visit parity you shouldn't use ponder. The time to reach that number of visits varies by position. Time parity matches can use ponder on separate hardware though, similar to how Alphago was tested.

Why not? I'm using separate GPU for each net.

Think about what you are trying to do in terms of mathematics. LZ has a chance to win a random game, let us call that probability P.

In a match with ponder turned on, you've introduce another variable - the total thinking time permitted for each engine due to use of ponder. On a given game either LZ or Elf is likely to get more overall thinking time. Since we already know that strength is directly related to thinking time, your chance of LZ winning a particular game is now the function P(x), where x is a random variable related to the strength at different thinking times.

That means that the statistical basis being used to evaluate strength is no longer valid, because with fixed visit count and ponder the result is a function of another random variable that we don't know anything about. The function P(x) is most likely Gaussian, but we don't know the standard deviation or anything along those lines. I'm not enough of a mathematician to know what that does to the conclusion over a 400 game match.

Also - you should put your queries about thread and batch count to the actual programmers on GitHub. Again you are introducing variables that you don't understand. I think I've seen some discussion about both batch size and number of threads having an impact on performance. You should definitely ask if you want to understand what is going on. Maybe post your results and see what GCP says about it.

iopq · **#391**

splee99 wrote:

iopq wrote:

Benchmark with batching, it would be faster than just threading

Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different.

Something like --batchsize I believe

hoa803 · **#392**

Quote:

Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different.

If you use Leela Zero engine version 0.17 it should choose the optimal thread and batch size for your hardware by default. Note that you cannot have more batches than threads.

Try this command in your LZ directory:

leelaz -h

You will see a list of some things you can do.

In terms of engine strength, this person has done a ton of work:

https://github.com/guitanj/AutoPK/blob/master/README.md

These data show that if we are interested in strength as a function on thinking time, LZ will be noticeably weaker since Elfv2 is only a 20-block network and therefore evaluates more quickly. In terms of pure network strength regardless of thinking time, Leela Zero is generally stronger except at high playouts. I find that very interesting.

Very cool data and a lot of work. This person deserves much credit for doing all these tests.

iopq · **#393**

I believe it will do something like batch size 5 and 10 threads.

Anyone try the new 15b net vs. the 40b at time parity?

Aram · **#394**

There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago.

The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive!
The 40b network is clearly still growing and learning and getting stronger at a good pace!

Uberdude · **#395**

Aram wrote:

There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago.

The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive!
The 40b network is clearly still growing and learning and getting stronger at a good pace!

I wonder what the improvement in winrate of this latest 40b compared to 10 networks ago would be versus a non-LZ bot (for fairest comparison would be time parity but even visit parity I expect same effect). Eg maybe #215 beats Elfv2 at 1600 visit parity 65% of the time (and maybe <50% at time parity), and #225 beats it only 67%. I expect quite a lot less difference than 19% (or equivalent ratio difference).

splee99 · **#396**

Here is a game between LZ-225 (B) + LZ0.17 and ELF (W) + LZ0.16 in time parity (20 seconds per move).
The ELFv2 was far ahead with winrate close to 90%, until move 135, when LZ-225 suddenly started a complicated fight. After that ELF still had lots of chance to win the fight, according to LZ-225, but eventually ELF gave up.

Attachment:

225-elf-20s.sgf [3.77 KiB]
Downloaded 2225 times

Aram · **#397**

One shouldnt use ELF winrates or ELF in general for analysis.

They sharpened the network a rather huge amount to make it stronger in self play and play versus other bots.
(Probably because they plateud and needed to get some more strength out of the relatively small 20-ish block network)
So basically ELFs possibility to grow and explore new and different moves was terminated and it became stronger on a narrow path of moves.

This also sadly led the network to believe that every match is either a 100% win or a 100% loss.

This is quite visible for anyone using both ELF and LZ networks. ELF will always be very black and white and whats good or bad, while the LZ networks provide several moves which are playable.

LZ could do the same and would probably see a boost in strength, but at the same time it probably wouldnt improve anymore and it wouldnt explore either.
Since the LZ network is larger, it still has capacity to grow and improve by searching and exploring, and as such its not needed to try to force it to its full strength right now.

Same thing with learning rates, one could probobly get a small increase in LZ strength by just dropping the learning rate, but that would kill any further increase in playing strength from exploration.

Vargo · **#398**

20 game match LZ017#225 v LZ017#215
no pondering, komi 7.5, -v 1600
twogtp 1.5.1 , no error, no duplicate game.

#225 wins 17-3
Wow ! It confirms the 69% , and it's only a 10 networks difference... :clap:

The stats :

The games (225 is B in the odd-numbered games) :

nbc44 · **#399**

Сity legend check (l0XXX 1600 visits vs elfv2 3200 visits):

2 x 1080ti

Code:

===================================================================================================================
From:May-07-19 07:12:33 to May-08-19 05:08:00. Spend 78927.58483s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 249  62.25%        win(B) = 115  57.50%        win(W) = 134  67.00%
ELFV2                    1600 visits  win(All) = 151  37.75%        win(B) =  66  33.00%        win(W) =  85  42.50%

===================================================================================================================
From:May-09-19 07:42:53 to May-10-19 06:46:29. Spend 83015.170982s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 212  53.00%        win(B) = 102  51.00%        win(W) = 110  55.00%
ELFV2                    2000 visits  win(All) = 188  47.00%        win(B) =  90  45.00%        win(W) =  98  49.00%

===================================================================================================================
From:May-08-19 05:08:00 to May-09-19 05:47:02. Spend 88741.358819s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 180  45.00%        win(B) =  91  45.50%        win(W) =  89  44.50%
ELFV2                    2400 visits  win(All) = 220  55.00%        win(B) = 111  55.50%        win(W) = 109  54.50%

===================================================================================================================
From:May-10-19 07:37:42 to May-11-19 07:16:09. Spend 85106.834626s
                         Game << 400 >>
LZ225                    1600 visits  win(All) = 207  51.75%        win(B) =  94  47.00%        win(W) = 113  56.50%
ELFV2                    2000 visits  win(All) = 193  48.25%        win(B) =  87  43.50%        win(W) = 106  53.00%

===================================================================================================================
From:May-11-19 07:16:09 to May-12-19 07:23:49. Spend 86860.492488s
                         Game << 400 >>
LZ225                    1600 visits  win(All) = 172  43.00%        win(B) =  81  40.50%        win(W) =  91  45.50%
ELFV2                    2200 visits  win(All) = 228  57.00%        win(B) = 109  54.50%        win(W) = 119  59.50%

===================================================================================================================
From:May-14-19 06:55:16 to May-15-19 06:12:54. Spend 83858.237752s
                         Game << 400 >>
LZ226                    1600 visits  win(All) = 212  53.00%        win(B) = 100  50.00%        win(W) = 112  56.00%
ELFV2                    2000 visits  win(All) = 188  47.00%        win(B) =  88  44.00%        win(W) = 100  50.00%

===================================================================================================================
From:May-15-19 06:12:54 to May-16-19 05:59:42. Spend 85607.877284s
                         Game << 400 >>
LZ226                    1600 visits  win(All) = 196  49.00%        win(B) =  94  47.00%        win(W) = 102  51.00%
ELFV2                    2200 visits  win(All) = 204  51.00%        win(B) =  98  49.00%        win(W) = 106  53.00%

P.S. If someone wants the games, I can upload them.

kityanhem · **#400**

Can I have the games, nbc44?
For studying, of course! :study:

LZ's progression

Who is online