Humans vs. Alpha Go Zero

luigi · **#41**

gennan wrote:

I'm much more interested in the komi gap between pros and AI, which is much easier to determine. And you don't even need pros for that. Anyone can determine their komi distance to KataGo (at some standardized number of playouts).

Does mirror play become a problem with large komi?

wineandgolover · **#42**

gennan wrote:

I assume you mean 1d EGF and 9d EGF. That would be like an even game match between a decent 1d EGF (~1900 Elo) and O Rissei 9p (~3200 Elo). That's a 1300 Elo gap, which suggests that the 1d wil win about 1 in 1800 games.

I’d like to think playing 1800 games with O'Rissei would be good for two or three stones, thus lowering the odds.m ;-)

Pippen · **#43**

I don't agree with you that AI is so far away. It uses almost exclusively heuristics which has always some significant blind spots that can be exploited. Give professionals enough time (not like when AlphaMaster played those 60 pros with fast pace on Tygem) and I claim that in 100 games we get a win in an even game. And if I was not just some random dude but Elon Musk we might have a project at hand. ;-)

wineandgolover · **#44**

Pippen wrote:

I don't agree with you that AI is so far away. It uses almost exclusively heuristics which has always some significant blind spots that can be exploited. Give professionals enough time (not like when AlphaMaster played those 60 pros with fast pace on Tygem) and I claim that in 100 games we get a win in an even game. And if I was not just some random dude but Elon Musk we might have a project at hand. ;-)

I’d for sure take the other side of that bet. And I said so in a video when LSD beat AlphaGo. I think that was the last even game a reasonably powered AI will ever lose to a human.

Even though I agree that humans are getting better, I suspect AI are getting better even faster. I also suspect that in the not too distant future, someone (a coder, not a player) will come up with a new algorithm, and the question will be beyond all doubt.

jlt · **#45**

One way to guess the percentage of games that a top human pro could win against a strong bot A is to make A play 10000 times against a bot B which is of the same strength than a top human pro. Did someone do that?

Vargo · **#46**

According to THIS POST, Leela Zero #125, with 3000-4000 playouts, was already much stronger than 2p.

Marcel Grünauer wrote:

It usually got to 3000-4000 playouts before hitting the time limit.

Marcel Grünauer wrote:

Baolong also thinks that if he could exploit bugs, he could maybe win one or two games out of a hundred. He thinks this configuration could already be stronger than AlphaGo Lee.

Here is a 10 game test at H4, ~ time parity.
(gogui 1.5.1 twogtp, average length 240 moves, av. time B : 420s, av. time W, 460s)

LZ#125 -v 3500
v.
KG 1.8.1 network: kata1-b40c256-s7306904576-d1772668053.bin.gz -v 1200 (KG always W)

Result :
LZ wins 6-4 (4 times W+R, 3 times B+R, 3 times B+xx)

gennan · **#47**

Was that H4 with 0.5 komi or 7.5 komi?

LZ doen't understand anything else than 7.5 komi and chinese rules and it's not trained to play with handicap or against handicap. So I think LZ would have significant disadvantages in handicap match, because of weakness in handicap games and ladder issues, both of which are exploitable issues that pros won't have. So I don't think handicap matches between LZ and KataGo are reliable to determine AI ranks.

I think it would be more informative to establish handicaps between different versions of KataGo.

Vargo · **#48**

It was komi 0 and chinese rules, but you're right about LZ and handicap.

And · **#49**

@gennan
thanks for such a detailed answer! (https://lifein19x19.com/viewtopic.php?p=264114#p264114)

What do you think, if conditionally 0 Elo is assigned to 30 kyu, we can assume that with each rank the same Elo value is added and which value is better suited? and it is possible to derive an approximate formula for calculating the probability of winning between rank A and B?

gennan · **#50**

And wrote:

@gennan
thanks for such a detailed answer! (https://lifein19x19.com/viewtopic.php?p=264114#p264114)

What do you think, if conditionally 0 Elo is assigned to 30 kyu, we can assume that with each rank the same Elo value is added and which value is better suited? and it is possible to derive an approximate formula for calculating the probability of winning between rank A and B?

Assigning 0 Elo to 30 kyu is possible, but it doesn't really matter where you anchor an Elo scale, because Elo ratings are relative. Only Elo differences matter (within the same system). Absolute Elo ratings only matter when comparing different Elo systems that use different anchors (similar as comparing Centigrade temperatures to Kelvin tempatures).

With the conversion formulae and the anchor I gave in that post, EGF 30k(-900 GoR) would correspond to about 1000 Elo. Shifting it to 0 Elo as you suggest, would just shift all ratings down by 1000 Elo, without changeing any win% prediction made by the system.

Analysis of the EGF historical data shows that using the same Elo gap between all ranks does not match observed winrates. A 15k has a much higher win% against a 14k (~43%), than a 6d against a 7d(~25%). A 1d against a 2d is somewhere in between (~36%). So by definition of Elo (based on probability in even game win%), the Elo gap between 15k and 14k (~50 Elo) is smaller than the Elo gap between 6d and 7d (~200 Elo) and the Elo gap between 1d and 2d is somwehere in between (~100 Elo).

To calculate the predicted winrate between 2 players with given EGF GoR, this will be used by the EGF rating system after the coming update:

Code:

Se = 1 / (1 + exp(β(GoR2) - β(GoR1)))
where β = -7 * ln(3300 - GoR)

This β function can be seen as yet another rating scale, similar to Elo, but using a different unit to avoid redundant conversions.

And · **#51**

@gennan
I know the Elo ratings are relative, but thanks a lot for the other part of the answer!

shinchane101 · **#52**

Thanks for sharing this with us.

Humans vs. Alpha Go Zero

Who is online