Humans vs. Alpha Go Zero

[Pippen]

Since AGZ (or put any other AI into) does not play thru Brute-Force, it also means that it is imperfect. So we should be able to beat it due to plain statistics by playing e.g. 1.000 pro's against it we should be able to get 1-2 wins at least. What do you think? Wouldn't that be a cool project? (Could this project be also made in Chess where the situation is quite the same?)

[And]

of course not! it's like if all people throw a ball at the moon and one ball will hit!

[bogiesan]

Michael Redmond 9P and Chris Garlock AGA analyze a long series of pro v A games on the YouTubes. Doesn't seem to be any chance a highly skilled player beating the machine. Ever.

When pros work together, does their go appear to be played better than any pro alone or does it, as in many cooperative creative endeavors where conflicting priorities and styles can neither be easily nor quickly resolved, actually add up to what might present on the board as the product of less than one?

[Pippen]

of course not! it's like if all people throw a ball at the moon and one ball will hit!

This is different because the winning probability would be 0.0. But in Go we always have winning probabilities below 1 and above 0 (for quite some parts of the game) which means by the law of large numbers that if you can summon enough Go players it should lead to some defeats of the AI machines. In fact I'd believe that if you summom 1.000 pros you'd beat AGZ 1-2 times. Don't u think?

[And]

calculate the probability if:
AlphaGo Lee 4:1 against Lee Sedol
AlphaGo Master 60:0 against professional players
AlphaGo Zero 100:0 against AlphaGo Lee
AlphaGo Zero 89:11 against AlphaGo Master
https://en.wikipedia.org/wiki/AlphaGo_Zero

[And]

A former world champion of the game Go says he's retiring because AI is so strong: 'Even if I become the No. 1, there is an entity that cannot be defeated'
https://www.businessinsider.com/deep-mi ... ?r=US&IR=T

[And]

if all the pros play all the time, they will never win

[ez4u]

of course not! it's like if all people throw a ball at the moon and one ball will hit!

This is different because the winning probability would be 0.0. But in Go we always have winning probabilities below 1 and above 0 (for quite some parts of the game) which means by the law of large numbers that if you can summon enough Go players it should lead to some defeats of the AI machines. In fact I'd believe that if you summom 1.000 pros you'd beat AGZ 1-2 times. Don't u think?

If you were correct, why would that be interesting?

[WriterJon]

Michael Redmond 9P and Chris Garlock AGA analyze a long series of pro v A games on the YouTubes. Doesn't seem to be any chance a highly skilled player beating the machine. Ever.

You have a link to these videos? They sound pretty cool!

[gennan]

Deepmind's reported Elo ratings for different AlphaGo versions seems to be fairly compatible with the pro Elo ratings reported at https://www.goratings.org/en/.

So we may estimate the Elo difference between AGZ (5200 Elo) and Shin JinSeo (3800 Elo) to be about 1400 Elo. That would mean Shin JinSeo has about 0.03% probability to win. So if you clone him into an army of Shin Jinseos, that army would need to play about 3000 games against AGZ to win 1 game.

If cloning is not allowed and you have to settle for "average" pros with an Elo ratings in the range 2800-3200, the pro army would be about 2400-2000 Elo weaker than AGZ. Such pros would have winning probabilities ranging from 0.0001% to 0.001%. So that "average" pro army would need to play 100,000 - 1,000,000 games against AGZ to win 1 game.

[lightvector]

I know gennan and many regulars are aware of this, but for casual forum-browsers who haven't seen other posts on it: Usually whenever you talk about anything like 1000 Elo or more corresponding to winning chances, all conclusions you make are pure extrapolation and there little or no evidence on whether it is accurate. It's close to just pulling guesses out of a hat for fun. Which can be still fun.

The Elo model is a simplistic model - the idea that you can take ratings differences between successive different players, simply add them up, and that will give you a guess of the winning chance between players you've never actually played before. Or equivalently, that you just multiply the odds. There's no reason why such a model has to work, and the real world is under no obligation to behave like this. It's the other way around - you calibrate the model until it produces not terribly-unreasonable answers for moderate differences, like up to maybe as much as 400 or 600 Elo, and it's anybody's guess outside the situations you calibrated your model for.

If A is, for example, 1600 Elo weaker than E in rating system, we're just joking and having fun to say that corresponds to some winning odds like 1:10000. In reality what it means is closer much closer something like: we think A has 1:10 odds against B, who we think has 1:10 odds against C, who we think has 1:10 odds against D who we think has 1:10 odds against E. Whether that actually means you can just multiply the odds, and say A has 1:(10 * 10 * 10 * 10) odds against E in some game and have that be remotely accurate - usually nobody has any evidence for that.

[And]

when there is such a difference in strength, it might not be enough to just count the probability? go game is not roulette

[gennan]

Lightvector is right ofcourse. It's hard to accurately measure large skill gaps by even game win% and the results may not even be very meaningful.

But I think go has a nice feature to measure large skill gaps: Determine the handicap needed to achieve about 50% winrate. One can use handicap stones and/or komi handicap for that.

There are some special addition rules for handicap, like: player A needs 2 stones against player B (3 half ranks difference), who needs 2 stones against player C, who needs 2 stones against player D, so player A needs 5 stones against player D (9 half ranks difference). In practice, this handicap system seems to hold up fairly well to measure skill gaps that would translate to quite large Elo gaps.

This method has been used to measure the skill gap between pros and AI, though only in casual games. From the casual games that I saw, it looks like pros usually need 3 stones handicap against a good AI.

[And]

@gennan
Let's suppose, in theory, that you are playing against 5 kyu, playing at full strength every game. do you agree that he will never win? but if you calculate the probability, it is not equal to 0. If you are not sure about 5 kyu, then 10 kyu

[And]

Is it always if network A is stronger than network B, and network B is stronger than network C, then network A is stronger than network C?

The paradox of transitivity
Szekely G.J. "Paradoxes in Probability Theory and Mathematical Statistics"