Humans vs. Alpha Go Zero

[gennan]

AmiGo 10k
https://github.com/breakwa11/GoAIRatings
and somewhere else it was 11k

I saw that site before and I don't see anything about where those ranks are coming from.

[And]

thank! I want to "simulate" something that looks like a pro - agz match. If you think the option I have chosen is not suitable, which programs are better? what is possible to do on a home pc

[Pippen]

If you were correct, why would that be interesting?

Because it could show if we (humans) still have a (small) chance. It would also be a nice marketing gig for Go and professional Go. I'd love Deepmind to dig out AGZ for such a purpose.

It is not an impossible task. AlphaGo did not solve the game of Go nor does it play by brute force which means there are always chances that a game can slip away from AI and if you increase the amount of Go players to play AGZ you will certainly get some wins. The question is not if, but how many you need. 100? 1000? 10.000? 100.000?

[gennan]

thank! I want to "simulate" something that looks like a pro - agz match. If you think the option I have chosen is not suitable, which programs are better? what is possible to do on a home pc

I don't know. I feel a general issue here that many computer go enthousiasts just post numbers and ranks and it's totally unclear how they arrived at those numbers. My impression is that they just pull some formula out of thin air without testing against a large number of real human players with a somewhat "certified" rank (including handicap games ofcourse).

[gennan]

If you were correct, why would that be interesting?

Because it could show if we (humans) still have a (small) chance. It would also be a nice marketing gig for Go and professional Go. I'd love Deepmind to dig out AGZ for such a purpose.

It is not an impossible task. AlphaGo did not solve the game of Go nor does it play by brute force which means there are always chances that a game can slip away from AI and if you increase the amount of Go players to play AGZ you will certainly get some wins. The question is not if, but how many you need. 100? 1000? 10.000? 100.000?

The gap is so large, that I don't really care if pros need to play 10,000 or 1,000,000 games to win 1 even game against AGZ or KataGo.

I'm much more interested in the komi gap between pros and AI, which is much easier to determine. And you don't even need pros for that. Anyone can determine their komi distance to KataGo (at some standardized number of playouts).

[gennan]

I think the odds of a 10k beating me is many orders of magnitudes greater than the monkeys typing all of Shapespeare's works within a billion years. Occasionally, I make a blunder of 100+ points. I don't know how often, but once in a 100 games or so is not far off, I think. Such odds are not crazy small.

if you make mistakes, you are not playing at full strength. the computer makes no mistakes

If I wouldn't make mistakes, I would be world champion. If a computer makes no mistakes, then how is it possible that Lee Sedol beat AlphaGo once?

We all make mistakes, humans and AI. The size and frequency of those mistakes determines our levels (distance from perfect play).

[AloneAgainstAll]

If you assume that FineArt ver E (currently playing on FGS) is around AGZ strenght, then there is pretty good measure of distance between it and humans. Onishi Ryuhei (not top pro for sure, but pretty strong one, currently in Honinbo League, if you need more info check his GoRatings) played sth like 200 games against FA (2 stones, 0 komi, chinese rules for handi games) - and scored around 10%. Time settings were pretty fast (1 min per move x10 byo, with 1 min basic time), not sure if it is to human or AI advantage, but whatever. Judgement is up to you.

[And]

@gennan
wondering how you calculate the probability of winning. can you show with an example? suppose 1dan vs 9dan

[And]

I think the odds of a 10k beating me is many orders of magnitudes greater than the monkeys typing all of Shapespeare's works within a billion years. Occasionally, I make a blunder of 100+ points. I don't know how often, but once in a 100 games or so is not far off, I think. Such odds are not crazy small.

if you make mistakes, you are not playing at full strength. the computer makes no mistakes

If I wouldn't make mistakes, I would be world champion. If a computer makes no mistakes, then how is it possible that Lee Sedol beat AlphaGo once?

We all make mistakes, humans and AI. The size and frequency of those mistakes determines our levels (distance from perfect play).

I meant the gross mistakes people make.
Lee Sedol is a champion, and it was a ingenious move, unexpected for the program! and after further self-learning, the program responded differently to such a move!

[gennan]

wondering how you calculate the probability of winning. can you show with an example? suppose 1dan vs 9dan

I analyzed the game results in EGF rating system (while preparing the coming EGF rating system update), which has a database of about 1 million real life European tournament games played over 25 years. Those games are annotated with the declared EGF ranks of the players.

EGF rating points are called GoR and they are defined as follows:
Amateur ranks are separated by 100 GoR and 1d EGF has a GoR of 2100. So an 7d EGF is expected to have 2700 GoR and a 10k EGF is expected to have 1100 GoR.
1p is expected to be equal to 7d EGF, so they also get 2700 GoR. Pro ranks are separated by 30 GoR, so 9p is expected to have 2940 GoR.

The strongest players in Europe have about 2800 GoR. These are 8-times European Champion Ilya Shikshin 4p and some former Korean Insei, like InSeong Hwang 8d EGF. (see https://europeangodatabase.eu/egdrc/cre ... country=**)
So we don't really have players as strong as 9d EGF ~ 2900 GoR in Europe. Such high ratings would be in the range of fairly strong pros (around #250 of the world). Shin JinSeo may be 10d EGF ~ 3000 GoR.

From my analysis of the EGF database, I arrived at the following conversions between EGF ratings (GoR) and Elo ratings:

Code: Select all

Elo = -7 * ln(3300 - GoR) * 400 / ln(10) + 10500
GoR = 3300 - exp(((10500 - Elo) / 7) * ln(10) / 400)

The 10500 constant can be chosen freely (because Elo is a relative rating scale), but this value gives a fairly decent match to the pro Elo ratings published at https://www.goratings.org/en/.

So to answer your question:
I assume you mean 1d EGF and 9d EGF. That would be like an even game match between a decent 1d EGF (~1900 Elo) and O Rissei 9p (~3200 Elo). That's a 1300 Elo gap, which suggests that the 1d wil win about 1 in 1800 games.

Ofcourse, this ignores the valid remarks that lightvector made earlier, that probabilities calculated for such large Elo gaps may not match reality very well. Elo seems to work well enough for smaller skill gaps (up to a few 100 Elo), but I'm extrapolating a lot here.

[luigi]

I'm much more interested in the komi gap between pros and AI, which is much easier to determine. And you don't even need pros for that. Anyone can determine their komi distance to KataGo (at some standardized number of playouts).

Does mirror play become a problem with large komi?

[wineandgolover]

I assume you mean 1d EGF and 9d EGF. That would be like an even game match between a decent 1d EGF (~1900 Elo) and O Rissei 9p (~3200 Elo). That's a 1300 Elo gap, which suggests that the 1d wil win about 1 in 1800 games.

I’d like to think playing 1800 games with O'Rissei would be good for two or three stones, thus lowering the odds.m

[Pippen]

I don't agree with you that AI is so far away. It uses almost exclusively heuristics which has always some significant blind spots that can be exploited. Give professionals enough time (not like when AlphaMaster played those 60 pros with fast pace on Tygem) and I claim that in 100 games we get a win in an even game. And if I was not just some random dude but Elon Musk we might have a project at hand.

[wineandgolover]

I don't agree with you that AI is so far away. It uses almost exclusively heuristics which has always some significant blind spots that can be exploited. Give professionals enough time (not like when AlphaMaster played those 60 pros with fast pace on Tygem) and I claim that in 100 games we get a win in an even game. And if I was not just some random dude but Elon Musk we might have a project at hand.

I’d for sure take the other side of that bet. And I said so in a video when LSD beat AlphaGo. I think that was the last even game a reasonably powered AI will ever lose to a human.

Even though I agree that humans are getting better, I suspect AI are getting better even faster. I also suspect that in the not too distant future, someone (a coder, not a player) will come up with a new algorithm, and the question will be beyond all doubt.

[jlt]

One way to guess the percentage of games that a top human pro could win against a strong bot A is to make A play 10000 times against a bot B which is of the same strength than a top human pro. Did someone do that?