It is currently Thu Oct 31, 2024 4:48 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 52 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #21 Posted: Mon Mar 22, 2021 11:59 am 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
gennan wrote:
I think the odds of a 10k beating me is many orders of magnitudes greater than the monkeys typing all of Shapespeare's works within a billion years. Occasionally, I make a blunder of 100+ points. I don't know how often, but once in a 100 games or so is not far off, I think. Such odds are not crazy small.

if you make mistakes, you are not playing at full strength. the computer makes no mistakes

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #22 Posted: Mon Mar 22, 2021 12:07 pm 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
gennan wrote:
And wrote:
@gennan
Let's suppose, in theory, that you are playing against 5 kyu, playing at full strength every game. do you agree that he will never win? but if you calculate the probability, it is not equal to 0. If you are not sure about 5 kyu, then 10 kyu

From my own Elo to EGF rank conversion estimate, I estimate the gap between me and a typical 5k to be about 600 Elo, which would imply a win% of about 5% for them (1 in 20 games).
And I would estimate the gap between me and a typical 10k to be about 900 Elo, which would imply a win% of about 0.5% for them (1 in 200 games).
I don't know if those estimates are realistic, but I don't intend to play very long matches against 5k and 10k players to accurately determine their winning probability.

I did lose an online game last December against a player with an OGS rating hovering around (2021) OGS 3k. And in the past 25 years I have lost several tournament games against 1k players. I just looked up my score against 1k in the EGD and it is 25-7, so 1k has about 20% winrate against me (suggesting a 250 Elo gap, though it might be a bit biased, because I usually only play against 1k in tournaments when they have a good day and I have a bad day).

I think there is always some probability that my opponent wins, no matter how weak they are. Yes, at some point the probability becomes less than 1%, but it's never going to be 0.
For example, I could get a stroke during the game. I don't know the exact odds for such a thing happening, but I think it's at least something like 1 in 1,000,000 games.
That probability is small, but clearly bigger than your example of throwing balls at the moon, because it's physically impossible for even the strongest humans to throw a ball with a velocity close to escape velocity (11.2 km/s) and then I'm even ignoring drag from the atmosphere.

the example with the moon is an allegory, I meant that it is just as unrealistic, and most understood me.
for fun: the probability of a ball hitting the moon is not zero, you did not take into account the possibility of a hurricane passing by a meteorite, which "took the ball with it to the moon", etc. :D

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #23 Posted: Mon Mar 22, 2021 12:18 pm 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
@gennan
I want to play a match of 1000 games AmiGo (~ 11 kyu) - KataGo b6c96-s48921344-d7092247, 1 playout (~ 2 kyu). can you calculate the probability of winning AmiGo?

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #24 Posted: Mon Mar 22, 2021 12:36 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
How did you determine that Amigo is ~11k and that this KataGo instance is ~2k?

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #25 Posted: Mon Mar 22, 2021 12:38 pm 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
gennan wrote:
How did you determine that Amigo is ~11k and that this KataGo instance is ~2k?

https://lifein19x19.com/viewtopic.php?p=261404#p261404
Amigo give a couple of minutes ...

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #26 Posted: Mon Mar 22, 2021 12:42 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
And wrote:
gennan wrote:
@gennan
... but clearly bigger than your example of throwing balls at the moon, because it's physically impossible for even the strongest humans to throw a ball with a velocity close to escape velocity (11.2 km/s) and then I'm even ignoring drag from the atmosphere.

the example with the moon is an allegory, I meant that it is just as unrealistic, and most understood me.
for fun: the probability of a ball hitting the moon is not zero, you did not take into account the possibility of a hurricane passing by a meteorite, which "took the ball with it to the moon", etc. :D

Yes, and there is also quantum tunneling. So indeed that probability is not exactly 0, but it would be astronomically smaller than me losing an even game to a 10k.

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #27 Posted: Mon Mar 22, 2021 12:46 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
And wrote:
gennan wrote:
How did you determine that Amigo is ~11k and that this KataGo instance is ~2k?

https://lifein19x19.com/viewtopic.php?p=261404#p261404
Amigo give a couple of minutes ...


That links to a post where you say "network g170 6 block d7.09M probably plays on 2k.". That post doesn't explain how you determined it.

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #28 Posted: Mon Mar 22, 2021 12:52 pm 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
AmiGo 10k
https://github.com/breakwa11/GoAIRatings
and somewhere else it was 11k

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #29 Posted: Mon Mar 22, 2021 12:57 pm 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
I don't want to argue if it plays 2k. I think something like this. I'm more interested in the outcome of the match. can you calculate the probability?

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #30 Posted: Mon Mar 22, 2021 1:22 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
I doubt if computer programs that score 50% against some level of human players are a good model for human-like play at that level. A nerfed KataGo that scores 50% against 2k players is more like a Formula 1 car with square wheels to make its lap time on a race track similar to a VW Beetle. That doesn't make it the same thing as a VW Beetle.

Also, I expect computer programs to be a lot more deterministic than humans. If one program plays in way that happens to exploit the other's weaknesses, the results will be more skewed than a match between humans of similar levels.

And I don't really know how accurate your estimates are for the level of these programs, so I'll include some uncertaincy for that in my estimate. I would also add some uncertaincy for short time settings (making blunders more likely for both players).

But anyway, for a non-blitz match between a human 11k(±2) and a human 2k(±1), I would guess that the weaker player will score around 1%-5% wins.

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #31 Posted: Mon Mar 22, 2021 1:29 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
And wrote:
AmiGo 10k
https://github.com/breakwa11/GoAIRatings
and somewhere else it was 11k


I saw that site before and I don't see anything about where those ranks are coming from.

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #32 Posted: Mon Mar 22, 2021 1:29 pm 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
thank! I want to "simulate" something that looks like a pro - agz match. If you think the option I have chosen is not suitable, which programs are better? what is possible to do on a home pc

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #33 Posted: Mon Mar 22, 2021 1:32 pm 
Lives in gote

Posts: 677
Liked others: 6
Was liked: 31
KGS: 2d
ez4u wrote:
If you were correct, why would that be interesting? :scratch:


Because it could show if we (humans) still have a (small) chance. It would also be a nice marketing gig for Go and professional Go. I'd love Deepmind to dig out AGZ for such a purpose.

It is not an impossible task. AlphaGo did not solve the game of Go nor does it play by brute force which means there are always chances that a game can slip away from AI and if you increase the amount of Go players to play AGZ you will certainly get some wins. The question is not if, but how many you need. 100? 1000? 10.000? 100.000?

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #34 Posted: Mon Mar 22, 2021 1:36 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
And wrote:
thank! I want to "simulate" something that looks like a pro - agz match. If you think the option I have chosen is not suitable, which programs are better? what is possible to do on a home pc


I don't know. I feel a general issue here that many computer go enthousiasts just post numbers and ranks and it's totally unclear how they arrived at those numbers. My impression is that they just pull some formula out of thin air without testing against a large number of real human players with a somewhat "certified" rank (including handicap games ofcourse).

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #35 Posted: Mon Mar 22, 2021 1:42 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pippen wrote:
ez4u wrote:
If you were correct, why would that be interesting? :scratch:


Because it could show if we (humans) still have a (small) chance. It would also be a nice marketing gig for Go and professional Go. I'd love Deepmind to dig out AGZ for such a purpose.

It is not an impossible task. AlphaGo did not solve the game of Go nor does it play by brute force which means there are always chances that a game can slip away from AI and if you increase the amount of Go players to play AGZ you will certainly get some wins. The question is not if, but how many you need. 100? 1000? 10.000? 100.000?


The gap is so large, that I don't really care if pros need to play 10,000 or 1,000,000 games to win 1 even game against AGZ or KataGo.

I'm much more interested in the komi gap between pros and AI, which is much easier to determine. And you don't even need pros for that. Anyone can determine their komi distance to KataGo (at some standardized number of playouts).

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #36 Posted: Mon Mar 22, 2021 2:13 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
And wrote:
gennan wrote:
I think the odds of a 10k beating me is many orders of magnitudes greater than the monkeys typing all of Shapespeare's works within a billion years. Occasionally, I make a blunder of 100+ points. I don't know how often, but once in a 100 games or so is not far off, I think. Such odds are not crazy small.

if you make mistakes, you are not playing at full strength. the computer makes no mistakes


If I wouldn't make mistakes, I would be world champion. If a computer makes no mistakes, then how is it possible that Lee Sedol beat AlphaGo once?

We all make mistakes, humans and AI. The size and frequency of those mistakes determines our levels (distance from perfect play).

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #37 Posted: Mon Mar 22, 2021 4:53 pm 
Lives with ko

Posts: 127
Liked others: 2
Was liked: 21
Rank: KGS 1d
If you assume that FineArt ver E (currently playing on FGS) is around AGZ strenght, then there is pretty good measure of distance between it and humans. Onishi Ryuhei (not top pro for sure, but pretty strong one, currently in Honinbo League, if you need more info check his GoRatings) played sth like 200 games against FA (2 stones, 0 komi, chinese rules for handi games) - and scored around 10%. Time settings were pretty fast (1 min per move x10 byo, with 1 min basic time), not sure if it is to human or AI advantage, but whatever. Judgement is up to you.

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #38 Posted: Tue Mar 23, 2021 7:02 am 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
@gennan
wondering how you calculate the probability of winning. can you show with an example? suppose 1dan vs 9dan

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #39 Posted: Tue Mar 23, 2021 7:10 am 
Gosei
User avatar

Posts: 1425
Liked others: 205
Was liked: 204
gennan wrote:
And wrote:
gennan wrote:
I think the odds of a 10k beating me is many orders of magnitudes greater than the monkeys typing all of Shapespeare's works within a billion years. Occasionally, I make a blunder of 100+ points. I don't know how often, but once in a 100 games or so is not far off, I think. Such odds are not crazy small.

if you make mistakes, you are not playing at full strength. the computer makes no mistakes


If I wouldn't make mistakes, I would be world champion. If a computer makes no mistakes, then how is it possible that Lee Sedol beat AlphaGo once?

We all make mistakes, humans and AI. The size and frequency of those mistakes determines our levels (distance from perfect play).

I meant the gross mistakes people make.
Lee Sedol is a champion, and it was a ingenious move, unexpected for the program! and after further self-learning, the program responded differently to such a move!

Top
 Profile  
 
Offline
 Post subject: Re: Humans vs. Alpha Go Zero
Post #40 Posted: Tue Mar 23, 2021 2:54 pm 
Lives in gote

Posts: 495
Location: Netherlands
Liked others: 272
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
And wrote:
wondering how you calculate the probability of winning. can you show with an example? suppose 1dan vs 9dan

I analyzed the game results in EGF rating system (while preparing the coming EGF rating system update), which has a database of about 1 million real life European tournament games played over 25 years. Those games are annotated with the declared EGF ranks of the players.

EGF rating points are called GoR and they are defined as follows:
Amateur ranks are separated by 100 GoR and 1d EGF has a GoR of 2100. So an 7d EGF is expected to have 2700 GoR and a 10k EGF is expected to have 1100 GoR.
1p is expected to be equal to 7d EGF, so they also get 2700 GoR. Pro ranks are separated by 30 GoR, so 9p is expected to have 2940 GoR.

The strongest players in Europe have about 2800 GoR. These are 8-times European Champion Ilya Shikshin 4p and some former Korean Insei, like InSeong Hwang 8d EGF. (see https://europeangodatabase.eu/egdrc/createalleuro3.php?country=**)
So we don't really have players as strong as 9d EGF ~ 2900 GoR in Europe. Such high ratings would be in the range of fairly strong pros (around #250 of the world). Shin JinSeo may be 10d EGF ~ 3000 GoR.

From my analysis of the EGF database, I arrived at the following conversions between EGF ratings (GoR) and Elo ratings:
Code:
Elo = -7 * ln(3300 - GoR) * 400 / ln(10) + 10500
GoR = 3300 - exp(((10500 - Elo) / 7) * ln(10) / 400)

The 10500 constant can be chosen freely (because Elo is a relative rating scale), but this value gives a fairly decent match to the pro Elo ratings published at https://www.goratings.org/en/.

So to answer your question:
I assume you mean 1d EGF and 9d EGF. That would be like an even game match between a decent 1d EGF (~1900 Elo) and O Rissei 9p (~3200 Elo). That's a 1300 Elo gap, which suggests that the 1d wil win about 1 in 1800 games.

Ofcourse, this ignores the valid remarks that lightvector made earlier, that probabilities calculated for such large Elo gaps may not match reality very well. Elo seems to work well enough for smaller skill gaps (up to a few 100 Elo), but I'm extrapolating a lot here.


Last edited by gennan on Tue Mar 23, 2021 11:51 pm, edited 2 times in total.
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 52 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group