It is currently Thu May 06, 2021 6:00 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 92 posts ]  Go to page Previous  1, 2, 3, 4, 5
Author Message
Offline
 Post subject: Re: How to make KGS better?
Post #81 Posted: Tue Dec 29, 2020 11:50 am 
Lives in gote

Posts: 400
Location: Netherlands
Liked others: 200
Was liked: 110
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
gennan wrote:
A non perfect player can never beat a (hypothetical) perfect player,


I disagree. A non perfect player has a tiny but nonzero chance of playing a perfect game. In a 300-move game, a random player has about 1/(361 x 359 x 357 x ... x 63) ~ 10-788 chances of playing the whole game perfectly.


You're right ofcourse. It would not be 0% but an exceedingly small number. So the perfect player would not have a rating of infinity, but it would be a very large number. But for all practical purposes, the perfect player would probably win all of the games that it can play before the heat death of the universe (I'm more a physicist than a mathematician).


Last edited by gennan on Tue Dec 29, 2020 11:54 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #82 Posted: Tue Dec 29, 2020 11:53 am 
Oza

Posts: 2347
Location: Ireland
Liked others: 661
Was liked: 431
Universal go server handle: Boidhre
Thank you (yes, consistency was unclear, I meant more consistent = higher expected win% against R-1 opponent). That graph is what I'm interested in and the rate on increase in elo gain per rank or 100 GoR and how well the EGF statistics match with the predictions from the EGF's rating model. What interests me most is the relationship between this and handicap stones and how the value of a handicap stone changes as players are stronger. How we have this discrete linear idea of strength difference on the board compared to the ELO differences shown in winrates. If that makes any sense. :)

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #83 Posted: Tue Dec 29, 2020 12:17 pm 
Lives in gote

Posts: 400
Location: Netherlands
Liked others: 200
Was liked: 110
Rank: EGF 3d
Universal go server handle: gennan
Boidhre wrote:
What interests me most is the relationship between this and handicap stones and how the value of a handicap stone changes as players are stronger. How we have this discrete linear idea of strength difference on the board compared to the ELO differences shown in winrates. If that makes any sense. :)

There are other rating systems that rate players on an Elo scale, such as https://www.goratings.org/en/. This is fine if you are only interested in ordering players by "skill", but the drawback of an Elo scale is that you cannot easily determine proper handicaps from rating gaps.
The EGD was specifically designed to rate players on a rank scale (based on handicaps), because that is the traditional "rating" scale used in go.

From my earlier Elo gap per rank table, you can say that the value of a full handicap stone is greater as players get stronger: 200 Elo for a 6d, 100 Elo for a 1d and 50 Elo for a 10k.
The same is true for handicap in chess. Apparently, the value of knight-odds is about 600 Elo at a rating of 2400, but only about 200 Elo at a rating of 1100.

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #84 Posted: Tue Dec 29, 2020 12:58 pm 
Lives in gote

Posts: 400
Location: Netherlands
Liked others: 200
Was liked: 110
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
In a 300-move game, a random player has about 1/(361 x 359 x 357 x ... x 63) ~ 10-788 chances of playing the whole game perfectly.

Per Elo gap of 2000, the winning probability of the weaker player decreases by a factor of about 105.
So when we put a random player at 0 Elo and use 10-788 as the probility of playing a perfect game, I get that a perfect player has a rating of about 2000 * 788 / 5 ~ 300,000 Elo on a 19x19 board.

This high number is still hard to put in context though. For purposes of handicap and ranks, I think an estimate of 13d EGF for a perfect player is much more meaningful. Also for AlphaGo Zero, for which Deepmind reported a rating of about 5200 Elo above a random player, I think an estimate of 12d EGF is much more meaningful.

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #85 Posted: Tue Dec 29, 2020 1:15 pm 
Lives in sente
User avatar

Posts: 1234
Liked others: 98
Was liked: 401
I made a mistake (confusion between ln and log10), it's 10-342 instead of 10-788 so a perfect player is about 140000 Elo above random play. That looks a huge number as we are used to thinking of Alphago as near-perfect, but what do we know about superhuman play? Maybe in 2050, Alphago will look very weak compared to more recent IAs.

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #86 Posted: Tue Dec 29, 2020 1:34 pm 
Lives in gote

Posts: 400
Location: Netherlands
Liked others: 200
Was liked: 110
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
I made a mistake (confusion between ln and log10), it's 10-342 instead of 10-788 so a perfect player is about 140000 Elo above random play. That looks a huge number as we are used to thinking of Alphago as near-perfect, but what do we know about superhuman play? Maybe in 2050, Alphago will look very weak compared to more recent IAs.

Thanks for the correction.

So the Elo ratings of AI still have a lot of room for improvement. The gap between 2050 AI and 2020 AI could be as large as 20,000 Elo in even games with perfect komi, but I expect that in terms of handicap, the gap between 2050 AI and 2020 AI will be less than 10 points (so 2020 AI can still win when taking black and a few points reverse komi against 2050 AI).

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #87 Posted: Tue Dec 29, 2020 2:46 pm 
Oza

Posts: 2347
Location: Ireland
Liked others: 661
Was liked: 431
Universal go server handle: Boidhre
gennan wrote:
There are other rating systems that rate players on an Elo scale, such as https://www.goratings.org/en/. This is fine if you are only interested in ordering players by "skill", but the drawback of an Elo scale is that you cannot easily determine proper handicaps from rating gaps.
The EGD was specifically designed to rate players on a rank scale (based on handicaps), because that is the traditional "rating" scale used in go.

From my earlier Elo gap per rank table, you can say that the value of a full handicap stone is greater as players get stronger: 200 Elo for a 6d, 100 Elo for a 1d and 50 Elo for a 10k.
The same is true for handicap in chess. Apparently, the value of knight-odds is about 600 Elo at a rating of 2400, but only about 200 Elo at a rating of 1100.


My interest is partially coming from talking to some Irish ddks and weaker sdks recently about how "going up a stone in rank" doesn't mean the same thing going from 15k to 14k as it does going from 2k to 1k. Being able to visualise it as different jumps in Elo might be helpful for people coming from chess or similar backgrounds where Elo or similar is used.

Thanks.


This post by Boidhre was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #88 Posted: Tue Dec 29, 2020 3:26 pm 
Lives in sente
User avatar

Posts: 798
Liked others: 303
Was liked: 131
Rank: German 2 dan
gennan wrote:
Boidhre wrote:
The EGD was specifically designed to rate players on a rank scale (based on handicaps), because that is the traditional "rating" scale used in go.


Well, yes, it was supposed to carry that correlation forward, but I don't see how it could actually work:

1. Players use the rating to determine their rank
2. There are very few handicap games in the data

This means that (1.) the rank is not an input anymore, and (2.) the value of handicap is not an input.

This comes on top of the general problem that rating systems do not get enough data to reach any kind of conclusive answer anyway.

_________________
A good system naturally covers all corner cases without further effort.

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #89 Posted: Wed Dec 30, 2020 4:03 am 
Lives in gote

Posts: 400
Location: Netherlands
Liked others: 200
Was liked: 110
Rank: EGF 3d
Universal go server handle: gennan
Harleqin wrote:
gennan wrote:
The EGD was specifically designed to rate players on a rank scale (based on handicaps), because that is the traditional "rating" scale used in go.

Well, yes, it was supposed to carry that correlation forward, but I don't see how it could actually work:

1. Players use the rating to determine their rank
2. There are very few handicap games in the data

This means that (1.) the rank is not an input anymore, and (2.) the value of handicap is not an input.

This comes on top of the general problem that rating systems do not get enough data to reach any kind of conclusive answer anyway.

Handicaps vs rank gaps
About 10% of the EGD games are handicap games. Indeed that is not really enough to ensure that rank gaps and handicaps stay aligned over time. However, when we analyse those handicap games, the actual results seem to align quite well with expected results. On average the error seems to be less than a stone. So apparently, the EGD managed to keep rank gaps fairly well aligned with handicaps over more than 2 decades. That is encouraging. The system must be doing something right.

Overall rating drift
Besides aligning ranks gaps with handicap, there is also the matter of overall rating drift (inflation/deflation).
1. Before the internet, players would determine their rank by handicap in club games (outside of the EGD). When players entered their 1st tournament, they would declare their club rank at that moment and that would be their initial rating in the EGD.
2. Also, the EGD has a reset mechanism. When a player has improved a lot since their previous tournament (again determined by their current club rank from handicap games), they would declare their new rank on entering the next tournament. If that new rank is more than 1 rank above their previous highest declared rank, the system resets that player's rating to that new rank. So the player is not forced to sandbag and "steal" all their rating points from weaker players.
So the EGD is constantly being fed by club ranks. This is mostly sufficient to keep EGD ranks fairly well aligned to club ranks.

Internet players do complicate this picture a bit, because a new tournament player may have never played in a club when entering their 1st tournament. So they only know their rank on their favourite go server. And the tournament organisers will then guesstimate their equivalent EGF rank to play in the tournament.

The system depends on the right balance of pessimistic and optimistic rank declarations by tournament players to stay well aligned.
When players stop declaring club promotions when playing in tournaments, it becomes difficult to keep EGD ranks aligned to club ranks. If a previous 5k knows he has improved to 3k in their club, but does not update their rank when entering the next tournament, taking away rating points from his opponents to make his own rating rise to 3k, he is effectively sandbagging and causing deflation.

When looking at the EGD data, it does seem that overall, players tend to be pessimistic/conservative, delaying promotion until their EGD rating supports it. There are insufficient optimistic promotions to counter the deflation of pessimistic promotions.

3. So to counter the deflationary effect of improving players that don't promote themselves, we increase the rating bonus in the upcoming EGD update to constantly inject rating points into the system (at least in the kyu range).

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #90 Posted: Thu Dec 31, 2020 7:48 am 
Oza
User avatar

Posts: 2508
Liked others: 1304
Was liked: 1127
SoDesuNe wrote:
b) Offer AI review (should be one which works with all komi variants and handicap of course): Please see lichess.org for this (it's fast, it points out mistakes to learn from, it's free for everyone, it even produces a nice graph to see where your biggest mistakes were made)


Seems by far the thing that would make a go server more attractive to me.

_________________
Patience, grasshopper.

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #91 Posted: Fri Jan 01, 2021 1:23 am 
Lives in sente
User avatar

Posts: 1234
Liked others: 98
Was liked: 401
Isn't that a feature of OGS?

Anyway I don't really see the point of having an AI analysis on a go server, as I can run Katago on my computer after the game.

Top
 Profile  
 
Offline
 Post subject: Re: How to make KGS better?
Post #92 Posted: Fri Jan 01, 2021 6:37 am 
Oza
User avatar

Posts: 2508
Liked others: 1304
Was liked: 1127
jlt wrote:
Isn't that a feature of OGS?

Anyway I don't really see the point of having an AI analysis on a go server, as I can run Katago on my computer after the game.


Hm. So another go server is more advanced, having a feature integrated that you would otherwise need use a separate program for, and therefore KGS should abandon hope of being competitive in that regard?

I personally don't know how to use Katago or Leela and don't really want to learn how to use a separate program to analyze my games. Seems too complicated to me. Imo, if KGS had something comparable to what Lichess or Chess.com offers in the way of an easy-to-use AI that identified mistakes and missed opportunities and gave me the opportunity to learn from my mistakes without needing a degree in computer science, KGS would be a better go server than it currently is.

But that's probably true about many of the suggestions that have been floated. Thanks for reminding me that OGS is still actively developing its product - I'll check it out.

_________________
Patience, grasshopper.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 92 posts ]  Go to page Previous  1, 2, 3, 4, 5

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group