It is currently Thu Mar 28, 2024 8:46 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 143 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 8  Next
Author Message
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #21 Posted: Sun Jan 26, 2020 3:57 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Sorry, I am obviously misunderstanding something. Consider six players

  • Player A: about 6 kyu EGF
  • Player B: 50 Elo stronger than A
  • Player C: 100 Elo stronger than A
  • Player D: 150 Elo stronger than A
  • Player E: 200 Elo stronger than A
  • Player F: 250 Elo stronger than A

According to what you say, one or several of the following statements are false. Which one(s)?

(1) B is about 1 stone stronger than A
(2) C is about 2 stones stronger than A
(3) D is about 3 stones stronger than A
(4) E is about 4 stones stronger than A
(5) F is about 5 stones stronger than A
(6) F is about 1 AGA rank stronger than A.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #22 Posted: Sun Jan 26, 2020 3:59 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
The consequence of these discrepancies between rating systems and reality is this:
  • Promotion of EGF kyu and low dan ranks is too though (so you get rank deflation in that range)
  • Promotion of AGA kyu and low dan ranks is even tougher than EGF (so you get even worse rank deflation in that range)
  • Promotion of KGS high dan ranks is too easy (so you get rank inflation in that range)

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #23 Posted: Sun Jan 26, 2020 4:37 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
It goes against the grain, I suppose, but I have long believed that ranking in terms of handicap stones, despite the fact that skill in giving or taking handicap stones is variable, it better than ELO ratings or score differences in differentiating skill levels. The main reason for my belief is that, over a large range of skill levels, handicap stones have a roughly linear relationship. E. g., if player A is three ranks stronger than player B, that means that a handicap of 3 stones with White giving komi normally gives each player around a 50-50 chance of winning the game. If player B is four ranks stronger than player C, then if A gives C a handicap of 7 stones, White giving komi, that will also normally give each player around a 50-50 chance of winning. Elo ratings, OTOH, will tell us that A will crush C in an even game. :lol:

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by 2 people: Harleqin, zermelo
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #24 Posted: Sun Jan 26, 2020 5:12 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
Sorry, I am obviously misunderstanding something. Consider six players

  • Player A: about 6 kyu EGF
  • Player B: 50 Elo stronger than A
  • Player C: 100 Elo stronger than A
  • Player D: 150 Elo stronger than A
  • Player E: 200 Elo stronger than A
  • Player F: 250 Elo stronger than A

According to what you say, one or several of the following statements are false. Which one(s)?

(1) B is about 1 stone stronger than A
(2) C is about 2 stones stronger than A
(3) D is about 3 stones stronger than A
(4) E is about 4 stones stronger than A
(5) F is about 5 stones stronger than A
(6) F is about 1 AGA rank stronger than A.


I suppose you are asking about true Elo and not mixing it up with rating points of the EGF, AGA or KGS systems.
So first I translate your question to winrates:

  • Player A: about 6 kyu EGF
  • Player B: 57% winrate in even games against A
  • Player C: 64% winrate in even games against A
  • Player D: 70% winrate in even games against A
  • Player E: 76% winrate in even games against A
  • Player F: 80% winrate in even games against A

Also, the data on https://wismuth.com/elo/calculator.html#elo_diff=250 does not contain enough data to answer each of those questions. So I have to consult some other pages, like https://senseis.xmp.net/?EGFRatingSystem and my own pages from 2017 at http://goratings.eu/Probabilities/P_ObservedEGD.
Also, I prefer to talk about ranks rather than stones (the 1st handicap stone is only half a move advantage, so I want to avoid confusion here)

So I would say that everything below is true:

Predicted by KGS
observed winrates form EGF historical data (1 million tournament games over a period of 20 years):

B is about 1.0 rank stronger than A.
C is about 2.0 ranks stronger than A.
D is about 3.0 ranks stronger than A.
E is about 4.0 ranks stronger than A.
F is about 5.0 ranks stronger than A.

Predicted by EGF:
B is about 0.25 ranks stronger than A.
C is about 0.75 ranks stronger than A.
D is about 1.0 rank stronger than A.
E is about 1.5 ranks stronger than A.
F is about 1.75 ranks stronger than A.

Predicted by AGA:
B is about 0.2 ranks stronger than A.
C is about 0.4 ranks stronger than A.
D is about 0.6 rank stronger than A.
E is about 0.8 ranks stronger than A.
F is about 1.0 rank stronger than A.

Now if you ask which system's predictions match observations best, I would say it's KGS (at least around 6k EGF). EGF is pretty far off and AGA is even worse.


Last edited by gennan on Sun Jan 26, 2020 5:21 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #25 Posted: Sun Jan 26, 2020 5:20 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
gennan wrote:
Predicted by KGS and observed winrates form EGF historical data:
(...)
F is about 5.0 ranks stronger than A.
(...)
Predicted by AGA:
(...)
F is about 1.0 rank stronger than A.



So you seem to say that

5 KGS ranks = 1 AGA rank

which is hard to believe...

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #26 Posted: Sun Jan 26, 2020 5:28 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
gennan wrote:
Predicted by KGS and observed winrates form EGF historical data:
(...)
F is about 5.0 ranks stronger than A.
(...)
Predicted by AGA:
(...)
F is about 1.0 rank stronger than A.

Now if you ask which system's predictions match observations best, I would say it's KGS.



So you seem to say that

5 KGS ranks = 1 AGA rank

which is hard to believe...


What exactly is hard to believe?
The AGA system expects a 5k to win 80% of even games against a 6k. I think that's more or less a fact.
In reality, a 6k wins about 57% in even games against a 6k. I think that's also more or less a fact.

The conclusion that 5KGS ranks = 1 AGA rank is only a secondary effect and it's not really working like that in practice.
But it does show that the rating systems that are in use nowadays are not perfect.


Last edited by gennan on Sun Jan 26, 2020 5:47 am, edited 3 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #27 Posted: Sun Jan 26, 2020 5:34 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Also, the EGF and AGA rating systems have mechanisms that kind of reduce the problem: new players entering the system at mid kyu range and rating resets will bring rating points into the sytem, basically correcting the system somewhat to align it better with reality.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #28 Posted: Sun Jan 26, 2020 5:40 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
But that doesn't fit observations in https://idex.github.io/go-rank-survey/g ... mean-ranks

which say that for each n between 1 and 13,

n kyu KGS = (n or n+1 or n+2) kyu AGA.

In other words, KGS ranks and AGA ranks are already quite close to each other throughout the kyu range.

Edit: I hadn't read your last message before posting.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #29 Posted: Sun Jan 26, 2020 5:50 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
It's not exactly that 5 AGA ranks = 1 KGS rank. But it is true that the AGA system has too high expectations on winrates (80% instead of 57% at 5k against 6k over a long time period). So players will tend to be stuck at their rank and promotions in the kyu range are overly delayed by the system, which will lead to deflation of kyu ranks. I would expect improving kyu players' AGA ranks to lag very much behind their KGS ranks, unless there are other promotion mechanisms in the AGA system.


Last edited by gennan on Sun Jan 26, 2020 6:08 am, edited 2 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #30 Posted: Sun Jan 26, 2020 6:01 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Another effect that can result from this is that strong player won't be able to give the nominal handicap to a weaker player. I don't live in the US, but from the AGA rating system characteristics, I would predict that a 3d AGA rated player will have a hard time giving a 6 stone handicap to a 3k-4k AGA rated player. This trend was mentioned as occuring in France. Do you happen to observe such a trend in the US?

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #31 Posted: Sun Jan 26, 2020 6:22 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
I don't know about the US, but I don't think that in France there is a big problem about players stuck at some given rank below their real level. According to

http://ffg.jeudego.org/echelle/echelle_algo.php

(1) A 6k winning a game against a 6k gains about 24 points.

(2) If during a tournament a player gains more that 60 points, then his rating is automatically adjusted prior to the tournament.

So it's not uncommon for a player around that level to gain 2-3 kyus after winning 5/5 games in a tournament.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #32 Posted: Sun Jan 26, 2020 6:28 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Let me put it in another way.

Let a 6k play a 10 game match on even against a 7k. The AGA system expects the 6k to win 8-2 (80%). Anything less and the 6k will lose rating points.

But from historical data, we should expect the match to end in 6-4 (60%) and nobody should gain or lose rating points in that case.

If we look for a player that the 6k can beat 8-2 (80% winrate) in a 10 game match on even, the historical data suggests we should match him against an 11k (5 ranks weaker corresponds to 80% winrate around these ranks).
The match will probably end 8-2 and nobody should gain or lose rating points when that is the case.

But the AGA system would expect a near 100% score, so if the 6k loses 1 or 2 games out of 10 against the 11k (as might be expected historically), he would lose rating points.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #33 Posted: Sun Jan 26, 2020 6:32 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
I don't know about the US, but I don't think that in France there is a big problem about players stuck at some given rank below their real level. According to

http://ffg.jeudego.org/echelle/echelle_algo.php

(1) A 6k winning a game against a 6k gains about 24 points.

(2) If during a tournament a player gains more that 60 points, then his rating is automatically adjusted prior to the tournament.

So it's not uncommon for a player around that level to gain 2-3 kyus after winning 5/5 games in a tournament.


Right, so the FFG has some extra mechanisms to update the ratings besides game results. The EGF also has such mechanisms. And I suppose the AGA has them too. (Even chess rating systems with weaker players have such mechanisms).

These corrective measures reduce the problem of unrealistic winrates by the system.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #34 Posted: Sun Jan 26, 2020 6:49 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
I don't know about the US, but I don't think that in France there is a big problem about players stuck at some given rank below their real level.


The FFG system is closer to the EGF system than the AGA system, so I would expect it to be less of an issue. But still there is this French tournament organiser that seemed to experience issues along the lines of deflation of strong kyu ranks: https://lifein19x19.com/viewtopic.php?p=233885#p233885

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #35 Posted: Sun Jan 26, 2020 7:07 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
That tournament organizer had some impressions, but they are probably not backed up by statistics, so it's hard to say if his impressions are correct or not. In this French tournament

http://ffg.jeudego.org/resultats/affich ... ?idt=14291

games were played at handicap minus one. The winning ratio W:B was:

Handicap 1: 3:1
Handicap 2: 6:2
Handicap 3: 2:2
Handicap 4: 2:5
Handicap 5: 3:0
Handicap 6: 1:1
Handicap 7: 1:0
Handicap 8: 3:3
(I am not taking into account H9 games which were sometimes played against players who were much more than 9 ranks apart).

Overall, White won a bit more than Black, as expected (since handicap minus one was used).


Last edited by jlt on Sun Jan 26, 2020 7:15 am, edited 2 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #36 Posted: Sun Jan 26, 2020 7:09 am 
Gosei

Posts: 1590
Liked others: 886
Was liked: 527
Rank: AGA 3k Fox 3d
GD Posts: 61
KGS: dfan
I have similar opinions about the AGA system, largely based on running the WHR rating algorithm on American Yunguseng Dojang games (there are over 5000 serious even games by now) and seeing how the resulting ratings compare with AGA ones.

There are a few problems that I believe I observe about the rating system. One is that the emphasis on handicap games means that it's hard for the system to be incredibly surprised at any particular result. In a chess tournament, you might be (the equivalent of) 10k and beat a couple of 5ks, and rocket up. If those are 5-stone handicap games, the fact that you won a couple of coin-flips isn't so impressive. Add to this a large amount of inertia in the rating system (and apparently the fact that it imposes unreasonable demands on winrate) and you get a relatively stagnant rating pool.

There is thankfully a pressure relief valve in the form of a rule that if you enter a tournament at least three ranks up from your established rating, and win at least one game, your rating resets entirely. The bad news is that this seems to be by far the most efficient way to get even a modest increase in skill recognized. If you are rated 4k and think you are 3k, I believe that the best way to achieve it is not by regular play but by entering tournaments at 1k until you win one game. That doesn't feel like a healthy system.


This post by dfan was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #37 Posted: Sun Jan 26, 2020 9:29 am 
Gosei

Posts: 1625
Liked others: 542
Was liked: 450
Rank: senior player
GD Posts: 1000
Some of the discussion above agrees with the idea that rating has no inherent meaning. Too many people fixate on raising their rating and we see concern with how to increase rating fast. If I am a 4k player and want to raise my rating to 3k and I do it by repeatedly entering tournaments at self-rated 1k until I win a game against a 1k, am I going to quit playing after my rating increase? Because if my "real" rating is 4k I'll probably lose the artificial 3k rating quickly after resuming tournament play. There is no guarantee that rating will increase over time. There are some time tested ways to improve your play. For example, you can take lessons and work on using what you are learning in tournament games. Or you can just play and analyze your games to see what mistakes you are making. The thing is you have to think about the moves you are making, so, probably, slower games are important, where you can think about your moves; that's how new ideas can be integrated into your play.

Actual ratings are not very meaningful, their purpose is to make it possible to play opponents at the same level, but they do not determine any particular playing strength. The name "3k" describes players of different strengths in the AGA and in the EGF. Even 1p strength is different in USA, Japan, and Korea.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #38 Posted: Sun Jan 26, 2020 9:51 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
The rating is not absolute, but can be useful to measure progress. The whole discussion shows that 1-stone improvements are hard to detect on KGS because KGS ratings are not always based on reliable anchors, and the idea of readjusting KGS ratings according to known AGA ratings is not good either because some AGA players are stuck a few stones below their "real" level (e.g. they are AGA n-kyu but of the same level as the median AGA (n-3)-kyu).

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #39 Posted: Sun Jan 26, 2020 1:02 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
If you see rating only as a means to measure short term progress, then a simple Elo system would be sufficient. But problems arise for long-term players if you assign hard go ranks to ratings and the rating system slowly deflates or inflates.

My country had about 10 6d EGF players 10 years ago. Now they were all forced to demote themselves to 5d, because the rating system says so. Did they all become 1 stone weaker? Perhaps they are a bit weaker now (there seems to be a trend that players get a bit weaker as they age). But some of those players aren't that old. My feeling is that at least part of the reason is deflation. You could ask who cares if 6d in 2007 are forced to be 5d in 2017. But I think it matters. If the EGF rating system deflates over time, long-term players who stopped improving will start to avoid playing in tournaments, because they know it will hurt their rating (=rank!) and thus their percieved status in the go community. I fear deflation has a negative effect on the go community.


This post by gennan was liked by: Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #40 Posted: Sun Jan 26, 2020 2:01 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
That tournament organizer had some impressions, but they are probably not backed up by statistics, so it's hard to say if his impressions are correct or not. In this French tournament

http://ffg.jeudego.org/resultats/affich ... ?idt=14291

games were played at handicap minus one. The winning ratio W:B was:

Handicap 1: 3:1
Handicap 2: 6:2
Handicap 3: 2:2
Handicap 4: 2:5
Handicap 5: 3:0
Handicap 6: 1:1
Handicap 7: 1:0
Handicap 8: 3:3
(I am not taking into account H9 games which were sometimes played against players who were much more than 9 ranks apart).

Overall, White won a bit more than Black, as expected (since handicap minus one was used).

The number of games in your example is much too low for statistical analysis.

You really need large samples of statistical data to distinguish between 65% and 75% winrate with any reliability,.
The EGD data contains about 100.000 handicap games (tournament games in Europe tend to be even games in McMahon tournament system). And if handicaps are used in tournaments, it's usually a reduced handicap, which favours white, obscuring any issues of handicaps being too large. Even "normal" handicaps favor white, as black's 1st handicap stone is only half a move advantage. But it's possible to correct the expected statistics for skewed handicaps.

I did collect those statistics in 2017 and as far as I can tell, over the whole of Europe in a period of 20 years, handicaps seem to be pretty consistent with rank differences. But when I zoom in to smaller samples (like only one country for a period of 5 years), the sample becomes too small for reliable statistics. The "signal" is pretty small and it becomes blurred by the "noise".
For example, the observed winrate between 6k and 7k in even games is about 57%. When the 6k player gives the 7k player 1 handicap, the winrate may be about 53% and with 2 handicap it may be about 45%. You need pretty large samples to detect such small winrate differences reliably.


This post by gennan was liked by: Harleqin
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 143 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 8  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group