It is currently Sat May 08, 2021 10:07 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 140 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next
Author Message
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #101 Posted: Tue Jan 28, 2020 3:26 pm 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
And while I'm complaining about some rating drift at the higher end of the EGF rating system, that's nothing compared to the mess that can be found in the kyu range (in general, not EGF in particular):
Image.

(image from https://www.emptytriangle.com/archive/26)


This post by gennan was liked by: ez4u
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #102 Posted: Tue Jan 28, 2020 3:31 pm 
Gosei

Posts: 1507
Liked others: 818
Was liked: 500
Rank: AGA 3k KGS 1k Fox 2d
GD Posts: 61
KGS: dfan
Yikes, do we we really have to make fun of the pronunciation and grammar of people for whom English is not their native language?

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #103 Posted: Tue Jan 28, 2020 3:35 pm 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
dfan wrote:
Yikes, do we we really have to make fun of the pronunciation and grammar of people for whom English is not their native language?

That was not my intention. I just posted it for the irony of the situation, not to make fun of the Japanese boy or to offend anyone.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #104 Posted: Tue Jan 28, 2020 11:39 pm 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
xela wrote:
Back to what started this conversation --

shimari65 wrote:
Yes, KGS rankings seem to be pretty far off, and have not been closely tied to appropriate anchors in a very long time. I am trying to gradually tweak the system, without making major shockwaves. Our goal would be for KGS ranks to align more closely with AGA ranks. There are so many servers, and such wide rank variations among them, that picking a standard is very hard. The merit of AGA ranks is that we have extensive records of in person play for hundreds of individuals.

If you find your rank, or someone else's has changed in a way that is totally irrational, let me know here. I can't promise to fix anything, but my actions may cause unintended fluctuations, and knowing about them can help me to make better decisions in the future.

_________________
Paul Barchilon,
AGF Vice President
KGS Manager


If the AGA can put resources behind this, it strikes me that this would be a great topic for a Kaggle competition. Can you publish a big file with the (anonymised/de-identified?) results of all games played in 2019 plus some metadata? Metadata might include handicap, komi, time settings, AGA ratings of the players where known, exact date and time when the game was played, which country/region people were logged in from if known.

I suspect the correlation between AGA and KGS rank would be weak, as some people play better online than over the board, some people the other way round, and different people take online games more or less seriously.

I also wonder whether you'd find cliques in the KGS players. For instance, group A plays mostly fast games, group B plays mostly slow games, they hardly ever play against each other, and 2k in group A is a different strength from 2k in group B. Or it could split up by time of day, country, or something else.

The goals of the Kaggle competition could be to design a better ranking system from scratch, or to suggest how to improve the current system without radical change, or to explain the various factors making ranks (appear to be) unstable or inconsistent.

Does anybody know if such historical data from KGS and/or AGA is accessible for rating system nerds like me?

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #105 Posted: Wed Jan 29, 2020 1:31 am 
Lives in sente
User avatar

Posts: 1235
Liked others: 98
Was liked: 401
gennan wrote:
Then there is this 5k player in his club whose skill didn't change and his rating didn't change either (even though he also plays tournaments, a lot actually, but those are mostly even games against other 5k players). His rating is not affected by the general contraction to 5k.

Since the skill gap between him and the (former) 2d remained the same in reality, the (former) 2d can still give the 5k a handicap of 6-7 stones when they play. And they play each other regularly.
So the 2d's demotion can have a domino effect in his club. The 5k knows he still needs 6-7 stones handicap and he may start to doubt his own rank. After all, the dan rank of his opponent has more weight than his own kyu rank. He may even feel some pressure to demote himself. This is what I meant with "pushing everybody below downward".


Did you actually observe people who play in a lot of tournaments and demote themselves? Personally I've never seen that.

Some people may register at a rank below their real strength, especially those who haven't participated in a tournament for a long time, but other people register at an unrealistically high rank, like in the cartoon you posted, so the two effects might cancel each other.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #106 Posted: Wed Jan 29, 2020 6:00 am 
Lives in gote

Posts: 430
Location: Adelaide, South Australia
Liked others: 173
Was liked: 224
Rank: Australian 2 dan
GD Posts: 200
jlt wrote:
Did you actually observe people who play in a lot of tournaments and demote themselves? Personally I've never seen that.

Yes, I've seen that. If someone notices that they've been losing all their games in recent tournaments, they sometimes conclude that they should be playing at a different rank.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #107 Posted: Wed Jan 29, 2020 9:19 am 
Gosei

Posts: 1507
Liked others: 818
Was liked: 500
Rank: AGA 3k KGS 1k Fox 2d
GD Posts: 61
KGS: dfan
In the US, AGA tournament rules do not allow someone to enter a tournament at a lower rank than their AGA rank (see https://www.usgo.org/sites/default/file ... ns2014.pdf, rule VI.A.1.d).

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #108 Posted: Wed Jan 29, 2020 3:21 pm 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
Did you actually observe people who play in a lot of tournaments and demote themselves? Personally I've never seen that.

I've seen it a lot actually. Looking at the top 25 of the Dutch rating list(http://www.europeangodatabase.eu/EGD/createalleuro3.php?country=NL&dgob=false), I already spot 8 players who demoted themselves in recent years, even though their previous higher rank was officially awarded by the Dutch ranking commission. It's just that most have seen their rating go down slowly over the years and at some point they give in to the rating system (or maybe they stop playing in tournaments).

Some examples from active strong players in the Netherlands:
http://www.europeangodatabase.eu/EGD/Player_Card.php?&key=10298596
http://www.europeangodatabase.eu/EGD/Player_Card.php?&key=10233861
http://www.europeangodatabase.eu/EGD/Player_Card.php?&key=10501216
http://www.europeangodatabase.eu/EGD/Player_Card.php?&key=10986514
http://www.europeangodatabase.eu/EGD/Player_Card.php?&key=10486410

I'm sure there are many cases like this in other countries and at all levels.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #109 Posted: Thu Jan 30, 2020 12:24 am 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
dfan wrote:
In the US, AGA tournament rules do not allow someone to enter a tournament at a lower rank than their AGA rank (see https://www.usgo.org/sites/default/file ... ns2014.pdf, rule VI.A.1.d).

I skimmed those AGA Tournament regulations. I think this part is interesting for the discussion:
Quote:
Appendix F: Re Self Promotion and Assignment of Provisional
Ratings

The TD has broad discretion to assign players to appropriate ratings for the event. Players who feel their
ratings are lower than their actual playing strength often request to be assigned a higher rating for the
purposes of pairing the tournament. The TD may and should take any evidence into consideration to
determine whether or not to grant the request. The TD's discretion in this matter is absolute. Any
adjustment (even less than a whole stone) upward is permitted. Downward adjustment will be made only
in a clearly demonstrable case of disability, such as a stroke. Lack of recent practice no matter how long,
is insufficient.
The TD has the same discretion regarding players who do not yet have an AGA rating. Ratings assigned
in other systems are the ideal gauge, but results from casual play reported either by the player or others
may also be used.
If the initial estimate turns out to be clearly wrong after a round or two, the TD may make an adjustment to
provide appropriate competition for the remainder of the tournament.

So an AGA Tournament Director (TD) has the power to bump a players' ratings upward by any (small) amount that he considers appropriate.

The EGF doesn't have such regulations about ranks and the rating system. Instead, each national federation has its own regulations (some even have their own rating systems).
But regardless of national regulations, an EGF TD doesn't have many options to change a player's EGF rating.
1. He can set the intitial rating of an unrated player (in many cases this will just be the declared rank of the player, but there is a lower limit of 20k.
2. He can adjust a rated player's rating by bumping a player's rank upward by 2 ranks from his previous highest rank in the tournament file upload. This will bump the player's rating to the higher rating.

The reason for triggering this bumping mechanism (called a rating reset) will differ in different countries. I think self-promotion is usually accepted up to a certain rank. In the Netherlands, self-promotion is allowed up to 1k (higher promotions are awarded by a committee, but those are almost always by 1 rank only, not triggering a rating reset). I think in Belgium and France the self-promotion limit is lower, but I don't know the exact limit (10k or so?). In Germany there is no limit AFAIK (it's always self-promotion).

So the EGF system has no mechanism to bump a player's rating by a small amount. So after their first tournament, players generally have to "earn" all their rating points by taking them from their peers, unless a player's strength increases so quickly that a 2 rank promotion is warranted (but still, the TD may not allow the triggering of a rating reset depending on national federation regulations).

I suppose this difference in rating bumping policies could make higher EGF ratings tougher than their equivalent AGA ratings, even though the AGA rating system has higher winrate demands. But it still depends on the frequency by which AGA TDs use their power to bump players' ratings in practice. Is it common or exceptional?

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #110 Posted: Thu Jan 30, 2020 1:02 am 
Lives in sente
User avatar

Posts: 1235
Liked others: 98
Was liked: 401
gennan wrote:
Looking at the top 25 of the Dutch rating list(http://www.europeangodatabase.eu/EGD/createalleuro3.php?country=NL&dgob=false), I already spot 8 players who demoted themselves in recent years, even though their previous higher rank was officially awarded by the Dutch ranking commission. It's just that most have seen their rating go down slowly over the years and at some point they give in to the rating system (or maybe they stop playing in tournaments).


If I understand correctly, the self-demotion refers to the blue line ("declared rank" = Dutch ranking system), while the red line (GoR) continues to move smoothly? And you suppose that self-demotions w.r.t. the blue line occur at all levels and in several countries?

Assuming that's the case, then I agree that these self-demotions deflate national rankings (blue line), but I don't think this deflates the GoR for players weaker than 5k.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #111 Posted: Thu Jan 30, 2020 1:07 am 
Lives in sente
User avatar

Posts: 1235
Liked others: 98
Was liked: 401
gennan wrote:
In the Netherlands, self-promotion is allowed up to 1k (higher promotions are awarded by a committee, but those are almost always by 1 rank only, not triggering a rating reset). I think in Belgium and France the self-promotion limit is lower, but I don't know the exact limit (10k or so?).


I don't know about Belgium, but in France you cannot decide by yourself to self-promote. On the other hand, a "reevaluation commission" can promote players, usually quickly improving young players or players who haven't participated in a tournament for several years and who have improved a lot since then.

http://ffg.jeudego.org/echelle/liste_jo ... values.php

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #112 Posted: Thu Jan 30, 2020 2:03 am 
Gosei

Posts: 1361
Liked others: 110
Was liked: 283
Oh, I thought that in France you can self promote up to the rank of 4 kyu. When did that change?

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #113 Posted: Thu Jan 30, 2020 2:21 am 
Gosei

Posts: 1361
Liked others: 110
Was liked: 283
gennan wrote:
The EGF doesn't have such regulations about ranks and the rating system. Instead, each national federation has its own regulations (some even have their own rating systems).


I think the EGF should really make the effort to change this situation. What is wrong with asking all its countries to agree to a standard system. If all the results are going to the same database, wouldn't it make sense to have results regulated in the same fashion? It's like every country has their own Brexit. Some countries don't even seem to know how to submit results to the database.

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #114 Posted: Thu Jan 30, 2020 4:33 am 
Lives in sente
User avatar

Posts: 1235
Liked others: 98
Was liked: 401
Javaness2 wrote:
Oh, I thought that in France you can self promote up to the rank of 4 kyu. When did that change?


Apparently in 2010: http://ffg.jeudego.org/informations/off ... 100508.pdf

see page 8 "Création d'une commission permanente de réévaluation" but I wasn't there so can't confirm.


This post by jlt was liked by: Javaness2
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #115 Posted: Thu Jan 30, 2020 1:49 pm 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
jlt wrote:
gennan wrote:
Looking at the top 25 of the Dutch rating list(http://www.europeangodatabase.eu/EGD/createalleuro3.php?country=NL&dgob=false), I already spot 8 players who demoted themselves in recent years, even though their previous higher rank was officially awarded by the Dutch ranking commission. It's just that most have seen their rating go down slowly over the years and at some point they give in to the rating system (or maybe they stop playing in tournaments).


If I understand correctly, the self-demotion refers to the blue line ("declared rank" = Dutch ranking system), while the red line (GoR) continues to move smoothly? And you suppose that self-demotions w.r.t. the blue line occur at all levels and in several countries?

Assuming that's the case, then I agree that these self-demotions deflate national rankings (blue line), but I don't think this deflates the GoR for players weaker than 5k.


The blue line is the self-declared rank (but for 1d+ it is in fact determined by the Dutch ranking commission). The red line is the EGF rating computed by tournament results. What you see is that first the red line drops below the lower bound of their official rank. When it stays lower for a long time, the player eventually gives in and requests a demotion to a lower rank by the ranking commission. When the rating is indeed lower than the lower bound of their rank for some years, the ranking commission usually grants the demotion.


Last edited by gennan on Thu Jan 30, 2020 1:54 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #116 Posted: Thu Jan 30, 2020 1:51 pm 
Lives in gote

Posts: 408
Location: Netherlands
Liked others: 209
Was liked: 112
Rank: EGF 3d
Universal go server handle: gennan
Javaness2 wrote:
gennan wrote:
The EGF doesn't have such regulations about ranks and the rating system. Instead, each national federation has its own regulations (some even have their own rating systems).


I think the EGF should really make the effort to change this situation. What is wrong with asking all its countries to agree to a standard system. If all the results are going to the same database, wouldn't it make sense to have results regulated in the same fashion? It's like every country has their own Brexit. Some countries don't even seem to know how to submit results to the database.


I think it will be hard to find a uniform system that all national federations will accept. There is a lot of variation and I expect that every national federation thinks their system is better than all of the others.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #117 Posted: Tue May 05, 2020 5:08 am 
Tengen

Posts: 5324
Liked others: 0
Was liked: 726
Today (2020-05-05), the effect of my opponent (with equal rank) losing his (even) game was that his rating-rank increased. Any rating system allowing such anomalies is fundamentally flawed. (Not to mention days with 80 or 90% wins not being enough to increase one's rank.)

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #118 Posted: Tue May 05, 2020 7:51 am 
Lives in sente
User avatar

Posts: 1235
Liked others: 98
Was liked: 401
How do you know this is the "effect" of losing his game?

Ratings continuously fluctuate, whether you play or not. Maybe he was n+0.99 dan, the game made him lose 0.02 dan but fluctuations made him gain 0.04 dan?

Three days ago I played a non-rated game on KGS. At the beginning of the game I was 3k, in the middle game 2k, and after the game back to 3k.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #119 Posted: Tue May 05, 2020 8:05 am 
Tengen

Posts: 5324
Liked others: 0
Was liked: 726
I know but it makes no sense. Ratings should be updated often enough so that anomalies do not occur.

Top
 Profile  
 
Offline
 Post subject: Re: KGS Ranking adjustment?
Post #120 Posted: Tue May 05, 2020 8:44 am 
Lives in gote

Posts: 500
Liked others: 1
Was liked: 151
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
I don't see why you consider these things "anomalies". The goal of a rating system is to estimate your true rating. With KGS rating system, each time you play against an opponent or each time a previous opponent has his "true rating evaluation" reevalued, your "true rating evaluation" gets reevalued.

Meaning it's because the rating is reevaluated more often than just after a game and with more information than just the last game that you see these "anomalies"

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 140 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group