Seeking Input on Rating System - New Server

[illluck]

This is a question for FunNode (viewtopic.php?f=9&t=7603). Ravi is trying to find a rating system that would allow quick adjustment of new players' ratings. The current system is Elo with max rating change K = 32 which makes it quite tedious to reach a reasonable rating for those who are considerably stronger/weaker than the 1000 initial rating.

The main constraint here is that he is trying to keep some sort of consistency with the other games hosted and so isn't too thrilled about self-set ratings (though if there's no other reasonable choice I'd imagine it can be used).

Currently the suggestion seems to be a confidence-based scheme where the rating changes depend on the rating certainties of the two players (different K values for each player), but as none of us are all that knowledgeable in rating systems we'd like additional input. Is a confidence based system a reasonable choice in this case? If so, what should be the range and determinants of the K value?

Many thanks in advance

[SmoothOper]

This is a question for FunNode (viewtopic.php?f=9&t=7603). Ravi is trying to find a rating system that would allow quick adjustment of new players' ratings. The current system is Elo with max rating change K = 32 which makes it quite tedious to reach a reasonable rating for those who are considerably stronger/weaker than the 1000 initial rating.

The main constraint here is that he is trying to keep some sort of consistency with the other games hosted and so isn't too thrilled about self-set ratings (though if there's no other reasonable choice I'd imagine it can be used).

Currently the suggestion seems to be a confidence-based scheme where the rating changes depend on the rating certainties of the two players (different K values for each player), but as none of us are all that knowledgeable in rating systems we'd like additional input. Is a confidence based system a reasonable choice in this case? If so, what should be the range and determinants of the K value?

Many thanks in advance

I would do something like a runs based scan statistic with a poison assumption. IE if a player is playing other players at specific rank, is the proportion of games that the player is winning significantly different from 1/2 if so then the player is either higher or lower than the rank they are playing at, the reason it needs to be a scan statistic is so that you can compare windows of games that are of different lengths so that both incremental improvement over long periods of time and dramatic improvement with many wins in a short period will be rewarded. Maybe like Karlin and Altschul statistics.

[gurujeet]

The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet

[SmoothOper]

The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet

Hmm... Normal distribution for modeling, players rating, explains much about the AGA.

[Kirby]

The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet

Hmm... Normal distribution for modeling, players rating, explains much about the AGA.

Why don't you like the normal distribution in this case? I'm kinda interested in this topic.

[Dusk Eagle]

Well, I wouldn't go so far as to say it explains a lot about the AGA, but as for why a normal distribution could be considered a bad choice, take a look at [sl=KGSRankHistogram]this sensei's page on KGS's rank distribution[/sl]. This curve is quite heavily left-skewed.

[badukJr]

But the AGA rating distribution is also skewed similarly.

http://senseis.xmp.net/?AGARatingHistogram

Maybe someone is defining normal distribution poorly

[Kirby]

But the AGA rating distribution is also skewed similarly.

http://senseis.xmp.net/?AGARatingHistogram

Maybe someone is defining normal distribution poorly

Maybe we just need more ranks

[yoyoma]

The graph Dusk Eagle pointed to is the distribution of player's ranks. I think the cumulative normal distribution in the AGA ratings is related to the probability of one player beating another, given the difference in their rating. So if the difference is 0, its 50-50. And it moves away from 50-50 in a way defined by that distribution.

Picking normal vs some other distribution would involve doing some studies on which one fits the data the best I guess. I don't know much about that. I don't know how a Poisson fits in to all this, ratings are usually real numbers so it's not necessary to use a discrete probability distribution. Maybe SmoothOper prefers some method different from what Elo and Glicko use.

[vash3g]

Using the AGA's rank system I logged into the database and took this screenshot of the current rank distribution. I should point out that there is a large skewed due to so many at go congress artificially raising them to that point (most are not 1d). I will also point out that this system has been in place and has been tweaked slightly for the last 20 years or so. The math and other information on the background behind it can be found @ http://www.usgo.org/ratings


Full Sized Image

[SmoothOper]

The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet

Hmm... Normal distribution for modeling, players rating, explains much about the AGA.

Why don't you like the normal distribution in this case? I'm kinda interested in this topic.

In this case there is no guarantee that there will be a significant difference between the ranks.

[illluck]

It feels like the thread got a bit derailed (in fact, I reported SmoothOper's comment about AGA last night), but it's actually shaping out to be quite interesting, thanks to all for taking the time to respond. However, sorry if I wasn't clear, but the main question is about a way to let new users' ratings adjust quickly to "correct" levels.

[pwaldron]

Hmm... Normal distribution for modeling, players rating, explains much about the AGA.

The normal distribution is for a single player's performance around his true rating. The AGA system makes no assumptions about the distribution of ratings within the population as a whole.

[HermanHiddema]

One system to consider is to use the first X games (say 15) as a basis for the initial rating. As long as the player has fewer than X games, calculate his rating as the rating that would not change given their score and their average opponent's rating. As soon as they have X games, their rating calculated as such is used as their starting rating, and from then on just use the usual system. Note that the calculation of this temporary rating is unstable as long as the player has only wins, or only losses. To fix that, add an additional virtual "game zero" with a draw against rating 1000 (this also elegantly results in a temporary rating of 1000 for players with no results yet.)