Hi all,

some time ago I was a bit puzzled by a thread in this forum, where a lot of people seemed to be unhappy with the KGS rating system, especially that the rank will get "stuck" at a certain level. This suurprised me a bit, because the math behind it seemed perfectly resonable to me. I believe a probablistic approach like the one used must be superior to other approaches (at least if the first priority of the rating system is to predict the correct handicap for a game).
I could not really understand what the perceived problem of the rating system is, until I found the page in senseins http://senseis.xmp.net/?HowToGetAlongWithKGSRatingMath, where the author makes the statement that the system works perfectly well, but only if you play regularly. I thought about it for a while, and I believe there is an obvious shortcoming of the rating system (it is so obvious that I am puzzled that it did not seem to come up before, but probably I have just missed it).
The problem is the way the half-life of the games is calculated, and especially that the half life seems to be temporal component:

Without loss of generality, if we just assume that the games are counted with a full weight and dropped from the calculation after a given time:
Assume that someone has played 20 rated games in that period and won 50% (that is 10, so 10 games lost, 10 won). The persons attends a workshop, reads a book, or for whatever else reason suddendly improves considerably. A promotion to the next higher rank now requires 14 straight wins (which would make it 24 won, 10 lost or 70% win rate, required for promotion).
If someone has played 200 rated games and won 50%, then he would require a win streak of 140 straight wins. (Not in reality, because some of the older games would disappear from the calculation. But he still would need considerably more wins than someone who does not play that often).

So obviously the rank gets stuck as a result of playing a lot of games. In effect, the rank will represent a fairly accurate estimation of the current strength of someone who plays not many games, but for someone who plays a lot, the rank will be an estimation of how he played some months ago (when the older games disappear from calculation and the win rate is again accurate). In that regard, the systems fails to be internally consistent, because it will predict wrong handicaps.

Of course older games have to be dropped from the calculation, or ranks will get permanently stuck. But I believe the right way to do this is to calculate the half-life of a game based on the number of games played since, not on the time. The introduction of time into the calculation is really arbitrary and not warranted by any logic. (The only logic might be that if someone has not played in a long time, you would not want to consider him in the calculation. But this might be easily fixed by dropping games older than a given number of days). By using a falf life based on the number of games I do believe the KGS system would probably be the best system one can imagine.

Or did I miss something here?

P.S. I have really posted this out of curiosity. At my current rate of playing games, my rank is never in danger of getting stuck. Well.. it might get stuck because of lack of progress, but certainly not because of the ranking system.

From what I understand, the reason for the time component is that the value of a win is assessed against the opponent's current rank, not the rank they held at the time of a win. If you don't time games out of the ranking system on time, then you could have a player who hasn't played in a few months or years who jumps a bunch of ranks because his opponents have gained in strength since then, instead of having the error rise to the point where they lose their rank altogether.

That said, there is probably a happy medium involving both aspects to some degree.

I am aware of that, this is why in my post I have said that you have to drop older games from the calculation. In the current system, however, this problem is solved at the expense of another one...
EDIT: I just relaized I was not totally aware of that, I thought you had to drop older games games because the rank would get stuck forever otherwise.



There would be: You would just have to calculate a weight based on the number of games played since and a weight based on the days that have passed and take whatever number is smaller.
I think that in practice it would be sufficient to just use a "hard" cutoff for the time. (such as: games older than 90 days are not counted)

The time component is still very important and should stay. The system should put much less confidence in the current rank of someone who played 10 games a month ago than someone who just finished 10 games.

Another problem I see is that while current time-based weight for each game is the same for both players in that game (since their rank is adjusted at the same time), your weight will be different for the two players. This might lead to complications.

Otherwise, I think this is a really great idea. A system where the weight is a product of both the time component and the "number of more recent games" component would have the advantages of both.

Actually, I think it makes sense to just have a strict cut-off for the number of games, instead of exponential decay - combined with exponential decay from the time weight. Just consider the last 40 games or whatever. Surely some number like that is enough for a high-confidence rank? Again, it's not clear what happens when we consider a game where one of the players has since played more games than the cutoff number, but the other one didn't.

1. If that's possible, I have very little sympathy for this hypothetical person and don't believe there is cause for complaints.

2. Just play one rated game a week, everything else free or somewhere else. That's enough to avoid a ?. Don't go on weekend benders of rated games that you won't be able to adjust for later. The rating system will catch up to the extent that it can. If it cannot, then create new account. It shouldn't something be that's happening all the time. If it is, re-read point 1).

Or just play what you think are appropriately handicapped games (you'll have to manually set them and get your opponent to agree to get over their fear that their rank will be adversely affected -- it won't). Then your rank will adjust much quicker without having to do all those other tricks.

Of course that should work in principle. I'd like to hear from people who have had success with that. Maybe it just takes patience because there are so many players who reject almost any challenge unless it meets a fairly narrow range of constraints.

In theory it should be possible to improve by reading books. If it wasn't, then why are there Go-Books? On the other hand: It never worked for me...



That you have to resort to such workarounds shows that the system does not work as it should. I personally prefer to play only rated games, I certainly prefer to keep my account, and it would be nice if the rating system could keep up with such an attitude. (Not that this would be my problem, I play infrequently anyways).

Would it? The KGS help page says: "Once we have this relationship, we can take any player A, and find the probability (prob) that the games they played would have the outcome that they actually did have. This is done by multiplying together the win probability (PA wins) for every game they won and the lose probability (PA loses) for every game they lost. Now, we can treat RankA as a variable, come up with a graph of "probability of all game results" (prob) vs. RankA, and solve to find the rank for A that maximizes prob. This will be the rank assigned to A."

So in my understanding the rank is calculated for each player individually. What would be the problem if the same game had a different weight for the two players? You could still run the calculation the way it is described.

Perhaps one point worth clarifying -- As long as you are playing at a regular pace, it will take the same amount of time for you to promote, regardless of how fast your regular pace happens to be. So either way it might take you 1 or 2 weeks to see the full effect of your new strength, but in both cases it will be 1 or 2 weeks. This is the part that throws a lot of people for a loop.

Also I would have to do some searching but I know wms has mentioned several places that every rating system he has tried with asymmetric ultimately becomes unstable (example: A, B, and C are rated 1k and all play each other and no one else, and on average win 50% of the time against each other....Instead of A, B, and C all staying the same rank, they all either spiral up to 9d or drop to 30k).

Personally I think that most cases are actually reversed. The hypothetical person wins four games in a row, thinks "Ahah, it must be that book I read last weekend. I'm stronger!", and then immediately thinks "Hey, why hasn't my rank gone up? There must be something wrong with KGS!". The next thing you know YAKRT (Yet Another KGS Rating Thread). There ought to be a law --> you have to wait four more games before posting!

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

what? most people don't improve linearly.

nonsense, the rating system should increase in accuracy if you add more data to it (obviously). The rating system should fit to my play schedule, I shouldn't have to play in such a way that gives me a good rating.

I only play rated games. I rank up as I improve and win more games. As I rank up, I play better opponents and am under more pressure to improve. Cycle. I hope this is how it will work out, in any case.

You're in luck, it does. Snorri's ideas won't help at all (see Mef's point).

The only difference is that if you play games slowly, you will not create deceptive streaks. That is, in a long string of games, there should be common times when you win 4 or 5 games in a row, and we humans are incapable of perceiving such sequences as random (sequences judged random by participants in studies are typically biased, while random sequences are judged to be non-random).

So if you play too many games too quickly, you will eventually fool yourself into thinking you've had a jump in rank and KGS is "stuck". In fact, you will not have had a jump, and KGS will be right more often than not. So play 2-7 rated games a week, and you won't feel stuck, your rank will be where it should, and you'll be happy.

Several years ago, I played up to 30 rated games a month, and my rank always felt nicely responsive, and I never felt like I had a jump that was not reflected in my rank.

_________________
Occupy Babel!

Perhaps I didn't state my whole point. I am not saying that a consistent 1 rated game per week pattern is different from 1 game per day or 20 games per day, for that matter. The fact that it is basically the same is one of the advantages of the KGS system. On a point based system, promotion does require a specific number of games.

We should try to be clear about the problem the OP is trying to solve, if it is to be considered a problem at all. What happens is that bunches of games played closely together at higher than your average play rate have a tendency to stabilize the rating if the win/loss rate is near 50%. So later, it takes a lot of games to move that number unless they've mostly aged out. Is this a problem? From one point of view, it isn't. If a player decides to invest a lot of games in a short period and have that anchor the rating for a few months, then so be it. That's when the player played the most games, and that's where most of the data is. Fine. So where's the problem? I think the thing that nags some players is that when they think they have improved, they may resent the idea that they might have to spend months (or months worth of games) to prove it. By playing rated games at a consistent rate, you accept the moving average for what it is. If you think you have improved, you have the option of waiting for adjustment or you can invest more. That is why I suggested 1 rated game a week. Then when you think you've improved, you can ramp it up to, say 1-2 rated games a day and have things move faster if you are right. Now if one is going to suggest it is bad to deny "the system" rated games, that's another discussion.

If you want a rating system to be more responsive to improvement, then maybe you'd have to reward streaks disproportionately, like WBaduk does. For example, if someone goes on a winning streak, you might lower the weightings of any games that precede that streak. I don't know if that would destabilize the system. Probably it would not be popular because you can't just reward winning streaks. You have to punish losing streaks as well, which might not be as popular.

Thanks, that is very useful information. If you find these references, I would be quite interested in it.

These two statements seem strangely at odds.
either way, you seem to be misrepresenting the arguments of the other side. I would be shocked if anyone was complaining about not getting promoted after 4 wins, but I once had an 18 game win streak, and only went up half a stone.

If you assume that your current rank is correct, (which would mean a 50% win ratio), then a streak of five won games in a row should indeed be common (the probability for this is 3%, so roughly this should happen once in 30 games). A streak of 10 won games in a row would have a chance of 0.1%, which could still happen once a year if you play a lot.... A streak of 15 won games would be around 0.003 %. In such a case I would assume it is much more likely that the person in question has actually improved. So I do see why people get frustrated if their likely improvement is not reflected by the rating system.

Also, if you play games slowly, you will also create deceptive streaks. The difference is that if you play slowly you are much more likely to be promoted on a deceptive streak than if you play a lot of games. But you are much more likely to be demoted as well...

Last one: There have been repeated reports that people have created new accounts that quickly got a higher rank than the original one. This would be a clear indication that there was a real improvement. On the other hand, I have no idea how often this happens in reality. It could also be that these people become quiet if the new account stabilizes at the same rank as the old one...

As someone who occasionally looks into/tracks this sort of thing...while there are some cases of this happening, there are also many cases of people who make a new account, play stronger opponents, get a solid rating 2 stones higher....then over the course of the next month see their rating slowly return to what it once was (the value they were complaining the old account was "stuck" at).

Edit later because this is in response to the poster that asked for it:

Wms talked a little bit about weighting systems here

The points you make are more or less sound, it really comes down to perception and what you want out of a rating system. For instance when you say "destabilize" the rating system, this might be a little ambiguous...It would not necessarily destabilize the system in the way mentioned up thread (where ranks will spiral one direction or another)...but it will make the ranks more fluid, and likely decrease the predictive capacity of the rating system. It's worth noting however that many critics don't consider predictive capacity to be a desirable characteristic of a rating system, they would prefer a descriptive system (that reflects what has already happened, and not necessarily what will happen).





This is very true as well...While it has been known to happen, it is much less common to hear people complain their rank is "stuck" when the 8 game losing streak does not cause demotion.