rank certificate issued by forum?

[rubin427]

I might qualify for a "web search challenged" forum certificate, since it seems half of my posts are asking questions that I could have easily answered with a web search.

(For those keeping score, the other half of my posts make bold assertions which are easily refuted by the results of a web search.)

[daniel_the_smith]

Not quite. All games in a single tournament are rated simultaneously, so if that 4k goes on to beat a bunch of strong player you're okay. The real problem shows up when that bogus 4k beat you in one tournament, and then beats up the dan players next week in a separate tournament.

I guess I shouldn't have glossed over that for brevity like I did...

The AGA ratings don't propagate data backwards in time, but it is on the to-do list.

That makes me happy

[Kirby]

...KGS's algorithm realizes that the 4k was probably not 4k and retroactively changes the impact that game has on my rating. KGS's algorithm is a massive improvement in my mind, especially if you don't play that many rated games. ...

This is actually an aspect of the ranking system that I don't care for that much. If everybody improved at the same rate, I would be more inclined to accept it, but I don't believe that's how it works in real life.

For example, suppose that you are 1k and I beat you in a game. To me, that means that the skills that I exhibited on that day were enough to win against you.

Now suppose that you didn't like losing to me, so you go home and study as much as you can. You do tsumego problems day and night, study pro games, get reviews at your club, and so on.

Suppose also that I don't feel like studying, so I sit on my couch and eat potato chips all day without studying. A couple of weeks later, let's say that your studying has payed off, and you advance to 2d. Meanwhile, I'm still sitting on my couch eating potato chips.

Is it likely that my skill has improved simply because you have started to win more games? I don't think so.

If everybody improved at the same rate, or if all people's ranks were static, I'd be all for this approach. But realistically, people do not improve at the same rates, in my opinion.

That is, just because player A is better than player B today does not mean that it will stay that way for very long. If we were trying to establish a maximum likelihood estimation of some static measure, then I think it'd be the way to go. But with something that grows as dynamically as rank, I don't think that it's an appropriate way of measuring.

[daniel_the_smith]

This is actually an aspect of the ranking system that I don't care for that much. If everybody improved at the same rate, I would be more inclined to accept it, but I don't believe that's how it works in real life.

That's the drift problem I refer to. I think WHR is not (as) susceptible to it but don't have time to verify that ATM.

[Kirby]

This is actually an aspect of the ranking system that I don't care for that much. If everybody improved at the same rate, I would be more inclined to accept it, but I don't believe that's how it works in real life.

That's the drift problem I refer to. I think WHR is not (as) susceptible to it but don't have time to verify that ATM.

What is whr?

[daniel_the_smith]

Whole History Rating

[topazg]

This is actually an aspect of the ranking system that I don't care for that much. If everybody improved at the same rate, I would be more inclined to accept it, but I don't believe that's how it works in real life.

That's the drift problem I refer to. I think WHR is not (as) susceptible to it but don't have time to verify that ATM.

I can see nothing in WHR that makes any attempt to handle it actually... I'd be interested to see if I'm mistaken, because it is one of the big flaws I see in any "standard" algorithm.

[daniel_the_smith]

I believe that allowing the players to have "time-varying strength" will naturally go a long ways towards fixing drift. I haven't tested it though. I could be wrong.

[topazg]

I believe that allowing the players to have "time-varying strength" will naturally go a long ways towards fixing drift. I haven't tested it though. I could be wrong.

My understanding was that wouldn't easily "inject" points into the system though. It's just a clever way of making more adaptive rating volatility isn't it?

[pwaldron]

I believe that allowing the players to have "time-varying strength" will naturally go a long ways towards fixing drift. I haven't tested it though. I could be wrong.

All rating systems have time varying strengths...that's what makes them rating systems. I assume that's not quite what you meant.

More seriously, all rating systems in general use are susceptible to drift. The probability of player A beating player B is expressed as a function of their rating difference, and that difference doesn't change if both players float up or down by a constant amount.

We understand this intuitively when we realize that different countries have different rating systems, and they all function just fine. There isn't a way to fix the strength of a shodan, so drift is always a problem.

[daniel_the_smith]

I think there's some term confusion going on here-- by drift I'm specifically referring to the KGS phenomenon where you log in 5 months later to find yourself ranked 4d because everyone you played against earlier got stronger. I'm not referring to the fact that the rating system as a whole can get weaker or stronger. Maybe I should call it "KGS drift", hehe.

pwaldron, as far as I know, the math in most rating systems actually assumes that a player's strength does not change. This is why, for example, KGS weights old games less and less as they age. You get the illusion of change if you ask the algorithm "what am I rated now?" periodically and chart the results, because the algorithm's estimation of your true strength will vary with the data points it has seen. WHR is unique in that it assumes (and tries to calculate) everyone's rating for their whole history.

(An easy way to understand this: KGS adds one pixel to your graph each day with what it thinks your true rating is; it never changes yesterday's pixel. WHR would generate an entirely new graph for you every day.)

[daniel_the_smith]

Since top pro strength remains approximately on the same level compared with what we assume to be perfect play, (snip)

This is probably not actually the case. Even if it were, you need more than one anchor point to stabilize a ranking system.

[daniel_the_smith]

The problem is that small errors accumulate and multiply the further you get from the anchor. I don't have a link off hand, sorry

[Harleqin]

As far as I understand, the current KGS algorithm determines a single current rank (plus uncertainty) from the whole history, weighing the games by currentness, for each player. A rank development occurs because old data are no longer being considered (or rather, considered less and less). In contrast, the WHR algorithm determines a complete time-dependent rank graph (plus uncertainty) for each player. A rank
development is thus completely modeled at each time.

It seems to me that this means that there is no longer a large drift, because the system understands that a player's rank is not constant.

Many thanks for pointing this out, daniel_the_smith. This is really a great improvement over all ranking and rating algorithms I have known so far. Is there already some work with respect to how handicap should be incorporated?

[daniel_the_smith]

Yes, obviously, but the Go-handicap system does not only allow the measurement of 1-stone difference but also 2-stone, 3-stone and so on. Overall the handicap system has always struck me as being remarkably accurate, and one also has to consider that the further away from pro strength the more will ability fluctuate from game to game as stated above. Is the accumulated errors really so big that it does it impossible to define shodan in a consistent way?

Edit: I am obviously not talking about Go-server algorithms here, but rather about if the thing can be considered in principle.

It might work, for a few levels anyway. I'm not sure. Assuming you could get pros to play that many high-handicap games seriously.

@Harleqin, yeah, I think everyone should be using WHR.