Have you got a link to that study? I'd be extremely curious about the caveats attached to that statement (I imagine there are playing thresholds below which the KGS algorithm stops producing sensible predictions and these can cluster if too many people in someone's history as well as themselves start playing irregularly and their ranks become unstable?).

Well this is certainly true, but that's largely because KGS aims for accuracy over volatility. Admittedly, for many people accuracy in a rank isn't necessarily a priority. It's also true that for some people having ranks introduce an element of randomness may make things more enjoyable. This is one of the reasons why games like poker and backgammon can be so appealing -- even though there is strategy and potentially large skill disparities, the randomness keeps the outcome unknown.

In fact for this same reason I wouldn't be surprised if the poor predictive power of Tygem's system were intentional. Tygem does have a betting feature, and there is a well known history of gambling being associated with baduk in Korea. From a gambler's perspective the worst thing in the world you can have is a public, perfectly accurate rating system, as this allows any bloke off the street to know the outcome in advance.

To summarize -- Do some people find Tygem's ranking system fun? Sure. Does Tygem's ranking system work for people who just want to play? Sure. Does Tygem's system provide significant predictive power in guessing game outcomes? No.

In that vein, I consider it to be a poor metric for measuring improvement for people who use ranks to do that sort of thing. If you just want to play games and have fun? Go to town, I'll be the last one to stop you!

I remember I got stuck at about ~6k for a while. I think I just got caught in a 6k mindset. I was playing maybe 6-7 games a day. I started playing over the board games and maybe played on KGS 1 or 2 times per week. The over the board games were against stronger people and it wasn't so much a gradual improvement, but more of a realization of what it takes at a higher level by experience it myself instead of being lazy and playing the same responses all the time. I think over two months I went up to 5k, finally played some F games even with a 2k and won them all so recreated and got re-established at 2k.

Its the culture of KGS combined with the ranking system. Nobody ever plays outside of their rank really, so weak people never get nice games with strong people. Its very selfish honestly.

Whole-History Rating: A Bayesian Rating System for Players of Time-Varying Strength

Description of comparison:



Table of results:

Wrong. It *does* take a longer time to get rank back up than with Tygem.

If you like thought experiments, try this one:

Let's say that two casual, non-studying players, Bob and Sally, are the same strength in real life.
Bob has converged to 3d on Tygem after playing 1000 games a day, every day.
Sally has converged to 1d on KGS after playing 1000 games a day, every day.

Now let's say that Bob and Sally both go to Go Congress for a week, and get fired up about studying go. They do lots of go problems, review pro games, and work hard to increase their strength. Let's say, as a result, that they both are one stone stronger in strength after this study. After coming back from Go Congress, let's say that Bob is 4d in Tygem strength and Sally is 2d in KGS strength.

Who needs to play more games to make their rank match their strength?




Logical fallacy: Strawman argument.
Your example does not address the point I was making. Let's go back to what I said:


My statement is about volatility of Tygem's ranks. Your statement is about the ability that Tygem gives for users to set their own initial rank. Personally, I would also prefer if Tygem did not provide this functionality, and determined rank on its own.

But this has nothing to do with how volitile Tygem's ranking system is.



I don't believe this is true. As I understand, you can only set initial rank up to 3d. Do you want to talk about rank setting? KGS provides the ability for users to set their initial rank as high as 1k when using automatch. Granted, this is not as direct as Tygem's methodology, but the rank setting Tygem allows is not what I'm most a fan of. I'm a fan of volitility, which allows you to move up in rank, even if you've played a lot of games.

Compared to Tygem, based on the "thought experiment" above, people are punished for playing more games on KGS more than they are on Tygem.

_________________
be immersed

Your statement is accurate for some cases, but not for others. For example, in the thought experiment I provided in the last post, Tygem is a more accurate indicator of the game outcome than KGS, because it will identify the growth in strength faster than KGS does.

_________________
be immersed

Agreed on this point. Although, before using Tygem much, it also succeeded in having me not play many rated games.

_________________
be immersed

My statement was not in reference to the fact that you can set your own rank. My statement was in reference to the fact that Tygem ranks do not necessarily converge even assuming that all players maintain consistent skill levels and play all games honestly. Being able to set your own rank should not be an issue for a rating system, as it should move you to the correct rank over time. Tygem does necessarily not do this. If you maintain a win rate of 33% on Tygem, you do not demote. It is quite easy to have a minority group that indefinitely self sustains a rank in spite of having both a significant number of games played with the greater group and no misconduct during any games. This effect would never be observed on KGS/IGS/DGS/OGS/etc.

On KGS you cannot set your rank to anything, your rank is always determined by the maximum likelihood of your future game results based upon your past performance.

If you are able to win against a 5d for 33% of your games after 10,000 games, your strength must be close to 5d. It would be highly unlikely for a kyu player to be able to achieve this.

In any case, this is unrelated to the point I am getting at - Tygem predicts changes in strength more easily than KGS does when users have played many games. A direct example of this can be seen in the "thought experiment" I provided.

_________________
be immersed

As an aside -- If we actually performed this thought experiment, Bob's rank would never be stable. You would expect his rank to jump +-2 stones everyday simply from random noise.

Why would you expect this? Bob could very well have a stable rank after having gone to go congress.

_________________
be immersed

Since it would seem you did not read it, I shall rewrite the thought experiment:



And regardless of thought experiments, as the actual study using 2.3 million KGS games as data showed: KGS predicts game results with higher accuracy than less sophisticated systems.

Playing 1000 games/day that are properly rated is equivalent to flipping 1000 coins in a row. Random variations will allow for streaks like winning 12/15 quite readily when you are playing that many games. You will even expect an 8 game winning streak once every 500 games.

Yep, I read it again. And it still relys upon Tygem's flaw in allowing the initial setting of ranks. If initial ranks are accurate to begin with, 33% win rate against a 5d is close to 5d.



Yes, someone wrote a paper on this, and he liked KGS's system. This is within the bounds of his definition of accuracy.

And I'll say it again, in case you are not reading what I've written: In the example I provided, KGS does worse job of determining difference in Bob's strength.

_________________
be immersed

No, the problem is that the rating system does not converge. If your rating system requires you to have perfect knowledge of your own rating prior to joining then why do you have a rating system in the first place?

Nope, not in this case. Here, Bob has > 50% chance of winning against even opponent since he has increased in strength.

_________________
be immersed

I am talking about the case prior to the go congress. If Bob is playing 1000 games/ day, he will never hold a rank for more than a few hours simply from random variation. Saying that Bob "converged" to 3d prior to the congress is meaningless because he would be oscillating around it.

Given the numbers of users on Tygem, for people that use automatch, ranks should come close to converging. The case you mentioned is for people that deliberately provided a bad initial rank.



Agreed, I don't care for this aspect of Tygem's system.

_________________
be immersed

Sure, I agree with this. Though, it's still easier for him to get a game matching to his strength than it is for Sally.

_________________
be immersed

Sally's games would be handicapped off by a stone for 10 days. Whereas Bob will have a significant number of his games with the wrong handicap on any given day.