It is currently Thu Oct 31, 2024 4:18 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 57 posts ]  Go to page 1, 2, 3  Next
Author Message
Offline
 Post subject: KGS ranking system is nonsense
Post #1 Posted: Mon Feb 24, 2014 6:10 am 
Beginner

Posts: 14
Liked others: 1
Was liked: 4
Rank: 3k
I made a new account and played a few games, went to rank x and then after losing another game dropped to rank y eventually. So far so good.

Now the KGS system kicked in:
Since I just dropped to y, I'm obviously pretty close to x, in case I can manage to win a few games I should go back to x.
However, first I lose 3 more games at rank y. Then I waited for about 2 weeks or so and then won 6 games in a row!
Result? Still super solid rank y.

So we can debunk myths about KGS system:
-It's definitely not going up as easily as it goes down
-Even if you only played a couple of games and that was about 2 weeks ago, and then you play twice as many games, the effect upwards will still be LESS than half of that amount of games downwards, even from 2 weeks ago.

This, gentlemen, is the reason why people have always been forced to create new accounts in the brilliant KGS system.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #2 Posted: Mon Feb 24, 2014 6:22 am 
Oza
User avatar

Posts: 2508
Liked others: 1304
Was liked: 1128
not enough data.

_________________
Patience, grasshopper.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #3 Posted: Mon Feb 24, 2014 9:16 am 
Lives in sente

Posts: 946
Liked others: 1
Was liked: 41
Rank: IGS 5kyu
KGS: KoDream
IGS: SmoothOper
I have just started on Tygem, so far I have been very pleased with the rating system, though I haven't found it documented anywhere, I guess it is based on a binomial window of wins, which makes sense, to me and is somewhat more transparent than a machine learning weighting system. I started as a 5kyu, then played on IGS for a while, and went back to Tygem and within maybe ten games it had figured out I was significantly stronger and bumped my rank down to 4kyu. I think I like it better than IGS. The "threshold of changing" thing gets old when you are trying to rank up, and even with the threshold, you have maybe three or four games to win at your new rank, before you slide back. It isn't a problem at the lower ranks, since you improve so quickly, but as your rank increases that slows down, so instead of yo-yo-ing every other game from 3kyu to 4kyu and playing tight games you find yourself yo-yo-ing between playing at rank that is too difficult and a rank that is too easy for five to ten games at a stretch, which is kind of annoying, and overall an artificial construct that detracts from playing the game.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #4 Posted: Mon Feb 24, 2014 9:30 am 
Tengen

Posts: 4382
Location: Caldas da Rainha, Portugal
Liked others: 499
Was liked: 733
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
Some other guy wrote:
I made a new account and played a few games, went to rank x and then after winning another game went up to rank y eventually. So far so good.

Now the KGS system kicked in:
Since I just rose to y, I'm obviously pretty close to x, in case I can sadly lose a few games I would go back to x.
However, first I win 3 more games at rank y. Then I waited for about 2 weeks or so and then lost 6 games in a row!
Result? Still super solid rank y.

So we can debunk myths about KGS system:
-It's definitely not going down as easily as it goes up
-Even if you only played a couple of games and that was about 2 weeks ago, and then you play twice as many games, the effect downwards will still be LESS than half of that amount of games upwards, even from 2 weeks ago.

This, gentlemen, is the reason why people have always been forced to create new accounts in the brilliant KGS system.
Sadly for you, the KGS rank algorithm is described on the website. And I put more faith in our collective ability to understand how that algorithm works than anything I might conclude from your story.

_________________
Occupy Babel!

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #5 Posted: Mon Feb 24, 2014 10:03 am 
Lives in sente

Posts: 852
Location: Central Coast
Liked others: 201
Was liked: 333
Rank: KGS [-]
GD Posts: 428
raptor wrote:

Now the KGS system kicked in:
Since I just dropped to y, I'm obviously pretty close to x, in case I can manage to win a few games I should go back to x.


These two assumptions are not true, this is likely where much of your confusion about how the rating system stems from.

Quote:
However, first I lose 3 more games at rank y. Then I waited for about 2 weeks or so and then won 6 games in a row!
Result? Still super solid rank y.


So it sounds like you are 6-4 at your present rank (assuming you haven't handwaved additional details)...That would appear to me that you are properly ranked.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #6 Posted: Mon Feb 24, 2014 10:15 am 
Oza

Posts: 2180
Location: ʍoquıɐɹ ǝɥʇ ɹǝʌo 'ǝɹǝɥʍǝɯos
Liked others: 237
Was liked: 662
Rank: AGA 5d
GD Posts: 4312
Online playing schedule: Every tenth February 29th from 20:00-20:01 (if time permits)
raptor wrote:
This, gentlemen, is the reason why people have always been forced to create new accounts in the brilliant KGS system.


How do you know that is doesn't have problems because people keep trying to cheat the system by creating new accounts? A plethora of random accounts with few games is likely to lead to greater uncertainty in those accounts and hence greater randomness in the ratings.

It is not possible to produce reliably conclusions from small sample sizes so unless you have several dozen games under your belt against solidly ranked players then any rating it may give for you is at best a guess. If you have multiple accounts with few games, as you appear to have, then all are likely to be useless as predictors until you have played more game.

_________________
Still officially AGA 5d but I play so irregularly these days that I am probably only 3d or 4d over the board (but hopefully still 5d in terms of knowledge, theory and the ability to contribute).

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #7 Posted: Mon Feb 24, 2014 12:22 pm 
Lives with ko

Posts: 197
Liked others: 0
Was liked: 81
Rank: weak
KGS: often
if you're strong enough, you'll win enough, and you'll be at your proper rank
what's the problem?

just play more.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #8 Posted: Mon Feb 24, 2014 1:28 pm 
Lives in sente
User avatar

Posts: 1103
Location: Netherlands
Liked others: 408
Was liked: 422
Rank: EGF 4d
GD Posts: 952
often wrote:
if you're strong enough, you'll win enough, and you'll be at your proper rank
what's the problem?

just play more.


In my experience, this is not really true. People respond to positive feedback. If a player starts winning games, but the challenge doesn't increase, it's easy for a player to lapse into bad habits.

Likewise, a player who experiences variation in style and challenge of games has more to learn from and improves faster.

KGS tries to reach a state where the rank is stable. I believe that's a mistake. A player should have a reasonable random walk around their rank. That random walk needs to be large enough that the challenge per game varies. That's not going to happen as long as the variance of a rating is contained inside a single pair of rating bars.

_________________
Tactics yes, Tact no...


This post by shapenaji was liked by: oren
Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #9 Posted: Mon Feb 24, 2014 2:07 pm 
Lives with ko

Posts: 197
Liked others: 0
Was liked: 81
Rank: weak
KGS: often
shapenaji wrote:
People respond to positive feedback. If a player starts winning games, but the challenge doesn't increase, it's easy for a player to lapse into bad habits.


Except his problem is he thinks he should be at rank X instead of rank Y. If he was really a X player then he should win a majority of his games when he is at rank Y, so he'll naturally get a higher rank.

Your opinion is like saying a real 1d player will go into such bad habits if he plays with only 8k players on KGS that he will never get a 1d rank if he started from a solid 8k rank.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #10 Posted: Mon Feb 24, 2014 2:12 pm 
Oza

Posts: 2356
Location: Ireland
Liked others: 662
Was liked: 442
Universal go server handle: Boidhre
often wrote:
shapenaji wrote:
People respond to positive feedback. If a player starts winning games, but the challenge doesn't increase, it's easy for a player to lapse into bad habits.


Except his problem is he thinks he should be at rank X instead of rank Y. If he was really a X player then he should win a majority of his games when he is at rank Y, so he'll naturally get a higher rank.

Your opinion is like saying a real 1d player will go into such bad habits if he plays with only 8k players on KGS that he will never get a 1d rank if he started from a solid 8k rank.


It's not just about beating players your own level though. Stronger players will "show" you moves you didn't consider and weaker players will play moves you didn't consider for different reasons. In both cases you end up with different situations and reading exercises than you get when playing people your own level.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #11 Posted: Mon Feb 24, 2014 2:21 pm 
Oza
User avatar

Posts: 2777
Location: Seattle, WA
Liked others: 251
Was liked: 549
KGS: oren
Tygem: oren740, orenl
IGS: oren
Wbaduk: oren
often wrote:
Except his problem is he thinks he should be at rank X instead of rank Y. If he was really a X player then he should win a majority of his games when he is at rank Y, so he'll naturally get a higher rank.


What I assume shapenaji is saying is that on KGS, you can win a lot of games and not move up as well as lose a lot of games and not move down. The KGS rating goal is stability but the end result can also be stagnation on some accounts. At least it's temporary, but it is a reason why people will create new accounts.


This post by oren was liked by: shapenaji
Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #12 Posted: Mon Feb 24, 2014 2:37 pm 
Lives in sente

Posts: 852
Location: Central Coast
Liked others: 201
Was liked: 333
Rank: KGS [-]
GD Posts: 428
oren wrote:
What I assume shapenaji is saying is that on KGS, you can win a lot of games and not move up as well as lose a lot of games and not move down. The KGS rating goal is stability but the end result can also be stagnation on some accounts. At least it's temporary, but it is a reason why people will create new accounts.




One clarification here -- it is incorrect to say that that the KGS rating system has stability as a goal. The rating system has accuracy as a goal. The purpose of the rating system is to do the best job it can of predicting the outcome of a game that has yet to be played between any two given players rated by the system.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #13 Posted: Mon Feb 24, 2014 10:25 pm 
Lives with ko

Posts: 197
Liked others: 0
Was liked: 81
Rank: weak
KGS: often
oren wrote:
What I assume shapenaji is saying is that on KGS, you can win a lot of games and not move up as well as lose a lot of games and not move down. The KGS rating goal is stability but the end result can also be stagnation on some accounts. At least it's temporary, but it is a reason why people will create new accounts.


I don't see how this is a bad thing. Who would want a rank that auto changes if you have a few bad games in a row? If i have a bad swing and lose 4 games in a row i don't want to lose 1-2 ranks, so why should the opposite happen?

Believe me, if you win 20 games in a row, no matter how "stagnant" your rank is, you'll be getting closer and closer to a higher rank anyway.

People just want to think they're stronger than they really are. Your rank is just a number that people get way too tied up over. Its not like not like anything will really change if you're "stronger". There is no secret KGS menu for a KGS 7d that we know of yet.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #14 Posted: Mon Feb 24, 2014 10:27 pm 
Oza
User avatar

Posts: 2777
Location: Seattle, WA
Liked others: 251
Was liked: 549
KGS: oren
Tygem: oren740, orenl
IGS: oren
Wbaduk: oren
often wrote:
I don't see how this is a bad thing. Who would want a rank that auto changes if you have a few bad games in a row? If i have a bad swing and lose 4 games in a row i don't want to lose 1-2 ranks, so why should the opposite happen?


4 may be too much, but 5 losses or 5 wins on wbaduk can get you a rank difference. I do prefer a more volatile system which I've mentioned before. A bad swing can lose a rank and a good one can raise it. On KGS, with enough games, nothing happens.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #15 Posted: Mon Feb 24, 2014 10:33 pm 
Lives with ko

Posts: 197
Liked others: 0
Was liked: 81
Rank: weak
KGS: often
oren wrote:
4 may be too much, but 5 losses or 5 wins on wbaduk can get you a rank difference. I do prefer a more volatile system which I've mentioned before. A bad swing can lose a rank and a good one can raise it. On KGS, with enough games, nothing happens.


If it's a difference in preference, then thats why there are other go servers.

I prefer KGS.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #16 Posted: Mon Feb 24, 2014 10:51 pm 
Beginner

Posts: 4
Liked others: 0
Was liked: 1
Mef wrote:
One clarification here -- it is incorrect to say that that the KGS rating system has stability as a goal. The rating system has accuracy as a goal. The purpose of the rating system is to do the best job it can of predicting the outcome of a game that has yet to be played between any two given players rated by the system.


The confusion here is because the KGS ranking model assumes that an accurate rank is a stable one. KGS model tries to converge to a stable rank, because the model interprets rank to work this way. This is debatable.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #17 Posted: Tue Feb 25, 2014 2:29 am 
Oza
User avatar

Posts: 2508
Liked others: 1304
Was liked: 1128
shapenaji wrote:
A player should have a reasonable random walk around their rank.


I'd like to see something like this too*. One model that I like is brain workshop's n-back game. Here, after crossing a threshold to a new level a single loss (or in the case of n-back, failure to achieve a certain percentage of correct answers) will not get you demoted. Instead, you stay at the higher level until you have accumulated three losses. This sort of system allows you to get a taste of higher level play without fear of immediate demotion, and although your rank may not be quite as accurate for some of the games, you still get a chance to collect some valuable experience.

*Of course, you could just join the ASR where you play even games against both stronger and weaker players. :)

_________________
Patience, grasshopper.


This post by daal was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #18 Posted: Tue Feb 25, 2014 3:15 am 
Lives in sente
User avatar

Posts: 1103
Location: Netherlands
Liked others: 408
Was liked: 422
Rank: EGF 4d
GD Posts: 952
often wrote:

I don't see how this is a bad thing. Who would want a rank that auto changes if you have a few bad games in a row? If i have a bad swing and lose 4 games in a row i don't want to lose 1-2 ranks, so why should the opposite happen?



Well, the typical method in club environments is to adjust handicap after 3 victories or 3 losses (Heck, I play 1-game kadoban with some of my closest rivals, we change the handicap after every game).

This creates reasonable challenges to reward improvement, but it also gives you consequences for losing.

Quote:
Believe me, if you win 20 games in a row, no matter how "stagnant" your rank is, you'll be getting closer and closer to a higher rank anyway.


The rank system does fine in making the large adjustments associated with someone who is so far off their rank that they can win 20 games in a row... it's much worse when the rank system is 1 stone off, and we're talking about a 60% win rate rather than a 50%.


Quote:
People just want to think they're stronger than they really are. Your rank is just a number that people get way too tied up over. Its not like not like anything will really change if you're "stronger". There is no secret KGS menu for a KGS 7d that we know of yet.


People thinking that they're stronger than they really are is how people get stronger. If you never believed that you were capable of more, you'd never try.

Also, Things absolutely change when you get stronger, even a difference in a rank or 2 can be a major difference in the primary focus of the games. Getting stronger is addictive for just that reason, you never know what's past the next rank horizon.

_________________
Tactics yes, Tact no...


This post by shapenaji was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #19 Posted: Tue Feb 25, 2014 3:17 am 
Lives in sente
User avatar

Posts: 1103
Location: Netherlands
Liked others: 408
Was liked: 422
Rank: EGF 4d
GD Posts: 952
daal wrote:
shapenaji wrote:
A player should have a reasonable random walk around their rank.


I'd like to see something like this too*. One model that I like is brain workshop's n-back game. Here, after crossing a threshold to a new level a single loss (or in the case of n-back, failure to achieve a certain percentage of correct answers) will not get you demoted. Instead, you stay at the higher level until you have accumulated three losses. This sort of system allows you to get a taste of higher level play without fear of immediate demotion, and although your rank may not be quite as accurate for some of the games, you still get a chance to collect some valuable experience.

*Of course, you could just join the ASR where you play even games against both stronger and weaker players. :)


Sounds similar to the Tygem system, of which I'm definitely a fan.

_________________
Tactics yes, Tact no...

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #20 Posted: Tue Feb 25, 2014 7:52 am 
Lives in sente

Posts: 852
Location: Central Coast
Liked others: 201
Was liked: 333
Rank: KGS [-]
GD Posts: 428
shapenaji wrote:
...when the rank system is 1 stone off, and we're talking about a 60% win rate rather than a 50%.


For KGS, one stone off would be closer to 80%. I think this is another one of those things that often throws people off on their expectations of the rating system.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 57 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group