It is currently Thu Mar 28, 2024 11:50 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 57 posts ]  Go to page Previous  1, 2, 3
Author Message
Offline
 Post subject: Re: KGS ranking system is bs
Post #41 Posted: Wed Feb 26, 2014 11:43 am 
Gosei
User avatar

Posts: 2011
Location: Groningen, NL
Liked others: 202
Was liked: 1087
Rank: Dutch 4D
GD Posts: 645
Universal go server handle: herminator
Boidhre wrote:
Isn't the issue in the EGF system at least that if you don't know the list of past opponents that someone's win rate is fairly meaningless? There's an expected win rate for a certain GoR difference but that's calculated on a game by game basis and has no relationship to someone's EGF rank really.


I'm not sure what you mean by this?

Win rates depend on how strong or weak your opponents are. If you have a win rate of 20% but played only players more than 3 stones stronger, then that is a good result. If you have a win rate of 80%, but played only players more than 3 stones weaker, then that is a bad result.

Of course, with most tournaments being McMahon, most people probably play a lot of opponents around their own level.

At the EGD you can find win rates for players, but only on their specific player pages, not in rating lists, because it wouldn't be meaningful there.

EGF ratings are calculated on a per event basis. If I visit an event and meet 5 opponents and win 3 games, it does not actually matter which of the games I win. As long as I play the same 5 opponents, any combination of 3 wins and 2 losses results in the same rating change.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #42 Posted: Wed Feb 26, 2014 11:53 am 
Lives in gote

Posts: 553
Liked others: 61
Was liked: 250
Rank: AGA 5 dan
HermanHiddema wrote:
shapenaji wrote:
I've seen people stuck with 60% win rate on KGS analytics never moving at all. And if Mef is correct, that's by design.
Isn't the same true for many rating systems? ....
IGS is the outlier in this respect. In the IGS rating system, you get points for every win, and you get promoted when your points reach a threshold level, so any win rate over 50% will eventually get you promoted. Of course right after such a promotion, you may have a much lower win rate and get demoted fairly quickly. This rating system is not as good as ELO type systems for evaluating player strength or predicting win/loss probabilities, but it has other properties some people like.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #43 Posted: Wed Feb 26, 2014 12:47 pm 
Oza

Posts: 2356
Location: Ireland
Liked others: 662
Was liked: 442
Universal go server handle: Boidhre
HermanHiddema wrote:
Boidhre wrote:
Isn't the issue in the EGF system at least that if you don't know the list of past opponents that someone's win rate is fairly meaningless? There's an expected win rate for a certain GoR difference but that's calculated on a game by game basis and has no relationship to someone's EGF rank really.


I'm not sure what you mean by this?

Win rates depend on how strong or weak your opponents are. If you have a win rate of 20% but played only players more than 3 stones stronger, then that is a good result. If you have a win rate of 80%, but played only players more than 3 stones weaker, then that is a bad result.

Of course, with most tournaments being McMahon, most people probably play a lot of opponents around their own level.

At the EGD you can find win rates for players, but only on their specific player pages, not in rating lists, because it wouldn't be meaningful there.

EGF ratings are calculated on a per event basis. If I visit an event and meet 5 opponents and win 3 games, it does not actually matter which of the games I win. As long as I play the same 5 opponents, any combination of 3 wins and 2 losses results in the same rating change.


I mean raw win rates aren't very interesting unless you know who people are playing and thus the expected win rate for them. Tournaments, eh, in the larger go countries sure, but the smaller ones won't be giving you opponents of similar level in most of your games, there'll normally be some handicap (-1 or whatever) involved.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #44 Posted: Wed Feb 26, 2014 12:54 pm 
Gosei
User avatar

Posts: 2011
Location: Groningen, NL
Liked others: 202
Was liked: 1087
Rank: Dutch 4D
GD Posts: 645
Universal go server handle: herminator
Boidhre wrote:
I mean raw win rates aren't very interesting unless you know who people are playing and thus the expected win rate for them.


Oh yeah, sure, but the same is true for any system. It is quite possible to maintain a 90% win rate on KGS and never be promoted, you just have to play lots of weaker players without handicap.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #45 Posted: Wed Feb 26, 2014 2:08 pm 
Lives in gote

Posts: 370
Liked others: 91
Was liked: 254
Rank: Weak
ez4u wrote:
lemmata wrote:
Longstride wrote:
The conflict between these two statements is my personal source of struggle with the KGS system. I looked myself up on kgs.gosquares.net and I have over a 65% win rate over the last two months (20+ games played per month), but my rank hasn't increased. This is troubling for me - I consistently feel that I'm playing at a level above my rank indicates, and the statistics reflect this, but the KGS ranking system does not... In fact, it took multiple months of win rates greater than 70% for me to achieve my last rank promotion (which was then mercilessly taken away from me during the recent anchor shift in late December).

KGS rankings are continuous, are they not? With a 65% winning percentage, you should been 7.7 dan or something like that according to the rank graph, which is better than 7 dan (if you were 7 dan at the beginning of those two months). You may have felt underrated because KGS rounds down all displayed ranks, but you were probably properly rated according to your raw rank.

What does it mean to be playing at a level above your KGS rank when 8 dan level means winning more than 70% of your games at 7 dan?

That said, I do prefer something like the Tygem system, as it is simple and allows a player to experience discrete variations in rank more frequently (adding variety/spice to life). I would agree with the general sentiment in this thread that the KGS ranking system is not as fun as it could/should be, but it is by no means bs. It's just different.
Actually KGS ranks are not continuous, only the ratings are.
You stated what I actually meant in more precise terms. :tmbup:
ez4u wrote:
In the end isn't the KGS rating system working just as advertised?
Agreed.
ez4u wrote:
However, the KGS ranking system and the KGS pairing system (based on the ranking system rather than the rating system) are the culprits, aren't they? According to their rating, as someone approaches promotion (e.g. their rating is 1.99, just short of jumping to 2d) they are systematically over-handicapped (receiving too much) against the average stronger player and under-handicapped (giving too little) against the average weaker player. That is why they need to win at very high rates in order to jump to the next level (and when they fail will join in the chorus here on how unfair KGS is). Once they make it over the hump to a 2.01 rating and are ranked as 2d they will switch to being systematically under-handicapped (receiving too little) against stronger players and over-handicapped (giving too much) against weaker players. As a result, they can survive at 2d despite losing half their games but they are unlikely to feel that they deserve it. Both situations are depressing.
Indeed, this is a good description of effects that could make the activity less fun for the players. It does not make the system bogus though.

Here's one more argument from me in favor of more rank volatility: Discourage this weird tendency toward rank protection. If players feel like they could erase the effects of some games played under less than ideal conditions fairly quickly, then they would be willing to play more often under more circumstances. The most important feature of a go server (to me personally) is how quickly I can find a decent game under common time settings. The reason I play on Tygem despite its horrific abomination that is the Tygem client is because I know that I can find a game fairly quickly there. Tygem does have a larger user base, but the number of users connected to a specific Tygem server (China 2/Korea 1) is only around 2-3 times larger than the number connected to KGS. Perhaps more rank volatility encourages people to push the automatch button more often. As little as I care about rank, I do care a lot about the number of people who are willing to push the automatch button on a server. Alas, I doubt that KGS will ever change...

Aside: OGS ladders are perfect in this regard. You can find a standard time-setting game on that server at any time of your convenience.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #46 Posted: Wed Feb 26, 2014 2:15 pm 
Oza

Posts: 2356
Location: Ireland
Liked others: 662
Was liked: 442
Universal go server handle: Boidhre
HermanHiddema wrote:
Boidhre wrote:
I mean raw win rates aren't very interesting unless you know who people are playing and thus the expected win rate for them.


Oh yeah, sure, but the same is true for any system. It is quite possible to maintain a 90% win rate on KGS and never be promoted, you just have to play lots of weaker players without handicap.



Yeah, sorry, I didn't mean to direct it at you but at the whole win rate discussion for pretty much the example you give there.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #47 Posted: Fri Feb 28, 2014 6:31 pm 
Beginner

Posts: 10
Liked others: 0
Was liked: 6
Rank: AGA 2 dan
I'm still waiting to see the first complaint from anyone that his rating hasn't gone down, even though he consistently loses 60% of his games.


This post by Charles Alden was liked by: ez4u
Top
 Profile  
 
Offline
 Post subject:
Post #48 Posted: Sat Mar 01, 2014 12:45 am 
Honinbo
User avatar

Posts: 8859
Location: Santa Barbara, CA
Liked others: 349
Was liked: 2076
GD Posts: 312
Charles Alden wrote:
I'm still waiting to see the first complaint from anyone that his rating hasn't gone down, even though he consistently loses 60% of his games.
Perhaps it happens more often than you think. (I've witnessed it first hand, in EGR, on multiple occasions.)

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #49 Posted: Sun Mar 02, 2014 8:02 am 
Dies in gote

Posts: 29
Liked others: 3
Was liked: 4
Rank: KGS 2 kyu
GD Posts: 543
uPWarrior wrote:
2 stone difference* means 50% win probability when the opponent has 2 stones on the board, 3 stones mean 50% when they have 3 stones on the board and so on.


Except that this is actually incorrect. A 2 stone handicap board is actually just 1.5 stones of advantage due to the lack of komi.

As you get closer to ranking up, you should be expected to perform 0.5 ranks better against an average even player and 1.0 ranks better in handicap games. Assuming the numbers on the http://senseis.xmp.net/?KGSRatingMath are accurate, that means an expected 60-66% (5k-2d) win rate in even games or a 70-79% win rate in handicap games against weaker players.

What would probably make the KGS system feel a lot better is if it did handicap based on exact rating differences instead of rankings. If a strong 2 kyu player plays an average 2 kyu player, it should be a no komi game where the weaker player starts. If a strong 2 kyu plays a weak 3 kyu, it should be a 2 handicap stone game. And so on.

You could also make it even more detailed by mixing handicap stones and komi, but that is probably a bit to unorthodox for a lot of people.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #50 Posted: Sun Mar 02, 2014 8:46 am 
Gosei
User avatar

Posts: 2011
Location: Groningen, NL
Liked others: 202
Was liked: 1087
Rank: Dutch 4D
GD Posts: 645
Universal go server handle: herminator
That's what IGS does: http://pandanet-igs.com/communities/pandanet/60

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is bs
Post #51 Posted: Sun Mar 02, 2014 1:18 pm 
Lives with ko

Posts: 199
Liked others: 6
Was liked: 55
Rank: KGS 3 kyu
Wildclaw wrote:
uPWarrior wrote:
2 stone difference* means 50% win probability when the opponent has 2 stones on the board, 3 stones mean 50% when they have 3 stones on the board and so on.


Except that this is actually incorrect. A 2 stone handicap board is actually just 1.5 stones of advantage due to the lack of komi.

As you get closer to ranking up, you should be expected to perform 0.5 ranks better against an average even player and 1.0 ranks better in handicap games. Assuming the numbers on the http://senseis.xmp.net/?KGSRatingMath are accurate, that means an expected 60-66% (5k-2d) win rate in even games or a 70-79% win rate in handicap games against weaker players.

What would probably make the KGS system feel a lot better is if it did handicap based on exact rating differences instead of rankings. If a strong 2 kyu player plays an average 2 kyu player, it should be a no komi game where the weaker player starts. If a strong 2 kyu plays a weak 3 kyu, it should be a 2 handicap stone game. And so on.

You could also make it even more detailed by mixing handicap stones and komi, but that is probably a bit to unorthodox for a lot of people.


That "*" after "2 stone difference" was there for a reason, I did mention the lack of symmetry by simply removing komi.

Anyway, you missed my point. The 60-66% or 70-79% win rates that go servers are enforcing on players 1 stone apart is pulled out of thin air. There is no mathematical or logical reason for that number to be 65% instead of 55% or 95%. As I said, ranks should be based on stone differences and not on win probabilities in even games. Go servers are optimizing for a metric that should be "learnt", not a constant of the algorithm.

Maybe this idea deserved a bigger and more detailed post as I have not seen this discussed anywhere else, but the way we are applying handicaps right now is totally disconnected to the way we are moving through ranks in go servers with a very large population. Most players are playing even games with each other (look at tygem, the percentage of even games is probably above 99%) and they are moving up and down on ranks based on win percentages in even games but there is no guarantee that players 4 stones apart win 50%* of their games with 4 stones handicap. THAT is what a 4 ranks difference is supposed to mean.


Just think about this hypothetical situation in your go club: a new player comes in and loses 4 games against a 5k and wins 1. Can you predict its rank in any way? Not until they adjust their handicap and get closer to 50%-50%. However, that is what go servers right now are doing, with the added constraint that even the 5k rank was calculated this way.

This is made worse by the fact that rank differences being disconnected from handicaps is self-reinforcing. Players stop playing handicap games because they feel their chances are not even and the risk on their ranking is not well adjusted.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #52 Posted: Sun Mar 02, 2014 8:47 pm 
Tengen

Posts: 4380
Location: North Carolina
Liked others: 499
Was liked: 733
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
If you believe in the efficient markets hypothesis, it is impossible for KGS rankings to be wrong about handicaps (relative to its assumptions about ranks in general) since if they were, players would be able to improve their ranks by simply choosing to play against players who the system would favorably handicap them. By favorably handicap them, I mean give them too many stones (or give their opponent too few stones). They win a bunch of those games and then go up in rank.

Sadly, this may be too much of an assumption.

In all seriousness, (1) the kgs system makes predictions about handicap games, (2) they are probably about as accurate as the regular rankings for small handicaps, since many games already effectively have a fraction of a stone handicap, and 1-2 stone handicap games are common. (3) Since the system makes predictions about handicaps, it can be tested for accuracy (though I don't know if it has been publicly analyzed).

_________________
Occupy Babel!

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #53 Posted: Fri Mar 07, 2014 4:15 pm 
Lives in gote
User avatar

Posts: 660
Liked others: 25
Was liked: 124
Rank: Miserable 4k
KGS: STOP STALKING ME
[12:22] X: I've now won 14 of my last 15. And KGS still doesn't want to give me my freakin dan rank
[12:39] S: time to get another account :V
[12:44] S: gotarist that's some great name there
[13:44] X: Bugger. I lost one. Now KGS will drop me to 3k for sure.

It's like the amount of wins to go from 1k to 1d is like the amount of energy to change state from ice to liquid water.

The guy used to be 4d at his peak, by the way (So I guess he's 1-3d now). A quick glance at kgs analytics gives him an 81% win percentage for ranked.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #54 Posted: Fri Mar 07, 2014 4:54 pm 
Lives in sente

Posts: 852
Location: Central Coast
Liked others: 201
Was liked: 333
Rank: KGS [-]
GD Posts: 428
Abyssinica wrote:
[12:22]A quick glance at kgs analytics gives him an 81% win percentage for ranked.


In order to properly evaluate this a little more context is necessary, namely how the games were handicapped and how his rank has changed over that period. For example, playing 100 games in six months, going 5-5 (solidifying middle 2k status), winning 35 of your next 40 (promote to 1k), win 30 of your next 50 against 1k's in even games. You would expect to settle as a strong 1k and have an 80% win rate.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #55 Posted: Fri Mar 07, 2014 6:20 pm 
Lives in gote
User avatar

Posts: 660
Liked others: 25
Was liked: 124
Rank: Miserable 4k
KGS: STOP STALKING ME
Mef wrote:
Abyssinica wrote:
[12:22]A quick glance at kgs analytics gives him an 81% win percentage for ranked.


In order to properly evaluate this a little more context is necessary, namely how the games were handicapped and how his rank has changed over that period. For example, playing 100 games in six months, going 5-5 (solidifying middle 2k status), winning 35 of your next 40 (promote to 1k), win 30 of your next 50 against 1k's in even games. You would expect to settle as a strong 1k and have an 80% win rate.


Apparently I'm not reading it right; I'm used to the bottom chart on kgs.gosquares to be all the games they've played, but for some reason that's just March. He has no games before Jaunary, and this is his actual score:

2014/1 - 2014/3
Win Lose PCT
All 37 18 67%
Black 11 9 55%
White 26 9 74%
Higher 2 6 25%
Even 20 7 74%
Lower 14 5 74%
With H. 1 2 33%
Against H. 4 2 67%

81% is just for this month

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #56 Posted: Fri Mar 07, 2014 6:33 pm 
Oza

Posts: 2356
Location: Ireland
Liked others: 662
Was liked: 442
Universal go server handle: Boidhre
Abyssinica wrote:
2014/1 - 2014/3
Win Lose PCT
All 37 18 67%
Black 11 9 55%
White 26 9 74%
Higher 2 6 25%
Even 20 7 74%
Lower 14 5 74%
With H. 1 2 33%
Against H. 4 2 67%

81% is just for this month


From that: they play almost all their games against even or weaker opponents and have a good win rate. They play few games against stronger opponents and lose a lot. They almost never take a handicap or give one.

Top
 Profile  
 
Offline
 Post subject: Re: KGS ranking system is nonsense
Post #57 Posted: Fri Mar 07, 2014 7:02 pm 
Lives in sente

Posts: 852
Location: Central Coast
Liked others: 201
Was liked: 333
Rank: KGS [-]
GD Posts: 428
Abyssinica wrote:
Mef wrote:
Abyssinica wrote:
[12:22]A quick glance at kgs analytics gives him an 81% win percentage for ranked.


In order to properly evaluate this a little more context is necessary, namely how the games were handicapped and how his rank has changed over that period. For example, playing 100 games in six months, going 5-5 (solidifying middle 2k status), winning 35 of your next 40 (promote to 1k), win 30 of your next 50 against 1k's in even games. You would expect to settle as a strong 1k and have an 80% win rate.


Apparently I'm not reading it right; I'm used to the bottom chart on kgs.gosquares to be all the games they've played, but for some reason that's just March. He has no games before Jaunary, and this is his actual score:

2014/1 - 2014/3
Win Lose PCT
All 37 18 67%
Black 11 9 55%
White 26 9 74%
Higher 2 6 25%
Even 20 7 74%
Lower 14 5 74%
With H. 1 2 33%
Against H. 4 2 67%

81% is just for this month


Thanks for the extra info, this is very helpful! 67% would normally be borderline promotion for a given rank. In this case it is not so clear, because they have played more games with lower ranks than higher. KGS has a pairing system that is slightly biased. Namely, the default handicaps for non-even games favor white slightly (this is known, and accounted for mathematically...but people playing don't always realize it). If you play more games against weaker opponents, you will be expected to have a higher win rate. What that means is that for this person, as they have been playing, 67% may be very close to but not enough for promotion.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 57 posts ]  Go to page Previous  1, 2, 3

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group