Life In 19x19

I used a type of ELO model to estimate the best players of all time, for my own amusement. I thought other players might be interested also. I will put the list first, then those who are curious about the method can read on:

1. Yi Ch'ang-ho
2. Cho Hun-hyeon
3. Cho Chikun
4. Chang Hao
5. Yi Se-tol
6. Yu Ch'ang-hyeok
7. Kobayashi Koichi
8. Yu Bin
9. Gu Li
10. Zhou Heyang
11. Kong Jie
12. Yoda Norimoto
13. Pak Yeong-hun
14. Go Seigen
15. Ma Xiaochun
16. Wang Lei Sr.
17. Kato Masao
18. Cho U
19. Xie He
20. Rin Kaiho
21. O Rissei
22. Sakata Eio
23. Kobayashi Satoru
24. Mok Chin-seok
25. Kitani Minoru


gritty details:


There certainly seems to be a strong bias towards more recent players. I think perhaps this is because younger players tend to player older ones at their weakest, which gives a statistical perception that the young players are stronger. But I intentionally didn't try to model player strength as changing over time, because it's a very tricky thing to get right and is sensitive to parameter choice.

I was a bit surprised to see Chang Hao rated so highly, and also that Yoda Norimoto was ranked higher than Cho U.

So what do you think of this list? Does it seem to represent the best players ever?

It's a good idea - I think it will always favour players at their peak though.

topazg wrote:It's a good idea - I think it will always favour players at their peak though.

Yes, if I find time I may try to come up with a way around this. It's worth noting that even in spite of the bias Go Seigen still made #14, I can't help but wonder where he would really stand, if we could pull him from 1950 and have him play today's top players.

Where is Dosaku? (13 p. ) Huang Longshi? Jowa? Shusaku?

Yes, if I find time I may try to come up with a way around this. It's worth noting that even in spite of the bias Go Seigen still made #14, I can't help but wonder where he would really stand, if we could pull him from 1950 and have him play today's top players.

I obviously don't know how you collated the raw data, but I have an inkling that people like Go Seigen would be affected because they played so many games with no komi.

There is also the possible problem that players who have published Collected Games get their early games included, but players like Yoda, Chang Hao and Cho U tend to have later games dominate.

I'm just wondering how on earth Yu Bin is above Gu Li and Ma Xiaochun (btw, where's Nie Weiping)?

interesting, thanks for sharing

however, i have to wonder how much are informations based on your method reliable (as you already said, no critics involved). i don't know enough about rating, statistics and stuff, but i think i would be a bit more inclined to believe results based on regular during time changing ELO (or GoR, as it should be pretty similar and i am more familiar with it) and taking maximum achieved value for each player, like in progor (data to 2008 included, then unfortunately discontinued).

by the way, i am not really sure what i am aiming at with this, but how much would positions (of older players) move if you made 'time-slices' by 5 (or 2 or n) years? i mean including only games up to some date, like to 2010, to 2005, 2000 and so on.

illluck wrote:I'm just wondering how on earth Yu Bin is above Gu Li and Ma Xiaochun (btw, where's Nie Weiping)?

i guess that maybe Nie Weiping scores lower because he is still active but not top anymore, so he collects loses against weaker players. but there might be a better explanation

illluck wrote:I'm just wondering how on earth Yu Bin is above Gu Li and Ma Xiaochun (btw, where's Nie Weiping)?

Any "best player" ranking will cause disagreement.

Kirby wrote:

illluck wrote:I'm just wondering how on earth Yu Bin is above Gu Li and Ma Xiaochun (btw, where's Nie Weiping)?

Any "best player" ranking will cause disagreement.

So all disagreement is pointless? That's a stretch...

As far as the rankings, you might try looking at games played before a certain age, to help with the peak issue, though this would still favor recent pros.

emeraldemon wrote:I used a type of ELO model to estimate the best players of all time, for my own amusement. I thought other players might be interested also. I will put the list first, then those who are curious about the method can read on:

1. Yi Ch'ang-ho
2. Cho Hun-hyeon
3. Cho Chikun
4. Chang Hao
5. Yi Se-tol
6. Yu Ch'ang-hyeok
7. Kobayashi Koichi
8. Yu Bin
9. Gu Li
10. Zhou Heyang
11. Kong Jie
12. Yoda Norimoto
13. Pak Yeong-hun
14. Go Seigen
15. Ma Xiaochun
16. Wang Lei Sr.
17. Kato Masao
18. Cho U
19. Xie He
20. Rin Kaiho
21. O Rissei
22. Sakata Eio
23. Kobayashi Satoru
24. Mok Chin-seok
25. Kitani Minoru


gritty details:

I started from the same assumption as elo: every player has a hidden ranking number, and the proabability of one player beating another increases as the rank difference increases. But instead of trying to model players' strength as increasing or decreasing over time the elo does, I just assumed a player has one strength their whole life. This turns all the results into a huge overdetermined system of linear equations, and I used least-squares fit to solve it. The results are what you see above.

I used my slightly old copy of GoGoD for these results, so newest info isn't there.

I threw out all players with fewer than 30 games in the record, to keep statistical anomalies from messing things up too much.

The equation used to calculate elo diff is
-log10(1/r - 1)
where r is the win %. The trouble with this is that if two players have a 0-1 record, the model thinks the second player has a 100% chance of winning and the rating difference between the two players is infinite. To soften this overconfidence, I gave every pro a single draw against every other pro, so the 0-1 becomes 0.5-1.5,

I don't try to catch players who changed names over time. Yasuda Shusaku is 89th on this list, but Honinbo Shusaku is 94h.

There certainly seems to be a strong bias towards more recent players. I think perhaps this is because younger players tend to player older ones at their weakest, which gives a statistical perception that the young players are stronger. But I intentionally didn't try to model player strength as changing over time, because it's a very tricky thing to get right and is sensitive to parameter choice.

I was a bit surprised to see Chang Hao rated so highly, and also that Yoda Norimoto was ranked higher than Cho U.

So what do you think of this list? Does it seem to represent the best players ever?

You can see the same bias towards young(er) or more modern players in chess.
This represent the idea that theory develops over time, and that modern players would beat old players because of that - unless the old players get time to adjust.

What I personally find interesting is the high percentage of Nihon/Kansai-Kiin players. 11 out of 25 or so? What does it say about the superiority of Korean/Chinese players over Japanese ones?

What I personally find interesting is the high percentage of Nihon/Kansai-Kiin players. 11 out of 25 or so? What does it say about the superiority of Korean/Chinese players over Japanese ones?

Actually, the very first thing I noticed about the list as shown was that it covers the players who are most represented in GoGoD, and this does have a (diminishing) bias towards Japanese players.

Obviously we have collected games of the most famous/successful players as these are more generally the most interesting/available, but I wonder if there is also the possibility, in the way the algorithm works, that a well-represented player who wins 10 games and scores 1 point each against moderate opponents can creep ahead of a "better" but under-represented player who wins 2 games and scores 4 points each against top opponents.

hyperpape wrote:

Kirby wrote:

illluck wrote:I'm just wondering how on earth Yu Bin is above Gu Li and Ma Xiaochun (btw, where's Nie Weiping)?

Any "best player" ranking will cause disagreement.

So all disagreement is pointless? That's a stretch...

As far as the rankings, you might try looking at games played before a certain age, to help with the peak issue, though this would still favor recent pros.

Disagreement isn't pointless. Although, I wonder if "best player" ranking lists are.

Rather than a "best player" ranking list, I would prefer a list of players sorted by characteristics such as “most wins in 2011”, or “longest winning streak from year X to year Y”, etc. Such statistics are objectively measurable, whereas characteristics such as “best player” or “coolest playing style” are not.

I think that you fold too much information into a single number.

When reading the title, I thought "Simplified? ELO is too simple already!"

Bill Spight wrote:Where is Dosaku? (13 p. ) Huang Longshi? Jowa? Shusaku?

Honinbo Dosaku: 149
Huang Longshi: didn't meet #games cutoff (may rerun later w/ more players in)
Honinbo Jowa: 169
Honinbo Shusaku: 94, Yasuda Shusaku 86 (no attempt to bring names together)

illluck wrote:I'm just wondering how on earth Yu Bin is above Gu Li and Ma Xiaochun (btw, where's Nie Weiping)?

Nie Weiping: 51

John Fairbairn wrote:I obviously don't know how you collated the raw data, but I have an inkling that people like Go Seigen would be affected because they played so many games with no komi.

I didn't make any attempt to adjust for komi, I just hoped people played approximately as many games as black and white in those days. I suppose the best thing to do would be to offer a bias based on true win%, so if black wins 60% on no komi, two equal players have that same value. I may go back and try this if I find time. I could even try something similar for handicap games, since they were more common in the old days I think.

Laman wrote:i think i would be a bit more inclined to believe results based on regular during time changing ELO (or GoR, as it should be pretty similar and i am more familiar with it) and taking maximum achieved value for each player, like in progor (data to 2008 included, then unfortunately discontinued).

Well, first I'd note those lists are actually pretty similar: all of his top 10 are in my top 25 except Nie Weiping. I chose specifically to avoid models with fiddly parameters that would need tuning, such as ELO or GoR. Not that they're bad necessarily, but I would need to spend time to actually tune those parameters (or trust the values chosen by others, which I'd rather not do!)

Laman wrote:by the way, i am not really sure what i am aiming at with this, but how much would positions (of older players) move if you made 'time-slices' by 5 (or 2 or n) years? i mean including only games up to some date, like to 2010, to 2005, 2000 and so on.

I actually think this is an interesting idea, and easy to try. The problem is that the numbers from different slices would no longer be directly comperable, but it might still be interesting for its own sake.

John Fairbairn wrote:Obviously we have collected games of the most famous/successful players as these are more generally the most interesting/available, but I wonder if there is also the possibility, in the way the algorithm works, that a well-represented player who wins 10 games and scores 1 point each against moderate opponents can creep ahead of a "better" but under-represented player who wins 2 games and scores 4 points each against top opponents.

This is certainly possible, but it's a quite difficult bias to remove. More wins means more evidence of a players' strength. When Han Taehee upset Yi Changho, it was certainly interesting, but it didn't mean he deserved to be at the top of the ratings. The model rewards players who win consistently, as I think any good model should. One option might be to build a second "confidence" parameter, that says "I think player X is really great, but my confidence in this assertion is low due to lack of data". Hmm, I'd have to think about how to implement such a thing.

Kirby wrote:Disagreement isn't pointless. Although, I wonder if "best player" ranking lists are.

Rather than a "best player" ranking list, I would prefer a list of players sorted by characteristics such as “most wins in 2011”, or “longest winning streak from year X to year Y”, etc. Such statistics are objectively measurable, whereas characteristics such as “best player” or “coolest playing style” are not.

Now that I've pulled the data, I can certainly make such lists if you'd like . Maybe that can be a seperate post.

From a statistics perspective, I'm actually trying to answer a very specific (if ultimately unanswerable) question: if any two players play against each other, which is more likely to win? And it is actually objectively measurable, in a way: I could take the data from 2000-2004, train the model, then see how well it predicts the results of the matches from 2004-2009, for example. This may be the next thing I do, actually.

As for coolest playing style, I can only say that in my opinion winning is very cool.

Harleqin wrote:I think that you fold too much information into a single number.

When reading the title, I thought "Simplified? ELO is too simple already!"

I agree. I simplified for two reasons: one, I only have so much time (I did this to take a break from the programming I'm supposed to be doing!), and two, as mentioned above, ELO (and most other models) have somewhat arbitrary numbers representing how much a win should improve your rating, etc. If I do move to a more complicated model, I would want to think for a bit about how to do that, and how to test if it's actually better (probably something like the test mentioned in response to Kirby's post).