Life In 19x19

TheCannyOnion wrote:Iyama has risen to #29 on Goratings, probably due to his recent wins against Yamashita Keigo in Kisei title defense.

However, this means Iyama is ranked higher than Park Yeong-hun. Riiiiight... I would find it believable if not for the fact that Park has reached the last two Chunlan Cup finals in a row, beating Lian Xiao (twice), Gu Zihao, Dan Yifei, Zhou Ruiyang, and Ke Jie in the process. These were all strong opponents and, with the exception of Lian Xiao, all world-title holders.

What is up with Goratings? It can't be all due to the data source. There has to be something awfully amiss.

pookpooi wrote:

TheCannyOnion wrote:Iyama has risen to #29 on Goratings, probably due to his recent wins against Yamashita Keigo in Kisei title defense.

However, this means Iyama is ranked higher than Park Yeong-hun. Riiiiight... I would find it believable if not for the fact that Park has reached the last two Chunlan Cup finals in a row, beating Lian Xiao (twice), Gu Zihao, Dan Yifei, Zhou Ruiyang, and Ke Jie in the process. These were all strong opponents and, with the exception of Lian Xiao, all world-title holders.

What is up with Goratings? It can't be all due to the data source. There has to be something awfully amiss.

Iyama Yuta is the last year's LG runner-up while #6 Lian Xiao isn't even a runner-up in any international tournament. And the #1 Shin Jinseo didn't top any international championship (Crownless King).

TheCannyOnion wrote:Park beat two players in the world top-10 to get to last year's finals, while Iyama beat NONE to reach LG Cup finals.

pookpooi wrote:

TheCannyOnion wrote:Park beat two players in the world top-10 to get to last year's finals, while Iyama beat NONE to reach LG Cup finals.

Iyama beat #26 Lee Yeongkyu in the first round, #24 Zhou Ruiyang (2013 Bailing Cup winner) in the second round , #8 Yang Dingxin (2019 LG Cup winner) in the third round, #2 Ke Jie in the semi-final, lose 1-2 against #11 Xie Erhao in final.

jlt wrote:Park Yeonghun has recently beaten strong opponents like Park Junghwan (3630) and Gu Zihao (3593), but also lost against low rated players like Kim Hyenchan (3246) and Song Gyusang (3329).

I wanted to check by myself if Iyama's rating reflects his performance against Chinese and Korean players. I considered a player with initial rating 3470, and I made him play the same 17 matches than Iyama's since January 2018 with the same result, and repeated that 10000 times. After each of the 17000 matches, the rating is incremented by c(match_result - 1/(10^D/100+1)) where c=20 is an arbitrary constant and D is the difference between the opponent's rating and the current player's rating.

The player ended with a rating of... 3339 points, which puts him at the 98th place on goratings.

The constant c=20 is arbitrary, but changing it to other reasonable values (between 1 and 40) doesn't change the conclusion much.

I repeated the same experiment with the 14 matches in 2017 against Chinese and Korean opponents. The final rating is 3524, which corresponds to the 15th place.

If we take into account all the 31 matches since January 2017, we get a rating of 3410 (rank=54th).

My conclusion is that maybe Iyama is a bit overrated, but the variations are so wild that it's impossible to get a reliable rating by only taking into account non-Japanese opponents.

Here is a scilab code, for those who would like to check.

rating=3470;

opp_ratings=[3652,3652,3639,3519,3641,3587,3475,3496,3502,3502,3500,3609,3531,3631,3578,3488,3516,3517,3517,3431,3647,3565,3447,3588,3585,3448,3483,2969,3458,3465,3542];

match_results=[1,0,0,0,0,0,1,1,0,1,0,0,1,1,0,0,0,1,0,0,0,0,0,1,0,0,0,1,1,1,0];

n=length(opp_ratings);
c=20;

for number_of_loops=1:10000;

for i=1:n;
D=opp_ratings(i)-rating;
S=1/(10^(D/400)+1);
rating=rating+c*(match_results(i)-S);
end

end
disp(rating)

jlt wrote:Park Yeonghun has recently beaten strong opponents like Park Junghwan (3630) and Gu Zihao (3593), but also lost against low rated players like Kim Hyenchan (3246) and Song Gyusang (3329).

I wanted to check by myself if Iyama's rating reflects his performance against Chinese and Korean players. I considered a player with initial rating 3470, and I made him play the same 17 matches than Iyama's since January 2018 with the same result, and repeated that 10000 times. After each of the 17000 matches, the rating is incremented by c(match_result - 1/(10^D/100+1)) where c=20 is an arbitrary constant and D is the difference between the opponent's rating and the current player's rating.

The player ended with a rating of... 3339 points, which puts him at the 98th place on goratings.

The constant c=20 is arbitrary, but changing it to other reasonable values (between 1 and 40) doesn't change the conclusion much.

I repeated the same experiment with the 14 matches in 2017 against Chinese and Korean opponents. The final rating is 3524, which corresponds to the 15th place.

If we take into account all the 31 matches since January 2017, we get a rating of 3410 (rank=54th).

My conclusion is that maybe Iyama is a bit overrated, but the variations are so wild that it's impossible to get a reliable rating by only taking into account non-Japanese opponents.

Here is a scilab code, for those who would like to check.

rating=3470;

opp_ratings=[3652,3652,3639,3519,3641,3587,3475,3496,3502,3502,3500,3609,3531,3631,3578,3488,3516,3517,3517,3431,3647,3565,3447,3588,3585,3448,3483,2969,3458,3465,3542];

match_results=[1,0,0,0,0,0,1,1,0,1,0,0,1,1,0,0,0,1,0,0,0,0,0,1,0,0,0,1,1,1,0];

n=length(opp_ratings);
c=20;

for number_of_loops=1:10000;

for i=1:n;
D=opp_ratings(i)-rating;
S=1/(10^(D/400)+1);
rating=rating+c*(match_results(i)-S);
end

end
disp(rating)

Hi Remi, I don't know too much about Taeil Bai's method. The rating system I usually follow is mamumamu. I agree with you that incremental ratings systems don't work well, and that your system is far superior to those.

My criticism of your system comes when comparing it to mamumamu's ratings. http://sports.geocities.jp/mamumamu0413/total.html It uses Glicko2-system for matches between strongly connected players and MLE concurrent with Glicko2 for international matches.

First, please allow me to confess that my understanding of statistics is probably elementary compared to yours. So I would appreciate patience in enlightening me if my criticism stems from my own ignorance.

When looking at Iyama Yuta's rating. He is current 3546 compared to Ke Jie at 3668. This suggests Iyama Yuta has a 33% chance of winning against Ke Jie. If you look at Mamumamu, it is 9.79 for Iyama Yuta and 10.867 for Ke Jie, which suggests a 22% win rate. This suggests mamumamu and goratings are off by about 100elo for Iyama Yuta.

When compiling the last 19 games (all games since 1/1/2015) of data for Iyama Yuta vs Chinese/Korean professionals, he has 8 wins when I calculate 9.8 wins expected based on goratings.org. This suggests his elo is about 70 points higher than it is. However, it is not statistically significant, as the deviation is less than 1 sigma. However, it is hard to get a statistically significant result with only 19 samples.

If I look at the next highest ranked Japanese player. Ichiriki Ryo. He has played 40 games against Chinese/Korean pros since 1/1/2015. Based on ratings from goratings I expect 16.8 wins, but there were 10. With Ichiriki Ryo, the deviation is just over 2sigma.

If combining the data for Iyama and Ichiriki, the deviation is well over 2sigma, but still under 3 sigma. Adding in more players from Japan to this, or taking the whole data for Japan vs China/Korea matches should further support this point.

I understand that I am only looking at part of the data, and ignoring the passage of time. However, I'm systematically picking out games that fit a certain neutral criteria, then calculating the expected # of wins and comparing to the actual # of wins and finding a statistically significant deviation. As ratings are used to predict future win probability, this suggests to me a flaw in the methodology used to produce the ratings.

My theory is that, although WHR does work better than incremental methods for strongly connected groups of players, it does not work well enough. My theory is further supported if you do a similar analysis for Taiwan vs China/Korea and NA/EU vs China/Korea. China and Korea play many games against each other, but there are far fewer opportunities for Japan/Taiwan/NA/EU to play against China/Korea.

FYI this is the description of mamumamu's methodology. http://sports.geocities.jp/mamumamu0413 ... le004.html He uses glicko2 first to calculate ratings for each country, then uses MLE to adjust the ratings for regional differences, accounting for the fact that glicko2 is insufficient for this adjustment.