Life In 19x19

Rémi wrote:Rating algorithms must be tested on real data. You can generate artificial data based on some model, and then the best rating system would be the rating system that assumes this model. But the fact that an algorithm is the best to predict the artificial data does not imply that it will be the best to predict the real data. The only way to measure the ability of an algorithm to predict real game outcomes is to measure how well it predicts real game outcomes.

Thank you for your answer, but I think you misunderstood my point. The goal would not be to test the algorithm itself, it would be to get a rough idea of the theoretical best prediction rate that could be achieved ("rough" because it would be an approximation of the KGS database).

Going from 55.7% to 55.8% of prediction rate would look very different if you can give a convincing argument the prediction rate should be capped at around, say, 57%, as compared as if you could (theoretically) reach 100%.

I can't see how a player that nearly never plays international games can be ranked #3 in the world.
I like Iyama but he doesn't deserve that spot.
All his competitors in Japan are out of top 20 in the world(most would actually think top 50) so how can his vicotories count so high?

Continual victories against his domestic opponents will continue to increase his rank, even if marginal, unless there's a cap in the algorithm where one will no longer gain points unless he plays someone strong enough. I haven't read the paper for the WHR algorithm.

He is on a hot streak. Unless I missed a game, still might hold all 7 big titles in Japan at the same time. But I would like to see Japan compete better at an international level.

Jhyn wrote:

Rémi wrote:Rating algorithms must be tested on real data. You can generate artificial data based on some model, and then the best rating system would be the rating system that assumes this model. But the fact that an algorithm is the best to predict the artificial data does not imply that it will be the best to predict the real data. The only way to measure the ability of an algorithm to predict real game outcomes is to measure how well it predicts real game outcomes.

Thank you for your answer, but I think you misunderstood my point. The goal would not be to test the algorithm itself, it would be to get a rough idea of the theoretical best prediction rate that could be achieved ("rough" because it would be an approximation of the KGS database).

Going from 55.7% to 55.8% of prediction rate would look very different if you can give a convincing argument the prediction rate should be capped at around, say, 57%, as compared as if you could (theoretically) reach 100%.

Keep in mind that every prediction needs to be based on data. Every rating algorithm I know of only uses past records, as it's ostensibly the most efficient option. A more convoluted model (thus adding variance, allowing for better prediction in the first place) would have to take more data into account than pure W/L and rating difference. These additional parameters could be game-related (winning margin - a terrible idea, i know), individual characteristics (e.g.: average thinking time) for the relevant match characteristics (here: time setting) and so on and so forth. Of course you would first have to have those data available, then invest additional computing power,... but there's only so much you can squeeze out of a single binary variable.

DrStraw wrote:One fact I find remarkable: before 1988 all the top three were Japanese; since 1994 not a single Japanese player has been in the top 3. This is an amazingly quick turnaround.

Also interesting is that in 36 years only 5 players have been at the top.

Something has now changed, and Cho Hunhyeon first hits 1st place in 1984. I supposed some older games might have been added? Unfortunately, there still aren't any of his games against Japanese players from the early 80s.

Good observations. Go4Go collection now has thousands of more Korean games covering late 1980's and the whole 1990's. Still a work in progress, but this already has significant impact to the early ratings.

Could you consider adding some of the games of Master/Alpha Go to go4go.net, I believe that this would help us to understand how much stronger it is to human now.

These are blitz games, and some opponent's names are not fully confirmed, but I understand that go4go.net also includes internet-based games, the Japan national team training games for example, so I think it makes sense to add those Alpha Go's games.

I actually want to make a poll on whether ranking (not just goratings) should include bots.
I think they're strong enough to have their own ranking, just like chess AI.
But if it'll continue to be in human list, just make sure it doesn't mess up human ranking order, as bot's progression is very different from human.

pookpooi wrote:I actually want to make a poll on whether ranking (not just goratings) should include bots.
I think they're strong enough to have their own ranking, just like chess AI.
But if it'll continue to be in human list, just make sure it doesn't mess up human ranking order, as bot's progression is very different from human.

I will probably remove them from goratings, anyway, because the statistical model I use does not fit the way their strength may changes in time.

Rémi wrote:

pookpooi wrote:I actually want to make a poll on whether ranking (not just goratings) should include bots.
I think they're strong enough to have their own ranking, just like chess AI.
But if it'll continue to be in human list, just make sure it doesn't mess up human ranking order, as bot's progression is very different from human.

I will probably remove them from goratings, anyway, because the statistical model I use does not fit the way their strength may changes in time.

A little update: Right now AlphaGo and DeepZenGo is removed from the list. So we're back to human only list again. Which is very good in my opinion.
Remi, do you plan to have a new model that human and bot can co-exist?

pookpooi wrote:A little update: Right now AlphaGo and DeepZenGo is removed from the list. So we're back to human only list again. Which is very good in my opinion.
Remi, do you plan to have a new model that human and bot can co-exist?

No. It is not really possible to find a statistical model of computer strength variation in time, anyway. Each new version should be considered a different player.

1. I don't like the new spelling, even though it's more accurate (like Lee Sedol to Yi Se-tol and changing order from Iyama Yuta to Yuta Iyama) When I search for Lee Hajin I found nothing, because she change to Yi Ha-chin.
2. Flag should be the nation in which that person receive professional graduation from, not his/her nationality. This way you don't have to update every time pro move out of his/her origin country.
3. Is data enough for expanding historical rating backward further from 1980 to 1970?

pookpooi wrote:1. I don't like the new spelling, even though it's more accurate (like Lee Sedol to Yi Se-tol and changing order from Iyama Yuta to Yuta Iyama) When I search for Lee Hajin I found nothing, because she change to Yi Ha-chin.

I agree with this. According to the wiki page for the revised romanization of Korean (https://en.wikipedia.org/wiki/Revised_R ... _of_Korean):

Revised Romanization is not expected to be adopted as the official romanization of Korean family names, and few people have voluntarily adopted it. According to a 2009 study by the National Institute of the Korean Language based on 63,351 applications for South Korean passports in 2007, for each of the three most common surnames Kim (김), Lee (이), and Park (박), less than 2% of applicants asked for their surname to be romanized in their passport by using the respective Revised Romanization spelling Gim, I, or Bak.[3] Given names and commercial names are encouraged to change, but it is not required.

My wife's maiden name is Lee, and she still spells it that way. I don't think it's useful to force someone's name to be spelled in a way that they don't want it to be spelled.

Besides, McCune–Reischauer isn't even the latest romanization scheme, anyway.

Thanks for your feedback about the new version of goratings. This version uses data from https://db.u-go.net/ to display player names in English, Chinese, Japanese, and Korean.

go4go.net only provides English and Chinese names. I might go back to using go4go.net's spelling for English and Chinese if visitors prefer it. I noticed a few problems, like the Chinese name of Yoshio Ishida.

The data at https://db.u-go.net/ is incomplete, as you can see by looking at the Korean and Japanese versions of goratings. Your contribution is welcome.

Maybe I should use the Chinese writing by default in the Japanese page, too.

Pardon me, but how is Yuta Iyama more accurate than Iyama Yuta?