Database with 37380 professional go games.

Prediction performance for elo ranking algorithm (with optimal K), not including first 1/3 of the professional games (which is used exclusively as training data), is 58.0221% when using logistic curve as is the prevailing choice in modern elo implementations. However when using exponential curve the prediction performance increases to 58.0494%. This effectively implies that professional games outcome are based more on the cumulative quality of their moves, and and less on outside factors (than the use of logistic curve would imply).

We can therefor calculate the odds of winning (when below 50%) with 0.5 ^ ( 1 + points * multiplier_per_point ) where each handicap stone is worth 2 * ideal_komi points, and getting to play first move is worth ideal_komi points. So we need to figure out ideal_komi and multiplier_per_point. In order to do that..

I looked for pairs of pro games, where same two players had played against each other, within a years time span, with same komi, with alternating colors.

I then counted how often the same player won both games, and how often black won games.

5.5 komi, 4288 samples, 54.81% both won by same player, 54.13% won by black.
6.5 komi, 8064 samples, 56.29% both won by same player, 50.06% won by black.
7.5 komi, 4672 samples, 50.47% both won by same player, 44.93% won by black.

Its worth noting, that when using pass stone where white always gets last move, territory and area scoring produce equal results. The 7.5 komi with no such rule in place, is equivalent to 7.0 komi when compared to territory scoring.

Obviously, if players were of equal strength and komi was fair, then 50% of the games should be won by same player and 50% won by black.

Anyways, putting these trough a simulated annealing, trying to estimate average strength difference between players (in terms of extra komi he can give), what is ideal komi, and finally what is multiplier per point for win odds assuming exponential curve. Simulated annealing results in..

ideal komi : 6.516
multiplier per point : 0.1737
strength difference in points (5.5) : 3.64
strength difference in points (6.5) : 3.27
strength difference in points (7.5) : 1.31

these values ends up predicting..

5.5 komi, 54.80% both won by same player, 54.08% won by black.
6.5 komi, 56.29% both won by same player, 50.22% won by black.
7.5 komi, 50.94% both won by same player, 47.43% won by black.

So each handicap stone given drops odds of winning by 79.2%.

Interesting sidenote, it seems like pairing in chinese pro games is more evenly matched. Or it could indicate that chinese pro players are in general closer in strength to each other than say japanese players. One possible explanation is bias from which games are recorded.

What does the expression "...not including first 1/3 of the professional games (which is used exclusively as training data)..." mean? Did you segregate your database into two parts based on the dates the games were played or did you randomly select 1/3 for training data? If you used dates, do you get the same result if you choose the oldest 1/3 games as your training data? I would expect this to be the case if no adjustment to the 6.5 komi occurred among professionals. However, I know from studying fuseki trends that around 2002-2003 no major fuseki system for Black won over 50% as (apparently) the pros had not worked out how to play against the increase komi.

A quick and dirty look at even games with 6.5 komi in my database gives the following results by time period:

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

Yes, I use first 1/3 of the games by date as training data. My intention is to exclude the high variance scenario where lots of new players are introduced into the system and everyones ranks are unreliable.



This fluctuation is probably mostly due to variance from much smaller sample size. Tho just to make sure, Ill run it again, using only more recent pro games.

By the way Oota, since you love ranking/rating stuff, I've tried running NGA monthly results through a pagerank-like algorithm (well, it's actually pagerank but applied to game results.) It gives somewhat interesting month-to-month gradings

_________________
Geek of all trades, master of none: the motto for my blog mostlymaths.net

By testing I found no significant trend over years.

When we go over professional go games and winning percentages, how do we account for the fact that all pros aren't actually within one stone of each other? Do you only count same dan against same dan? And even then, strengths wax and wane while the dan rank never goes down.

It doesn't matter, since rank doesn't affect who gets to play with black (instead nigiri or lottery is used). On average black is stronger player just as often as white is. So the strength difference cancels out.

Just realized my error in the math. Problem is if there is a nonzero different in players strength, in that case it would skew the results towards 50% wins for both sides. As the winner was predetermined by his strength, not leaving room for the komi to affect the result. So the make my original calculations work, I would need to determine which players are of equal strength, ie I would need some statistics on how well they have done against each other over multiple games.

So the numbers I produced are for minimum effect of komi, and its bound to be much greater in practice. Which explains the surprisingly high odds for underdog.

With some quick changes to take this problem into account, I got the result that multiplier per point should be 0.093 instead of 0.080, but it was pretty ugly way I managed to pull out that number.

I don't understand this. Are you saying ideal komi is 6.75929 (based on this calculation)? What does multiplier per point mean?

Its a measure of how much chances of winning decrease for each point of komi opponent gets. Using the formula 0.5^(1 + ( opponents_komi - ideal_komi ) * multiplier_per_point) = odds of winning (when opponents_komi > ideal_komi ). So the higher the multiplier the more playing say without komi favors black.

So that's the multiplier.
But don't you think 6.75 komi is, to put it blatantly, a bit of a nonsense?
Say we were to ask the same pros to repeat those thousands of games with 6.75 komi this time. Expecting a win ratio closer to 0.5 doesn't make sense, as 6.1 komi and 6.9 komi gives the same result in a game.

His result doesn't tell us we should set komi at 6.75; it tells us that 6.5 is not quite enough and 7.5 is a bit too much. We have to translate the continuous value the equations calculate to the .5 increments used for real komi.

Well, isn't that ("6.5 not enough, 7.5 too much") obvious given that 7.5 komi leads to 47.977% win and 6.5 komi is 50.713% win.
Alternatively, if you fit a straight line instead of an exponential curve, it would also fit the data perfectly and give another ideal_komi. That value would also be between 6.5 and 7.5, leading to the same deduction that 6.5 is not enough, 7.5 is too much. So what's the point of the exponential model, or the significance of the value of 6.75?

It sounds like the exponential model was chosen as a nod to other ranking systems. I suppose knowing the "real" value of a continuous Komi might be interesting in some situations (e.g. Predicting the value of each handicap stone in high level games could be used to select an appropriate reverse komi), but I think this is mostly an exercise undertaken because math is fun.

To get around the problem. I looked for pairs of games, where same two players had played against each other, within a years timespan, with same komi, with alternating colors.

I then counted how often the same player won both games, and how often black won games.

5.5 komi, 53.26% both won by same player, 54.23% won by black.
6.5 komi, 56.74% both won by same player, 49.88% won by black.
7.5 komi, 51.15% both won by same player, 48.06% won by black.

Obviously, if players were of equal strength and komi was fair, then 50% of the games should be won by same player and 50% won by black.

Anyways, putting these trough a simulated annealing, trying to estimate average strength difference between players (in terms of extra komi he can give), what is ideal komi, and finally what is multiplier per point for win odds assuming exponential curve. Simulated annealing results in..

ideal komi : 6.8143
multiplier per point : 0.1148
strength difference (5.5) : 3.9187
strength difference (6.5) : 5.7777
strength difference (7.5) : 2.2413

these values ends up predicting..

5.5 komi, 53.19% both won by same player, 53.83% won by black.
6.5 komi, 56.77% both won by same player, 50.79% won by black.
7.5 komi, 51.21% both won by same player, 47.71% won by black.

Interesting sidenote, it seems like pairing in chinese pro games is more evenly matched. Or it could indicate that chinese pro players are in general closer in strength to each other than say japanese players. One possible explanation is bias from which games are recorded.

The ideal komi is a byproduct. What I find interesting, is being able to calculate the odds a player wins if he is 1 stone stronger, or two stones stronger, etc. Or in reverse, determine the strength difference between two players, based on their win ratio over multiple games.

Further discovery, ELO based on exponential was only better estimation, when ideal komi wasn't calculated correctly. Ie when games were assumed to be fair (say 5.5 komi) while it infact favored one of the players, in this case exponential curve produced better results. But when we have a more precise measure of komi, logistic curve actually produces better results. Why does this have to be so complicated

You don't mention your sample sizes, but it looks like you've got a huge case of false precision going on. Unless you specifically give error bars, you should only give the number of significant figures that is warranted by the data, and I think it is unlikely that your data is accurate to within 0.01%

The first post, which I had updated to reflect the most recent results also contains sample sizes. Forgot to include it in the last post.