Life In 19x19
http://www.lifein19x19.com/

Ideal komi and winning odds, based on pro games.
http://www.lifein19x19.com/viewtopic.php?f=45&t=10517
Page 1 of 2

Author:  Ootakamoku [ Thu Jul 03, 2014 4:49 am ]
Post subject:  Ideal komi and winning odds, based on pro games.

Database with 37380 professional go games.

Prediction performance for elo ranking algorithm (with optimal K), not including first 1/3 of the professional games (which is used exclusively as training data), is 58.0221% when using logistic curve as is the prevailing choice in modern elo implementations. However when using exponential curve the prediction performance increases to 58.0494%. This effectively implies that professional games outcome are based more on the cumulative quality of their moves, and and less on outside factors (than the use of logistic curve would imply).

We can therefor calculate the odds of winning (when below 50%) with 0.5 ^ ( 1 + points * multiplier_per_point ) where each handicap stone is worth 2 * ideal_komi points, and getting to play first move is worth ideal_komi points. So we need to figure out ideal_komi and multiplier_per_point. In order to do that..

I looked for pairs of pro games, where same two players had played against each other, within a years time span, with same komi, with alternating colors.

I then counted how often the same player won both games, and how often black won games.

5.5 komi, 4288 samples, 54.81% both won by same player, 54.13% won by black.
6.5 komi, 8064 samples, 56.29% both won by same player, 50.06% won by black.
7.5 komi, 4672 samples, 50.47% both won by same player, 44.93% won by black.

Its worth noting, that when using pass stone where white always gets last move, territory and area scoring produce equal results. The 7.5 komi with no such rule in place, is equivalent to 7.0 komi when compared to territory scoring.

Obviously, if players were of equal strength and komi was fair, then 50% of the games should be won by same player and 50% won by black.

Anyways, putting these trough a simulated annealing, trying to estimate average strength difference between players (in terms of extra komi he can give), what is ideal komi, and finally what is multiplier per point for win odds assuming exponential curve. Simulated annealing results in..

ideal komi : 6.516
multiplier per point : 0.1737
strength difference in points (5.5) : 3.64
strength difference in points (6.5) : 3.27
strength difference in points (7.5) : 1.31

these values ends up predicting..

5.5 komi, 54.80% both won by same player, 54.08% won by black.
6.5 komi, 56.29% both won by same player, 50.22% won by black.
7.5 komi, 50.94% both won by same player, 47.43% won by black.

So each handicap stone given drops odds of winning by 79.2%.

Interesting sidenote, it seems like pairing in chinese pro games is more evenly matched. Or it could indicate that chinese pro players are in general closer in strength to each other than say japanese players. One possible explanation is bias from which games are recorded.

Original First Post wrote:
In the database there are 12012 games with 7.5 komi, 19151 games with 6.5 komi. Black player won 47.977% of the games with 7.5 komi, compared with 50.713% when komi was 6.5. Assuming an exponential curve, we can get the following two equations.

0.5^(1 + ( 7.5-x) * y) = 0.4799
0.5^(1 + ( x- 6.5 ) * y) = ( 1 - 0.50713 )

where x is ideal komi, and y is multiplier per point. solving the equations we end up with ideal komi of 6.75929 and multiplier per point of 0.0799152.

Since komi is compensation for half a move. a full move is worth twice that of komi. Meaning, one handicap stone is worth 13.51858 points.

As an example, based on these results the odds of winning an even game against someone 3 stones stronger would be 5.2%, as per 0.5^( 1 + 3 * 13.51858 * 0.0799152 ) = 0.052...

Author:  ez4u [ Thu Jul 03, 2014 6:43 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

Ootakamoku wrote:
Database with 37380 professional go games.

Prediction performance for elo ranking algorithm (with optimal K), not including first 1/3 of the professional games (which is used exclusively as training data), is 58.0221% when using logistic curve as is the prevailing choice in modern elo implementations. However when using exponential curve the prediction performance increases to 58.0494%. This effectively implies that professional games outcome are based more on the cumulative quality of their moves, and and less on outside factors (than the use of logistic curve would imply).

In the database there are 12012 games with 7.5 komi, 19151 games with 6.5 komi. Black player won 47.977% of the games with 7.5 komi, compared with 50.713% when komi was 6.5. Assuming an exponential curve, we can get the following two equations.

0.5^(1 + ( 7.5-x) * y) = 0.4799
0.5^(1 + ( x- 6.5 ) * y) = ( 1 - 0.50713 )

where x is ideal komi, and y is multiplier per point. solving the equations we end up with ideal komi of 6.75929 and multiplier per point of 0.0799152.

Since komi is compensation for half a move. a full move is worth twice that of komi. Meaning, one handicap stone is worth 13.51858 points.

As an example, based on these results the odds of winning an even game against someone 3 stones stronger would be 5.2%, as per 0.5^( 1 + 3 * 13.51858 * 0.0799152 ) = 0.052...

What does the expression "...not including first 1/3 of the professional games (which is used exclusively as training data)..." mean? Did you segregate your database into two parts based on the dates the games were played or did you randomly select 1/3 for training data? If you used dates, do you get the same result if you choose the oldest 1/3 games as your training data? I would expect this to be the case if no adjustment to the 6.5 komi occurred among professionals. However, I know from studying fuseki trends that around 2002-2003 no major fuseki system for Black won over 50% as (apparently) the pros had not worked out how to play against the increase komi.

A quick and dirty look at even games with 6.5 komi in my database gives the following results by time period:
Code:
Period           Games  Black win %
2002 to 2003    1,873    49.1%
2004 to 2005    3,326    49.5%
2006 to 2007    4,421    50.7%
2008 to 2009    4,483    52.4%
2010 to 2011    4,004    49.9%
2012 to 2014    4,670    50.6%
.....
2002 to 2014    22,777    50.6%

Author:  Ootakamoku [ Thu Jul 03, 2014 7:05 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

ez4u wrote:
What does the expression "...not including first 1/3 of the professional games (which is used exclusively as training data)..." mean? Did you segregate your database into two parts based on the dates the games were played or did you randomly select 1/3 for training data? If you used dates, do you get the same result if you choose the oldest 1/3 games as your training data? I would expect this to be the case if no adjustment to the 6.5 komi occurred among professionals. However, I know from studying fuseki trends that around 2002-2003 no major fuseki system for Black won over 50% as (apparently) the pros had not worked out how to play against the increase komi.

Yes, I use first 1/3 of the games by date as training data. My intention is to exclude the high variance scenario where lots of new players are introduced into the system and everyones ranks are unreliable.


ez4u wrote:
A quick and dirty look at even games with 6.5 komi in my database gives the following results by time period:
Code:
Period           Games  Black win %
2002 to 2003    1,873    49.1%
2004 to 2005    3,326    49.5%
2006 to 2007    4,421    50.7%
2008 to 2009    4,483    52.4%
2010 to 2011    4,004    49.9%
2012 to 2014    4,670    50.6%
.....
2002 to 2014    22,777    50.6%

This fluctuation is probably mostly due to variance from much smaller sample size. Tho just to make sure, Ill run it again, using only more recent pro games.

Author:  RBerenguel [ Thu Jul 03, 2014 11:14 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

By the way Oota, since you love ranking/rating stuff, I've tried running NGA monthly results through a pagerank-like algorithm (well, it's actually pagerank but applied to game results.) It gives somewhat interesting month-to-month gradings :D

Author:  Matti [ Fri Jul 04, 2014 4:35 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

Ootakamoku wrote:
ez4u wrote:
What does the expression "...not including first 1/3 of the professional games (which is used exclusively as training data)..." mean? Did you segregate your database into two parts based on the dates the games were played or did you randomly select 1/3 for training data? If you used dates, do you get the same result if you choose the oldest 1/3 games as your training data? I would expect this to be the case if no adjustment to the 6.5 komi occurred among professionals. However, I know from studying fuseki trends that around 2002-2003 no major fuseki system for Black won over 50% as (apparently) the pros had not worked out how to play against the increase komi.

Yes, I use first 1/3 of the games by date as training data. My intention is to exclude the high variance scenario where lots of new players are introduced into the system and everyones ranks are unreliable.


ez4u wrote:
A quick and dirty look at even games with 6.5 komi in my database gives the following results by time period:
Code:
Period           Games  Black win %
2002 to 2003    1,873    49.1%
2004 to 2005    3,326    49.5%
2006 to 2007    4,421    50.7%
2008 to 2009    4,483    52.4%
2010 to 2011    4,004    49.9%
2012 to 2014    4,670    50.6%
.....
2002 to 2014    22,777    50.6%

This fluctuation is probably mostly due to variance from much smaller sample size. Tho just to make sure, Ill run it again, using only more recent pro games.

By testing I found no significant trend over years.

Author:  Abyssinica [ Fri Jul 04, 2014 6:08 pm ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

When we go over professional go games and winning percentages, how do we account for the fact that all pros aren't actually within one stone of each other? Do you only count same dan against same dan? And even then, strengths wax and wane while the dan rank never goes down.

Author:  Ootakamoku [ Sat Jul 05, 2014 1:48 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

Abyssinica wrote:
When we go over professional go games and winning percentages, how do we account for the fact that all pros aren't actually within one stone of each other? Do you only count same dan against same dan? And even then, strengths wax and wane while the dan rank never goes down.


It doesn't matter, since rank doesn't affect who gets to play with black (instead nigiri or lottery is used). On average black is stronger player just as often as white is. So the strength difference cancels out.

Author:  Ootakamoku [ Wed Aug 13, 2014 3:59 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

Just realized my error in the math. Problem is if there is a nonzero different in players strength, in that case it would skew the results towards 50% wins for both sides. As the winner was predetermined by his strength, not leaving room for the komi to affect the result. So the make my original calculations work, I would need to determine which players are of equal strength, ie I would need some statistics on how well they have done against each other over multiple games.

So the numbers I produced are for minimum effect of komi, and its bound to be much greater in practice. Which explains the surprisingly high odds for underdog.

Author:  Ootakamoku [ Wed Aug 13, 2014 4:37 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

With some quick changes to take this problem into account, I got the result that multiplier per point should be 0.093 instead of 0.080, but it was pretty ugly way I managed to pull out that number.

Author:  kivi [ Wed Aug 13, 2014 5:39 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

I don't understand this. Are you saying ideal komi is 6.75929 (based on this calculation)? What does multiplier per point mean?

Author:  Ootakamoku [ Wed Aug 13, 2014 6:00 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

kivi wrote:
I don't understand this. Are you saying ideal komi is 6.75929 (based on this calculation)? What does multiplier per point mean?


Its a measure of how much chances of winning decrease for each point of komi opponent gets. Using the formula 0.5^(1 + ( opponents_komi - ideal_komi ) * multiplier_per_point) = odds of winning (when opponents_komi > ideal_komi ). So the higher the multiplier the more playing say without komi favors black.

Author:  kivi [ Wed Aug 13, 2014 6:31 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

Ootakamoku wrote:
kivi wrote:
I don't understand this. Are you saying ideal komi is 6.75929 (based on this calculation)? What does multiplier per point mean?


Its a measure of how much chances of winning decrease for each point of komi opponent gets. Using the formula 0.5^(1 + ( opponents_komi - ideal_komi ) * multiplier_per_point) = odds of winning (when opponents_komi > ideal_komi ). So the higher the multiplier the more playing say without komi favors black.

So that's the multiplier.
But don't you think 6.75 komi is, to put it blatantly, a bit of a nonsense?
Say we were to ask the same pros to repeat those thousands of games with 6.75 komi this time. Expecting a win ratio closer to 0.5 doesn't make sense, as 6.1 komi and 6.9 komi gives the same result in a game.

Author:  jeromie [ Wed Aug 13, 2014 7:33 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

kivi wrote:
So that's the multiplier.
But don't you think 6.75 komi is, to put it blatantly, a bit of a nonsense?

His result doesn't tell us we should set komi at 6.75; it tells us that 6.5 is not quite enough and 7.5 is a bit too much. We have to translate the continuous value the equations calculate to the .5 increments used for real komi.

Author:  kivi [ Wed Aug 13, 2014 7:53 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

jeromie wrote:
kivi wrote:
So that's the multiplier.
But don't you think 6.75 komi is, to put it blatantly, a bit of a nonsense?

His result doesn't tell us we should set komi at 6.75; it tells us that 6.5 is not quite enough and 7.5 is a bit too much. We have to translate the continuous value the equations calculate to the .5 increments used for real komi.


Well, isn't that ("6.5 not enough, 7.5 too much") obvious given that 7.5 komi leads to 47.977% win and 6.5 komi is 50.713% win.
Alternatively, if you fit a straight line instead of an exponential curve, it would also fit the data perfectly and give another ideal_komi. That value would also be between 6.5 and 7.5, leading to the same deduction that 6.5 is not enough, 7.5 is too much. So what's the point of the exponential model, or the significance of the value of 6.75?

Author:  jeromie [ Wed Aug 13, 2014 8:44 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

kivi wrote:
So what's the point of the exponential model, or the significance of the value of 6.75?

It sounds like the exponential model was chosen as a nod to other ranking systems. I suppose knowing the "real" value of a continuous Komi might be interesting in some situations (e.g. Predicting the value of each handicap stone in high level games could be used to select an appropriate reverse komi), but I think this is mostly an exercise undertaken because math is fun. :-)

Author:  Ootakamoku [ Thu Aug 14, 2014 1:11 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

To get around the problem. I looked for pairs of games, where same two players had played against each other, within a years timespan, with same komi, with alternating colors.

I then counted how often the same player won both games, and how often black won games.

5.5 komi, 53.26% both won by same player, 54.23% won by black.
6.5 komi, 56.74% both won by same player, 49.88% won by black.
7.5 komi, 51.15% both won by same player, 48.06% won by black.

Obviously, if players were of equal strength and komi was fair, then 50% of the games should be won by same player and 50% won by black.

Anyways, putting these trough a simulated annealing, trying to estimate average strength difference between players (in terms of extra komi he can give), what is ideal komi, and finally what is multiplier per point for win odds assuming exponential curve. Simulated annealing results in..

ideal komi : 6.8143
multiplier per point : 0.1148
strength difference (5.5) : 3.9187
strength difference (6.5) : 5.7777
strength difference (7.5) : 2.2413

these values ends up predicting..

5.5 komi, 53.19% both won by same player, 53.83% won by black.
6.5 komi, 56.77% both won by same player, 50.79% won by black.
7.5 komi, 51.21% both won by same player, 47.71% won by black.

Interesting sidenote, it seems like pairing in chinese pro games is more evenly matched. Or it could indicate that chinese pro players are in general closer in strength to each other than say japanese players. One possible explanation is bias from which games are recorded.

Author:  Ootakamoku [ Thu Aug 14, 2014 1:17 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

jeromie wrote:
kivi wrote:
So what's the point of the exponential model, or the significance of the value of 6.75?

It sounds like the exponential model was chosen as a nod to other ranking systems. I suppose knowing the "real" value of a continuous Komi might be interesting in some situations (e.g. Predicting the value of each handicap stone in high level games could be used to select an appropriate reverse komi), but I think this is mostly an exercise undertaken because math is fun. :-)


The ideal komi is a byproduct. What I find interesting, is being able to calculate the odds a player wins if he is 1 stone stronger, or two stones stronger, etc. Or in reverse, determine the strength difference between two players, based on their win ratio over multiple games.

Author:  Ootakamoku [ Thu Aug 14, 2014 3:10 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

Further discovery, ELO based on exponential was only better estimation, when ideal komi wasn't calculated correctly. Ie when games were assumed to be fair (say 5.5 komi) while it infact favored one of the players, in this case exponential curve produced better results. But when we have a more precise measure of komi, logistic curve actually produces better results. Why does this have to be so complicated :(

Author:  HermanHiddema [ Thu Aug 14, 2014 3:36 am ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

You don't mention your sample sizes, but it looks like you've got a huge case of false precision going on. Unless you specifically give error bars, you should only give the number of significant figures that is warranted by the data, and I think it is unlikely that your data is accurate to within 0.01%

Author:  Ootakamoku [ Thu Aug 14, 2014 9:50 pm ]
Post subject:  Re: Ideal komi and winning odds, based on pro games.

HermanHiddema wrote:
You don't mention your sample sizes, but it looks like you've got a huge case of false precision going on. Unless you specifically give error bars, you should only give the number of significant figures that is warranted by the data, and I think it is unlikely that your data is accurate to within 0.01%


The first post, which I had updated to reflect the most recent results also contains sample sizes. Forgot to include it in the last post.

Page 1 of 2 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/