Fun with sanrensei

[gennan]

Back in the 1970s statistical analysis of Japanese pro-pro handicap games suggested that, up to 9 stones, each handicap stone was worth a little less than 14 pts. The relationship was surprisingly linear. It was on that basis that I predicted that Japanese komi would increase to 6½ pts. by the year 2000. Almost! OC, each stone corresponds to a one amateur rank difference.

I just did a quick test with KataGo to estimate the actual value of handicap stones in terms of points. I also listed an idealized value using 14 points / stone.

Code: Select all

handicap actual idealized difference 
1          9        7       +2
2         28       21       +7
3         42       35       +7
4         55       49       +6
5         59       63       -4
6         70       77       -7
7         83       91       -8
8        101      105       -4
9        120      119       +1

So indeed it's fairly linear and close to 14 points / stone (within half a stone).
It would be interesting to find a simple and symmetric handicap stone placement scheme that would be actually linear with 14 points / stone.

[Uberdude]

Interesting how KataGo thinks tengen when going from 4 to 5 stones isn't very useful at just +4 points ("just one more stone to kill" ), but when going from 8 to 9 is +19. But how much has KataGo been trained on positions as lopsided as an initial handicap so that it can make reliable judgements? Or do we just trust the magic extrapolation abilities of neural networks?

[gennan]

Interesting how KataGo thinks tengen when going from 4 to 5 stones isn't very useful at just +4 points ("just one more stone to kill" ), but when going from 8 to 9 is +19. But how much has KataGo been trained on positions as lopsided as an initial handicap so that it can make reliable judgements? Or do we just trust the magic extrapolation abilities of neural networks?

I think it reflects "first corners, then sides, then center". The 5th handicap stone at tengen is premature. It's too far from black's other stones to create a good synergy. With 7 stones, there is synergy with the side hoshi and with 9 handicap, tengen is very nicely located and maximizing the synergy of all black's stones.

I don't know how accurate KataGo's evaluation is in these positions, but it is fairly consistent with the pro assesment and my feeling of the relative value of the handicap stones. It "feels" right to me.

[xela]

Or do we just trust the magic extrapolation abilities of neural networks?

No we don't :-) But it's fun to read the tea leaves.

Here's a couple of interesting things. First, the winrates (my scores are only slightly different from gennan's: this could be the random number see or a different number of playouts. I'm using 15,000).

Code: Select all

handicap score winrate
1         9.0   74.6%
2        28.7   96.9%
3        44.0   98.8%
4        56.8   99.2%
5        59.8   99.1%

So for more than 2 stones, the winrate difference for each extra stone is too small to measure accurately.

Second, the above numbers (and I guess gennan's too) are obtained by putting the handicap stones on the board, set komi to zero and ask KataGo to evaluate the position. But if you try different choices of komi, then it explores different moves, and this affects the results.

On 2 stones with 29 points komi, it puts black 7 points behind, winrate 31%. But 2 stones with 22 points komi gives black only 0.5 point behind, 48% winrate.

On 4 and 5 stones, the most balanced komi is 53 points for 4 stones and 63 points for 5 stones. So by this measure, the 5th stone might be worth a bit less, but the difference isn't so drastic.

Is either measure more accurate (or less meaningless) than the other?

[xela]

Oh, for completeness:

Code: Select all

handicap   komi for balance     difference from previous row
1           7                   N/A
2          22                   15
3          37                   15
4          53                   16
5          63                   10

Yes, if you run KataGo on an empty board with no komi, it tells you black is 9 points ahead, but if you give white 9 points komi, you don't get an equal position, you get a position with black just over 3 points behind. That's more or less what we'd expect if fair komi is 7.

[Bill Spight]

Or do we just trust the magic extrapolation abilities of neural networks?

No we don't But it's fun to read the tea leaves.

Here's a couple of interesting things. First, the winrates (my scores are only slightly different from gennan's: this could be the random number see or a different number of playouts. I'm using 15,000).

Code: Select all

handicap score winrate
1         9.0   74.6%
2        28.7   96.9%
3        44.0   98.8%
4        56.8   99.2%
5        59.8   99.1%

So for more than 2 stones, the winrate difference for each extra stone is too small to measure accurately.

Second, the above numbers (and I guess gennan's too) are obtained by putting the handicap stones on the board, set komi to zero and ask KataGo to evaluate the position. But if you try different choices of komi, then it explores different moves, and this affects the results.

On 2 stones with 29 points komi, it puts black 7 points behind, winrate 31%. But 2 stones with 22 points komi gives black only 0.5 point behind, 48% winrate.

On 4 and 5 stones, the most balanced komi is 53 points for 4 stones and 63 points for 5 stones. So by this measure, the 5th stone might be worth a bit less, but the difference isn't so drastic.

Is either measure more accurate (or less meaningless) than the other?

The question I would ask, as you did, is what is the proper komi for each handicap.

IMX, however, one stone per rank is a better measure than X pts. per rank. Somewhat surprising, and I only have a few examples. But I did give 40 stones once and won by 10 pts.