I recently looked at this 'modern' pattern, with the two space low pincer which is usually the highest rated pincer by AI standards.

In the pattern, there are two main variations one can play, one of which ends with a local seki.
This seki is interesting, because it involves two eyes, of which one side has a bigger eye.

From how I understand Japanese/Chinese rules, in this corner, under Japanese rules, the players do not earn points for the eyes in seki, and thus have the same amount of points locally by each having one capture.

In Chinese rules however, from how I see it, white will have the chance to place two additional stones, meaning that white has 2 more points.

If you analyse this position with Katago (I am using the latest 40 block network with 25k+ playouts)
you will find that it's evaluation of the position changes depending on the ruleset, and the parity of the komi.

Japanese rules 6.5 komi
Seki: Black leading by 0,1 with 53,5% winrate
Black corner: White leading by 1,0 with 60,2% winrate

Chinese rules 7.5 komi
Seki: White leading by 1,0 with 57,2% winrate
Black corner: White leading by 1,3 with 64,4% winrate

Chinese rules 6.5 komi
Seki: White leading by 1,2 with 54,1% winrate
Black corner: White leading by 0,4 with 49,7% winrate

Chinese rules 5.5 komi
Seki: Black leading by 0,9 with 56,7% winrate
Black corner: Black leading by 0,5 with 49,9% winrate


We see when comparing the two positions in the SGF file, with japanese rules there is approximately one point difference.
In Chinese rules, we see that depending on the parity, we have a difference in the two positions that varies from being less than half a point, to being close to 1 point.

I want to ask this forum, if my understanding of the difference in Chinese/Japanese counting is correct, and if anyone can make a good argument as to why the difference in evaluation of the two positions could be as it is.

Of course the first point is that the AI may not fully understand the position, but then what would be the theoretical difference between the position when comparing Japanese and Chinese rulesets?

As I see it, in practice, playing Chinese rules (throwing away prisoners), the corner is big enough for Black to play in sente at B18, White captures and Black throws in in sente again at B18, so the result becomes exactly equivalent to Japanese rules. White has a theoretical opportunity to play at B18 first and then he will have 2 points (zi) versus 1 in the seki under Chinese rules, but that would apply only very late in the game, and of course would also imply giving Black a move elsewhere. Further, and in again in practice, Black can compensate for that tiny extra opportunity for White by keeping his sente moves in reserve as very big ko threats.

So, yes, it does seem strange for the bot to make such a big difference in score out of something that is, at most, a tiny difference in practice.

We need Bill!

If you count territory in seki, as under Chinese and AGA rules White has 4 pts. in the corner and Black has 2 pts. for a difference of 2 pts. to White. Under Japanese and Korean rules each side has 1 pt. for one captured stone. White just has to capture the stone at or after the dame filling stage. Under Japanese or Korean rules Black could give up two extra stones in a losing ko threat, but that would be unusual. The dame in seki alters the parity of the board unless there is another such seki on the board, which would also be unusual.

Assuming no losing ko threat and even parity we have the following table, with 7½ area komi and 6½ territory komi.



The usual relationship, with equal territory and odd parity, is shown in the next table.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

I realised later that I was forgetting to count the stones on the board after White captures. Funny how these things come to your in your subconscious!

I find the given summary of results a little tricky hard to read because different rows switch which side the value is quoted for. So here's one where all values are white's perspective:


Firstly, you should be aware that KataGo's judgments on complex sekis have a lot more noise/bias than usual. The problem is that there are a lot of weird seki shapes, and they're not all that common, so you don't get a lot of data. In the case of sekis, KataGo's score estimation may be "off" by easily one third to one half of a point, and sometimes by around a point. The winrate judgment can also be off by anywhere from a few percent to rarely as much as 10%.

I think KataGo might be mildly worse in certain sekis than other bots due to being faced with a much harder problem - understanding in general how sekis interact with variable komi and variable board sizes and variable rules, instead of being able to specialize to just 7.5 komi 19x19 Tromp-Taylor. As has become apparent here, seki can interact with the parity of the board in a confusing way and the rules in a nontrivial way, and the neural net finds this hard too! And sekis are specifically a point of major difference between Japanese and Chinese or Tromp-Taylor rules. However, it's a one-time bias for any board position with an unusual seki - you should be able to continue to "trust" (or not) KataGo's evaluations for everything else on the board subsequently as you would normally, since the seki position will be shared in all the variations elsewhere. And also the error will probably diminish as you get close to the end of the game.

Okay having said all that, let's talk about the evaluations in this specific position.

Corner variation
The corner variation evaluations look good and consistent to me:

Assuming we take Japanese rules 6.5 komi as a baseline, then by standard parity arguments Chinese 7.5 komi should be about a half a point better for white, so we'd expect "1,5" lead instead, and a corresponding bump in winrate. We see "1,3" instead, and bump in winrate as expected. So a possible error of perhaps 0.2 points. And the last two results fall right in line with expectations. The score/lead should drop by 1 point each time, but the winrate should only drop during 7.5 -> 6.5. Winrate should basically not drop from 6.5 -> 5.5 because of parity. And since the baseline expected lead is symmetric ("0,5" vs "-0,5"), that means that actually 50% winrate is what we expect for both. Which is what we get!

So what we see is a variability in score estimation by about 0.1 or 0.2, and several tenths of a percent in winrate, but KataGo clearly understands the parity and is overall pretty consistent. And it's considering the different positions with their rules and komis entirely independently, without "knowing" that it's going to be compared against itself for consistency.

Seki variation



For the score/lead estimates, this is obviously much worse. With the seki on the board, again we'd expect Chinese 7.5 komi to be 0.5 points better than Japanese due to parity, and a further 2 points better due to this seki for a total of 2.5 points. So if we take as a baseline of -0,1 for Japanese 6.5 komi, we'd naively expect "2,4" for Chinese 7.5 komi instead of "1,0".

However, if you look at the ownership map, KataGo guesses a substantial chance that seki collapses - somewhere from a 10% to a 20% chance (the net isn't precisely sure). This would either be due to a ko fight, or due to one of the bordering groups getting traded away later. Also, given the frequency with which bots force each other into difficult ko fights when the game is on the line, it's quite possible that black plays the point-losing threats in some of the games where the seki doesn't collapse, which would also eliminate the 2 points gap - the threats may be losing, but they're also decently large, making them useful in some kos.

If we blindly suppose that, say, 25% of the time of the time either the seki is eliminated or the gap is closed by point-losing threats, then we'd expect something more like "1,9" instead of "2,4" for Chinese 7.5 komi, and the remainder of the difference is KataGo's score judgment being off by a bit. I'd guess the Japanese rules case is responsible for part of the error in score judgment (i.e. properly not counting the seki points). But if we simply assume the Japanese result is the true baseline, then we'd expect "1,9", "0,9" and "-0,1" as the three results, and actual observed by YakaGo was "1,0", "1,2", and "-0,9". So consistency errors less than 1 point, but close to it.

The winrate values actually look reasonable to me - they're probably overall biased a little in a way that's hard to know, but relatively speaking they seem better than the lead estimates here. The total winrate drop from 7.5 komi to 5.5 komi is about 14%, and the total drop in the earlier corner variation case was also 14%, so that matches well. And KataGo seems to understand that the parity has changed! (this is really cool, I wasn't sure KataGo would get this right). Due to the odd dame in the seki, now it's 7.5 and 6.5 komi that are equivalent, not 6.5 and 5.5, so this time the large winrate drop should happen from 6.5 -> 5.5, instead of 7.5 -> 6.5, and that's what it shows.

There actually still is a small winrate drop from 7.5 -> 6.5, but I think this is also reasonable! Because whereas odd-dame seki are uncommon, once one does occur, as we discussed earlier the chance that it collapses is not trivial, so some of the time the game will in fact end up back on the original parity where 7.5 and 6.5 are different instead of the same.

Okay, that's my analysis.

Wow! You may write up a short journal article just with this one shape. I am also surprised to see that KataGo really "understands" what's going on with this shape.

I almost gave up explaining these things to some Korean Go AI makers. They simply declined to listen to, thinking that it will make a negligible difference in increasing their odds against other AIs. (which is true, to be fair, but a sad attitude, to be honest.) At least KataGo seems to care.

_________________
Jaeup Kim
Professor in Physics, Ulsan National Institute of Science and Technology, Korea
Author of the Book "Understanding the Rules of Baduk", available at https://home.unist.ac.kr/professor/jukim/bbs/board.php?bo_table=notice&wr_id=5

I forgot a very simple experiment that one could make with this position.

Simply, I let white capture the single stone in the corner, to help clarify the final position under japanese rules. Black passes in the meantime.

When I do this, under japanese rules, white loses 1 point in katago's evaluation (along with a drop in winrate). Which suggests to me that that katago thought white had more points than is the case.

This now also shows the more clear 2 pt. difference between area and territory counting.

There are of course the details of the seki possibly being broken, in which case the two moves could indeed be a straight up loss for white.
The possible point losing ko threats are gone as well, but if black intended to use them, it should not be a benefit that he doesn't even have the chance.

I.e. for Lightvector's analysis, the assumption that japanese rules was the true baseline, might not be right.

Hi,
I can't see the position in the first message.

I see the message "There was a problem retrieving the game data." in the diagram.