Board evaluations unbiased by komi

[hakuseki]

I would like to know the best way to evaluate a position using a computer. For example, this position:

$$Wc White to play
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------

Click Here To Show Diagram Code

[go]$$Wc White to play
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

When I set up this position in KaTrain with Japanese rules, I get different estimates depending on the komi, as follows:

Code: Select all

Komi    Value    Value+Komi
0       B+45.7   B+45.7
46      B+0.9    B+46.9
47      B+0.2    B+47.2
47.5    W+0.2    B+47.3

I am using this "Value+Komi" number as an estimate of the (mathematical) value of the position. I believe this "Value+Komi" estimate is most accurate when "Value" is closest to zero, although this is just a guess. Still my best estimate may be something like 47.2 or 47.3.

I have two questions. Is there a better way to do this? And is there a more convenient way to do this? Fiddling with the komi values and waiting for KataGo to re-analyze the SGF is not super convenient. I'd like it if there were a way to get the estimate in one shot. Perhaps something to do with the parameters winLossUtilityFactor, staticScoreUtilityFactor, dynamicScoreUtilityFactor, dynamicScoreCenterZeroWeight, and dynamicScoreCenterScale in KataGo's analysis config?

Any tips would be appreciated.

[Yakago]

Another thing you can try is to find the komi which makes the win% evaluation closest to 0.5

The argument would be that the main training target is to win the game, and therefore one would expect this number to be the most accurate.
But the score prediction has also been a target for katago, so take the above with a grain of salt
And this is of course no help when it comes to convenience.

[RobertJasiek]

I would like to know the best way to evaluate a position using a computer.

Do not expect a single best way.

First, understand that typical computer evaluations are a) computer winning probabilities or b) final score predictions but not human-like current positional judgements. Second, understand the systematic weaknesses of computer evaluations. In particular, computer evaluations of the same type tend to be relative numbers for the AI's own application but not absolute numbers for human application. If we human players want to use such numbers nevertheless, our interpretation must be cautious and defensive.

To interpret computer evaluations, a) compare relative to other computer evaluations of the same type by the same program, the same program version and preferably the same position, or b) compare to human-like current positional judgements of territory, influence, statuses, fights etc., whose territory count is not the AI's final score prediction.

[hakuseki]

If we human players want to use such numbers nevertheless, our interpretation must be cautious and defensive.

I have a somewhat different philosophy. I would rather accept information from AI uncritically and apply it as broadly as I can imagine. The risk is that I will make a mistake in a game. But I think this risk is less than the risk of going on with my unguided 2d amateur intuition. And when I do make a mistake, then I can go back and re-evaluate whatever lesson I may have misapplied.

[lightvector]

Yes, estimates are most likely to be the best when the winrate is as close to 50% as possible. There isn't any substitute for doing multiple queries to hone in the komi that makes the game the most fair, so if you want it in one shot from a GUI from a user perspective then one way or another somewhere it has to be implemented as a bunch of separate queries.

[Kirby]

The risk is that I will make a mistake in a game.

There's also risk in thinking less. It's easy to look at an AI evaluation and think, "Oh, AI says increase of 10% for white, so it's gotta be good". But that's more like imitating AI than coming up with your own way of evaluating.

AI can probably be good for inspiration or new ideas. But it is easy to follow what it's saying without thinking. That's a risk we need to be careful of.