Temperature

xela · **#21**

RobertJasiek wrote:

Definition of global temperature: If you want to do it CGT style, ko thermography must be taken into account and this is far beyond of what I have completely understood of CGT. So, no, I cannot provide an even more thorough definition. As Francesco Criado points out, ko thermography is incomplete. So before creating somthing better, one should consider first completing CGT definitions... Will my ko definition be needed?

Yes and no. In order to understand the concept of temperature and apply it in practical situations, thermography is very useful. But the definition of "temperature" in CGT is already well established and does not rely on thermography at all. Unfortunately, for most realistic full board positions in go, we have no practical way to calculate the exact temperature. We can only use approximations and heuristics.

By the way, who is Francesco Criado? A google search just sends me back to this site, or else gives false positives for thermal imaging in other sciences.

RobertJasiek · **#22**

xela wrote:

who is Francesco Criado?

Typo for Francisco Criado, of course. Sorry.

User name Criado here.

Coauthor of 6 proofs of the 149 theorems in [22], in particular, main author of the proof of non-existence of local double sente.

Inventor of a few important counter-examples in [22].

Proofreader of the first half of [22].

Assistant mathematician at Technische Universität Berlin.

Gérard TAILLE · **#23**

kvasir wrote:

I'll try to respond to the OP.

I think this quote quite sufficient to understand temperature.

Mathematical Go: Chilling Gets the Last Point wrote:

[...]it is sufficient to think of temperature as a numeric estimate on the value of a move. The units are half the gote-value of a move in Japanese Go literature.

Later the idea, which I doubt is true, was spread around that Go players talked a lot about temperature. I think that is only true when Go players talk about theories such as those presented in the quoted monograph.

Like many concepts it has various realizations, depending on the intention.

For example

Let G be a game G := { L | R }, L and R also games.
Then define an operator, that we call cooling,

cool(G, t) = { cool(L, t) - t | cool(R, t) + t }

unless for some T < t, { cool(L, t) - t | cool(R, t) + t } is a number x (i.e. L - T <= R + T), then

cool(G, t) = x

Now G is said to have temperature T, mean(G) x and to freeze to cool(G, T).

Many other definition that allows for the following properties could be called temperature, even if they are not exactly the same, and I think you could be justified to call something temperature even if these properties are only usually correct. If one wished, then could be very precise about what is meant in each case.

Linearity: cool(G, t) + cool(H, t) = cool(G + H, t)
Order preserving: G >= H implies cool(G, t) >= cool(H, t)
mean(G + H) = mean(G) + mean(H)
temperature(G + H) <= max(temperature(G), temperature(H))

I doubt that these hold if we define

G := { f(L) | f(R) }

where f(x) is katago's score evaluation function, L and R as before. The most obvious violation is that f(x) isn't exact and will violate equalities and inequalities for that simple reason. Another problem is that while statements like G + H may work in form you usually can't actually add two 19x19 game together on a 19x19 board.

But it sure is similar in many ways, especially in form, to be useful.

I like that it is similar on form, I think that can be useful sometimes. If I were to suggest a less problematic definition that doesn't need to be similar in form then that would probably be

mean(G) := (f(L) + f(R)) / 2
temperature(G) := (f(L) - f(R)) / 2

I understand the first part of your post but I have difficulties with the second part when you use katago for the f function.

When you write
"Let G be a game G := { L | R }, L and R also games."
that means that the start of the game defined above is a POSITION and not a SITUATION for which we know who is to play first. The issue is the following : because the f function (katago) apply only on SITUATION and not on POSITION what does mean f(L) or f(R)?

For that reason I use in my defintion the PASS move and defined the value of a move by
M = magic(P + M) - magic(P + pass)
and not
M = (magic(P + M) - magic(P + pass)) / 2

IOW when you apply your proposal to the empty board what temperature are you reaching? 7 or 14?

kvasir · **#24**

Gérard TAILLE wrote:

I understand the first part of your post but I have difficulties with the second part when you use katago for the f function.

When you write
"Let G be a game G := { L | R }, L and R also games."
that means that the start of the game defined above is a POSITION and not a SITUATION for which we know who is to play first. The issue is the following : because the f function (katago) apply only on SITUATION and not on POSITION what does mean f(L) or f(R)?

You seem to be asking me to define it completely. Maybe, to be clearer I should have written

G = { { f(x, "White") : x in L } | { f(x, "Black") : x in R } }
meaning that who is to play is specified for the function f, or
G = { f(sup(L), "White") | f(inf(R), "Black") }
where sup(X) and inf(X) are the supremum and infimum of a set X,
or even simply
G = { f(L, "White") | R3 = f(R, "Black") }
which means f operates on the set of possible games, and I think that can be usefully simplified in the way wrote originally as
G = { f(L) | f(R) }
since we do know who is to move and not necessarily need to reflect on that everywhere.

Or I could just have stated instead of "f(x) is katago's score evaluation function" something vague like "f(...) is a suitable score evaluation function". So I think there are various ways to interpret what I wrote, which was the intention

Gérard TAILLE wrote:

For that reason I use in my defintion the PASS move and defined the value of a move by
M = magic(P + M) - magic(P + pass)
and not
M = (magic(P + M) - magic(P + pass)) / 2

I gather what you mean by P + x is that the move x is played in situation P (one where we know who is to play). Then I'd assume that the left hand side M is not the same as the right hand side M.

But the main difference to what I wrote is that I was trying to be more general and express that this doesn't in general behave like temperature and it doesn't really make use the notation at all (which I should have mentioned), so you just write (using the preferred interpretation of f(x)) that

mean(G) := (f(L) + f(R)) / 2
temperature(G) := (f(L) - f(R)) / 2

Gérard TAILLE wrote:

IOW when you apply your proposal to the empty board what temperature are you reaching? 7 or 14?

The convention for temperature, which I quoted, is to divide by two. That is the start position G ~= 0 and black can makes a move to go to L ~= komi or white could make a move to go to R ~= -komi. I think the desire is that if G = { L | R }, then mean(G) + temperature(G) = mean(L) and mean(G) - temperature(G) = mean(R), rather than mean(G) + temperature(G) / 2 = mean(L) and mean(G) - temperature(G) / 2 = mean(R).

This convention means that when you add and subtract temperatures from mean(A), depending on if you take a left or right edge in a path AB in the game tree, and get mean(B). Right?

Gérard TAILLE · **#25**

kvasir wrote:

Gérard TAILLE wrote:

I understand the first part of your post but I have difficulties with the second part when you use katago for the f function.

When you write
"Let G be a game G := { L | R }, L and R also games."
that means that the start of the game defined above is a POSITION and not a SITUATION for which we know who is to play first. The issue is the following : because the f function (katago) apply only on SITUATION and not on POSITION what does mean f(L) or f(R)?

You seem to be asking me to define it completely. Maybe, to be clearer I should have written

G = { { f(x, "White") : x in L } | { f(x, "Black") : x in R } }
meaning that who is to play is specified for the function f, or
G = { f(sup(L), "White") | f(inf(R), "Black") }
where sup(X) and inf(X) are the supremum and infimum of a set X,
or even simply
G = { f(L, "White") | R3 = f(R, "Black") }
which means f operates on the set of possible games, and I think that can be usefully simplified in the way wrote originally as
G = { f(L) | f(R) }
since we do know who is to move and not necessarily need to reflect on that everywhere.

Or I could just have stated instead of "f(x) is katago's score evaluation function" something vague like "f(...) is a suitable score evaluation function". So I think there are various ways to interpret what I wrote, which was the intention

Gérard TAILLE wrote:

For that reason I use in my defintion the PASS move and defined the value of a move by
M = magic(P + M) - magic(P + pass)
and not
M = (magic(P + M) - magic(P + pass)) / 2

I gather what you mean by P + x is that the move x is played in situation P (one where we know who is to play). Then I'd assume that the left hand side M is not the same as the right hand side M.

But the main difference to what I wrote is that I was trying to be more general and express that this doesn't in general behave like temperature and it doesn't really make use the notation at all (which I should have mentioned), so you just write (using the preferred interpretation of f(x)) that

mean(G) := (f(L) + f(R)) / 2
temperature(G) := (f(L) - f(R)) / 2

Gérard TAILLE wrote:

IOW when you apply your proposal to the empty board what temperature are you reaching? 7 or 14?

The convention for temperature, which I quoted, is to divide by two. That is the start position G ~= 0 and black can makes a move to go to L ~= komi or white could make a move to go to R ~= -komi. I think the desire is that if G = { L | R }, then mean(G) + temperature(G) = mean(L) and mean(G) - temperature(G) = mean(R), rather than mean(G) + temperature(G) / 2 = mean(L) and mean(G) - temperature(G) / 2 = mean(R).

This convention means that when you add and subtract temperatures from mean(A), depending on if you take a left or right edge in a path AB in the game tree, and get mean(B). Right?

Let me propose an example as a support for the discussion.
Let's suppose the board is made of 14 independant gote areas with the miai values 14, 13, 12, ... 2, 1 or if you prefer with swing values 28, 26, 24, ... 4, 2.

What is the final score if it is black to play?
f(L) = 14 -13 + 12 - 11 ... + 2 - 1 = +7
if is white to play then
f(R) = -14 + 13 - 12 + 11 ... -2 + 1 = -7
Note that if you choose komi = +7 this game G is even for the players.

I agree with you that mean(G) = (f(L) + f(R)) / 2 = 0
but I have still difficulties to say that this game with miai values 14, 13, 12, ... 2, 1 or swing values 28, 26, 24, ... 4, 2 is at temperature +7. I understand you prefer to divide the temperature by 2 but for me it is far more natural to say that temperature = 14. For the time being I do not see really your goal but maybe I have to think about it a litttle more.

RobertJasiek · **#26**

Gérard TAILLE wrote:

Let me propose an example as a support for the discussion.
Let's suppose the board is made of 14 independant gote areas with the miai values 14, 13, 12, ... 2, 1 [...]

What is the final score if it is black to play?
f(L) = 14 -13 + 12 - 11 ... + 2 - 1 = +7
if is white to play then
f(R) = -14 + 13 - 12 + 11 ... -2 + 1 = -7

Let me propose another example showing the limits of yours:

Let's suppose 27 local gotes without follow-ups with the miai values

14, 13, 13, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1.

The final score with Black to play is f(L) = 14.
The final score with White to play is f(R) = -14.

Tedomari matters.

Gérard TAILLE · **#27**

RobertJasiek wrote:

Gérard TAILLE wrote:

Let me propose an example as a support for the discussion.
Let's suppose the board is made of 14 independant gote areas with the miai values 14, 13, 12, ... 2, 1 [...]

What is the final score if it is black to play?
f(L) = 14 -13 + 12 - 11 ... + 2 - 1 = +7
if is white to play then
f(R) = -14 + 13 - 12 + 11 ... -2 + 1 = -7

Let me propose another example showing the limits of yours:

Let's suppose 27 local gotes without follow-ups with the miai values

14, 13, 13, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1.

The final score with Black to play is f(L) = 14.
The final score with White to play is f(R) = -14.

Tedomari matters.

Your point is valid Robert and it conforts my approach.
In your example we have f(L)- f(R) = 28

But know you can also take the counter example with 28 gotes without follow-ups with the miai values
14, 14, 13, 13, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1.
The final score with Black to play is f(L) = 0.
The final score with White to play is f(R) = 0
and f(L)- f(R) = 0
Miai matters also

Taking gotes without follow-ups with the miai values 14, 13, 12, ... 2, 1 acts as a kind of average of all games. Isn't it the approach commonly used in endgame theory?

Knotwilg · **#28**

I applied the earlier reasoning on another joseki. This time I put 4-4 stones in every other corner, reducing the ambient temperature (I think). Again, the number of playouts is not very large, only a couple of thousand. Mostly I stopped the KG analysis when the score was stable for more than 10 seconds.

Repeating the idea: a move is sente if the local temperature is higher than ambient temperature. Local/ambient temperature means the value of a move/locally elsewhere, computed as (half) the difference between Black or White playing the best move locally/elsewhere. The measure is KataGo's score estimator.

Since I haven't done sufficient playouts, I'm taking 1 point as an error margin. If the difference between Local and Ambient is smaller than that, I'm calling the position ambiguous.

Click Here To Show Diagram Code: [go]$$B Joseki $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . 6 . O . . . | $$ | . . . O . . . . . , . . . . c b 4 . . | $$ | . . . . . . . . . . . . d . . 1 2 . . | $$ | . . . . . . . . . . . . . . . 5 3 . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . a . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . 7 . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . X . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

In this "ambiance"
:b1:

is sente (13.2 > 11.9)
:w2:

is sente (14.9 > 12.0)
:b3:

is sente (18.6 > 10.4)
:w4:

is sente (13.8 > 11.9)
:b5:

is ambiguous, might be sente (12.1 ~ 11.6)
:w6:

is ambiguous, might not be sente (11.7 ~ 12.5)
:b7:

is not sente (10.3 < 12.0)

Hence we can consider the sequence one whole joseki up until :b7:

included.
White's best local move after :b7:

is A, which we can consider a "follow-up" of the joseki.
Black's follow-up is not so clear: B and C are more like forcing moves. D is the candidate I selected for the evaluation.

As for the overall temperature of the game, it starts out at 13.2, heats up considerably at B3 (18.6), then cools down again to (slightly above) 12.

Comments welcome.

Gérard TAILLE · **#29**

Knotwilg wrote:

I applied the earlier reasoning on another joseki. This time I put 4-4 stones in every other corner, reducing the ambient temperature (I think). Again, the number of playouts is not very large, only a couple of thousand. Mostly I stopped the KG analysis when the score was stable for more than 10 seconds.

Repeating the idea: a move is sente if the local temperature is higher than ambient temperature. Local/ambient temperature means the value of a move/locally elsewhere, computed as (half) the difference between Black or White playing the best move locally/elsewhere. The measure is KataGo's score estimator.

Since I haven't done sufficient playouts, I'm taking 1 point as an error margin. If the difference between Local and Ambient is smaller than that, I'm calling the position ambiguous.

Click Here To Show Diagram Code: [go]$$B Joseki $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . 6 . O . . . | $$ | . . . O . . . . . , . . . . c b 4 . . | $$ | . . . . . . . . . . . . d . . 1 2 . . | $$ | . . . . . . . . . . . . . . . 5 3 . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . a . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . 7 . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . X . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

In this "ambiance"
:b1:

is sente (13.2 > 11.9)
:w2:

is sente (14.9 > 12.0)
:b3:

is sente (18.6 > 10.4)
:w4:

is sente (13.8 > 11.9)
:b5:

is ambiguous, might be sente (12.1 ~ 11.6)
:w6:

is ambiguous, might not be sente (11.7 ~ 12.5)
:b7:

is not sente (10.3 < 12.0)

Hence we can consider the sequence one whole joseki up until :b7:

included.
White's best local move after :b7:

is A, which we can consider a "follow-up" of the joseki.
Black's follow-up is not so clear: B and C are more like forcing moves. D is the candidate I selected for the evaluation.

As for the overall temperature of the game, it starts out at 13.2, heats up considerably at B3 (18.6), then cools down again to (slightly above) 12.

Comments welcome.

Interesting analysis Knotwilg.
I understand the main point of your post is to have some criteria for the start and the end of a joseki and I agree with you on this point.
I note also a secondary point in your post. In the fuseki the temperature between two joseki is more probably equal to 12 rather than equal to 14. Maybe someone can confirm this point with more playouts.

kvasir · **#30**

Gérard TAILLE wrote:

I note also a secondary point in your post. In the fuseki the temperature between two joseki is more probably equal to 12 rather than equal to 14. Maybe someone can confirm this point with more playouts.

I did an analysis of how KataGo evaluates handicap stones to komi. It is something I might share. I found that if h is the handicap then 13.5 * (h - 0.5) - 0.25 is a function for the komi that KataGo evaluates as fair after substantial playouts. This function was designed to give whole or half komi

A robust estimator found 13.33 and if we are only interested in small handicaps then 13.2 was the average for the first 3 handicap stones.

I actually wanted to make more analysis on it before mentioning it here. One question I have is if this komi actually give equal winning chances in games. I know it is not that far off for 9 stones, black wins some and white wins some.

Basically, I can confirm 13.2 in the early game with open corners.

Btw the range was [12, 15.5] for handicap stones. I have seen it down to about 10 in actual games during the opening, maybe even less.

==Edit
I meant to write 13.5 * (h - 0.5) - 0.25 when I wrote 13.5 * (h - 0.5) + 0.25, so I corrected this.

Knotwilg · **#31**

Click Here To Show Diagram Code: [go]$$B Avalanche $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . b . . . | $$ | . . . . . . . . . . . . . . 3 O c . . | $$ | . . . O . . . . . , . . . . . 1 2 . . | $$ | . . . . . . . . . . . . . . . X O . . | $$ | . . . . . . . . . . . . . . . a 4 . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . X . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

The avalanche is not so popular anymore, although it's only a fraction of a point (0.3) worse than the hane.

Obviously :b1:

is sente (19.2 > 7.9). So is :w2:

(15.3 > 12.1).
:b3:

is also sente but what surprised me is that is "even more" sente than :b1:

(19.8 > 7.5). Intuitively allowing Black to cut through the keima with a solidly connected triangle is worse than allowing atari to one stone.
:w4:

cools down the position but remains ambiguous: not only is the local temperature (12.9) only slightly bigger than the ambient (12.7), also KataGo wants to play elsewhere now, against the temperature argument.

Indeed, an analysis that includes passing doesn't necessarily agree with what's the best move. It might mean that my version of temperature used here is not well defined.

There are other moves available for :w4:

, most notably A which leads to the small avalanche. This variation keeps the area "hot" for a while and can even be considered to turn into a "whole board joseki"

It remains interesting that an increase of local temperature seems to always induce a decrease of the ambient temperature. In the words of the definition used: a high value of a local move decreases the value of the biggest move elsewhere. This is somewhat intuitive but not trivial. A priori one might expect that ambient temperature remains relatively stable and only local temperature goes up or down considerably. But that is not the case. Allowing an early atari in a corner makes the position in that corner lopsided, so that the relative value of the other corners decreases.

xela · **#32**

Knotwilg wrote:

I applied the earlier reasoning on another joseki. This time I put 4-4 stones in every other corner, reducing the ambient temperature (I think).

If you really want to reduce the ambient temperature, try 3-3 points? Although I'm not sure if minimising the ambient temperature is actually what you want...

Gérard TAILLE · **#33**

Knotwilg wrote:

There's SL's definition by Bill, closely related to CGT: https://senseis.xmp.net/?Temperature
I have added my own understanding: https://senseis.xmp.net/?Dieter%2FTemperature

I just have read your new version of temperature on sensei.

I completly agree with your defintion:

"The temperature (T) of a board position is the difference between playing a move and passing in that position."

But for the late endgame I cannot agree wtih you:

Late endgame
In the (late) endgame, we can treat local positions as isolated, since their outcome won't affect other positions. In that case, we consider the local temperature to be the temperature as above, assuming that the local position is alone on the board.

Let me try to explain. You cannot confused the temperature of a board position with the temperature of local position with a more a less "ideal" environment. These two situations are very different.
It will be simpler to understand with an exemple.
Let's take a game made of pure gotes area without follow-ups with the miai values 14, 13, 12, ... 2, 1 or swing values 28, 26, 24, ... 4, 2

What is the temperature of the board

S(P,B) = 14 - 13 + 12 ... + 2 - 1 = +7
S(P,W) = -14 + 13 - 12 ...- 2 + 1 = -7
T(P) = |S(P,B) - S(P,W)| = +7 + 7 = +14

Now what is temperature of the biggest local area I call G (miai 14 or swing value 28).
If you consider this area G is alone on the board then
S(G,B) = 14
S(G,W) = -14
T(G) = |S(G,B) - S(G,W)| = +14 + 14 = +28

As you see you cannot handled in the same way the all board and a local area.

Gérard TAILLE · **#34**

kvasir wrote:

Gérard TAILLE wrote:

I note also a secondary point in your post. In the fuseki the temperature between two joseki is more probably equal to 12 rather than equal to 14. Maybe someone can confirm this point with more playouts.

I did an analysis of how KataGo evaluates handicap stones to komi. It is something I might share. I found that if h is the handicap then 13.5 * (h - 0.5) - 0.25 is a function for the komi that KataGo evaluates as fair after substantial playouts. This function was designed to give whole or half komi

A robust estimator found 13.33 and if we are only interested in small handicaps then 13.2 was the average for the first 3 handicap stones.

I actually wanted to make more analysis on it before mentioning it here. One question I have is if this komi actually give equal winning chances in games. I know it is not that far off for 9 stones, black wins some and white wins some.

Basically, I can confirm 13.2 in the early game with open corners.

Btw the range was [12, 15.5] for handicap stones. I have seen it down to about 10 in actual games during the opening, maybe even less.

==Edit
I meant to write 13.5 * (h - 0.5) - 0.25 when I wrote 13.5 * (h - 0.5) + 0.25, so I corrected this.

You said that with open corner and according to katago, the temperature of the board is about 13.2. That means that with a komi = 6.5 black as a very small advantage.
With not a lot of playouts katago seems to give advantage to white in the initial position with komi = 6.5.
I would be interesting to know how many playouts is needed to find a black advantage in these conditions?
Robert said that millions of playouts are needed to begin to trust katago's results. is it the case to find the best komi according to katago?

RobertJasiek · **#35**

Gérard TAILLE wrote:

Robert said that millions of playouts are needed to begin to trust katago's results.

I have said that 100k per top candidate is about my lower bound of trust. Depending on circumstances, (many) millions can be necessary.

Anecdote: Just today, I have had 500k for each of the top three moves. A or B seemed correct while C seemed to be a mistake. These assessments were stable since 100k. Nevertheless, I smelled the fish and continued search. Suddenly, A and B became mistakes while C became correct. Stable afterwards. Ugh!

xela · **#36**

RobertJasiek wrote:

Anecdote: Just today, I have had 500k for each of the top three moves. A or B seemed correct while C seemed to be a mistake. These assessments were stable since 100k. Nevertheless, I smelled the fish and continued search. Suddenly, A and B became mistakes while C became correct. Stable afterwards. Ugh!

Interesting! Would you be willing to share the position, to see if others can replicate these results or offer any other insights? What features caused it to seem fishy?

There was a lot of discussion a few years ago about interrogating the AI: not just letting it run for a long time on a single position, but exploring variations, alternative candidate moves, followups, possible refutations and so on -- looking for supplementary information to better "understand" its choice.

RobertJasiek · **#37**

xela wrote:

Would you be willing to share the position

Not at this moment, as it would reveal too closely what I am working on. You can ask me again in a few months.

Quote:

What features caused it to seem fishy?

The position is a joseki branch at a moment when each of the three interesting candidates leads to complicated follow-ups with very different developments after either moving out immediately or first going for the local territory and eyespaces. Therefore, I suspected that not only I but also AI might have difficulties to assess the candidates with an only intermediate amount of thought.

The early values of the alleged mistakes were just beyond the threshold which I have chosen to identify mistakes. They were not far beyond the threshold. Accordingly, still I was sceptical. My preferred threshold is 2.0 for the sum of the differences of winning percentages and empirical scores compared to the current top candidate.

Quote:

There was a lot of discussion a few years ago about interrogating the AI: not just letting it run for a long time on a single position, but exploring variations, alternative candidate moves, followups, possible refutations and so on -- looking for supplementary information to better "understand" its choice.

Besides doing all of that, what have been the suggestions?

Gérard TAILLE · **#38**

RobertJasiek wrote:

Gérard TAILLE wrote:

Robert said that millions of playouts are needed to begin to trust katago's results.

I have said that 100k per top candidate is about my lower bound of trust. Depending on circumstances, (many) millions can be necessary.

Anecdote: Just today, I have had 500k for each of the top three moves. A or B seemed correct while C seemed to be a mistake. These assessments were stable since 100k. Nevertheless, I smelled the fish and continued search. Suddenly, A and B became mistakes while C became correct. Stable afterwards. Ugh!

Yes Robert it is not surprising. If the position needs a very deep analysis with typically a difficult semeai then katago can easily missed the best move for a long time.
I showed already some endgames that need a deep analysis. In such situation katago also is unable to find the best endgame sequence.

Temperature

Who is online