A note on percentages

[Elom0]

I love intelligence shows. I remember an activity from the website of the Dara O'Brian's Scool of Hard Sums TV show. One question asked how much the water content in some jelly had been reduced if the percentage of mass had increased from 98% to 99%. That that I didn't figure it out then shows my dumbness, haha




































The answer is of course that the amount of water has reduced by 50%. Per cent quite literally means per 100 which quite literally means for every hundred--and in any case the expedient question is for every hundred of what? This is ignored by the silly way in which we use percentages. %=1/100 so whenever you say percentage the correct usage is to follow it up with 'of blah', and it grates on me when people leave it out.

The problem with the way people use AI percentages is that it's always the percentage of combined winrate of both sides, which is useless for both sides. What you actually want is the winrate of each side proportional to each other. If you are at 1% you could never make a move that loses more than that no matter what, however bad the move is. Yet in reality, a move that goes from 1% to 0.5% is just as bad as a move that goes from 50% to 33.6734694%, since in both cases the proportion of your winrate to your opponent's winrate nearly doubles.

Percentage derived from change in winrate proportional to combined winrate of all players becomes less the more biased to one side a game gets so in those situations points estimates are useful for judging how good or bad a move was. Percentage derived from the player with highest winrate it's not that different, but percentage derived from the player with lower winrate it's much better since that value will always be close to the proportional winrate between all players, however there's still a transfer of percentage that makes it slightly off. The winrate proportional to the other players' winrate or winrates is best, but this is best displayed not as a percentage but as a multiple, so in addition to the standard percentage derived from winrate proportional to combined winrate, a number that shows the multiple of how many times bigger the winrate of the player with the highest winrate's winrate is compared to the player with the lowest winrate should be shown, and this is the number that should be used to judge the quality of moves, of course . . .

[dfan]

Yet in reality, a move that goes from 1% to 0.5% is just as bad as a move that goes from 50% to 33.6734694%, since in both cases the proportion of your winrate to your opponent's winrate nearly doubles.

If we play games with $100 at stake, a move that goes from 1% to 0.5% loses (on average) 50 cents, while a move that goes from 50% to 33.67% loses (on average) $16.33.

A similar argument holds if rating points rather than dollars are at stake.

[Elom0]

Yet in reality, a move that goes from 1% to 0.5% is just as bad as a move that goes from 50% to 33.6734694%, since in both cases the proportion of your winrate to your opponent's winrate nearly doubles.

If we play games with $100 at stake, a move that goes from 1% to 0.5% loses (on average) 50 cents, while a move that goes from 50% to 33.67% loses (on average) $16.33.

A similar argument holds if rating points rather than dollars are at stake.

Yes, but I'm referring to judging the quality of a move rather than the from total winrate value of a move. In a situation where there are many moves on the board that can lose 16+1/3 %, where as.

Let's say there is a board position in which a move loses 16+1/3 %. Then let's say you give the other side enough komi points so that your winrate is so low that that same move now just loses 0.5%. It is exactly the same The only possible argument is that if the move was a slow or safe move, then when you fall further behind in winrate then the move itself is objectively a worse move for a non-perfect player, however if the move was a risky move then it just helps the argument, as the move has now objectively increased in quality for a non-human to play even though it's from total winrate value is less. Looking at a different kind of value, we shouldn't study late endgame much at all since the 2 and 1 and fractional point values are way less than the values of moves in earlier phases of the game! Move quality is how small a loss in value a move is compared to the correct move or moves which lose no value in from total winrate or points

[RobertJasiek]

May we even compare percentages for the positions before and after a particular move? Why? I think we may not because a) percentages rely on different sets of playouts and b) empirical scores are also taken into account.

[jlt]

If your winrate is 1%, then a "perfect" move that maximizes your score may decrease your winrate to 0.5%: since you are so much behind, you need to complicate the game, so you may accept to play suboptimal moves (in terms of score) if this increases the likelihood that your opponent makes a blunder.

[Knotwilg]

This is why we talk about percentage and percentage points.

If your winning percentage is 99% and then 98% your chances have been reduced by 1/99 which is 1,0101 ... % while your opponent's chances have raised from 1% to 2% which is an increase of 1/1 which is 100%. It's more intuitive to say one has decreased by 1% and the other has increase by 1% but technically those are percentage points.

BTW, I don't know how the portion of water has changed if the mass of the jelly in the jar has gone from 98% to 99% because I don't know the relative density of jelly and water. I'm splitting hairs of course, what is meant is volume I assume.

[Yakago]

"judging the quality of a move" based solely on percentages given by an AI program seems dubious at best..

[Elom0]

This is why we talk about percentage and percentage points.

If your winning percentage is 99% and then 98% your chances have been reduced by 1/99 which is 1,0101 ... % while your opponent's chances have raised from 1% to 2% which is an increase of 1/1 which is 100%. It's more intuitive to say one has decreased by 1% and the other has increase by 1% but technically those are percentage points.

I'm not quite sure what you're saying . . . Since in both cases, those are percentages of games projected to win over games played, and my entire point is we should include percentages of games won over games the opponent is projected to win!

BTW, I don't know how the portion of water has changed if the mass of the jelly in the jar has gone from 98% to 99% because I don't know the relative density of jelly and water. I'm splitting hairs of course, what is meant is volume I assume.

probably

"judging the quality of a move" based solely on percentages given by an AI program seems dubious at best..

Huh?

If your winrate is 1%, then a "perfect" move that maximizes your score may decrease your winrate to 0.5%: since you are so much behind, you need to complicate the game, so you may accept to play suboptimal moves (in terms of score) if this increases the likelihood that your opponent makes a blunder.

This is truer for AI neural nets than humans. Let's remember that alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win. Therefore moves AI consider as losing points but or more risky and complicated would have a higher winrate. The only reason why the move you play would have a lower winrate is if at your skill level it would make the game fairly complicated but to AI it's still simple so it loses points for no benfit.

[John Fairbairn]

alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win.

Since I am not a fan of the AlphaGo/DeepMind invented chewing gum, lobotomy surgery, Star Wars and the penny post school of thought that seems so popular now, I'd like to ask about this.

Was playing for a half-point win not a characteristic first of the Monte Carlo method (Remy somebody?) and was it not used in programs before (even on kgs)? I have never played go often enough in recent decades to have personal experience, but this claim does set off a tinkle in my head.

I don't wish to deny the AlphaGo achievement, but human progress is more often than not based on building on the shoulders of giants.

[gennan]

Who (other than pros) still looks at AI evaluation by winrate since KataGo started offering evaluation by score, more than 4 years ago?

If my winrate drops from 56% to 43% by a move I made, this is not very useful information for me, other than telling me it was probably a mistake.
However, if KataGo says this move changed my 2.5 point lead to a 2.5 point deficit, it gives me a much better feel for how big of a mistake that move was.

Also, when I see the score graph of my game staying within a 5 point band around even during the whole game, I understand that overall this was probably a decent game for players around my level.
At the same time, a winrate graph of the same game could still show huge swings (possibly going below 10% and above 90%), because (especially later in the game) a 5 point lead or deficit is pretty decisive to a strong AI (it's assuming an opponent of its own level).

[lightvector]

alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win.

Since I am not a fan of the AlphaGo/DeepMind invented chewing gum, lobotomy surgery, Star Wars and the penny post school of thought that seems so popular now, I'd like to ask about this.

Was playing for a half-point win not a characteristic first of the Monte Carlo method (Remy somebody?) and was it not used in programs before (even on kgs)? I have never played go often enough in recent decades to have personal experience, but this claim does set off a tinkle in my head.

I don't wish to deny the AlphaGo achievement, but human progress is more often than not based on building on the shoulders of giants.

Yes, as far as I'm aware this is was a decently common feature of MCTS bots ever since MCTS was discovered to be effective in Go after 2007. I also wouldn't consider it an "achievement", just an unintentional and often actively unhelpful-to-human-users feature of how bots tend to behave. If you don't tell them to care about score, then they don't care about score. If anything, older MCTS bots found it was a minor weakness - you could sometimes improve the strength (very slightly) by adding a slight bonus for improving score, not too large, but more than zero, encouraging bots to prefer to increase the margin a little to give a safety buffer against things they couldn't see if it didn't expose them to more risk.

[jlt]

The only reason why the move you play would have a lower winrate is if at your skill level it would make the game fairly complicated but to AI it's still simple so it loses points for no benfit.

Some positions are still complicated for bots, i.e. require hundreds of thousands of playouts.

[Elom0]

alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win.

Since I am not a fan of the AlphaGo/DeepMind invented chewing gum, lobotomy surgery, Star Wars and the penny post school of thought that seems so popular now, I'd like to ask about this.

Was playing for a half-point win not a characteristic first of the Monte Carlo method (Remy somebody?) and was it not used in programs before (even on kgs)? I have never played go often enough in recent decades to have personal experience, but this claim does set off a tinkle in my head.

I don't wish to deny the AlphaGo achievement, but human progress is more often than not based on building on the shoulders of giants.

Thank you for the correction! I am really not good at the technological fluffery, quite especially the game-playing bots. I've always much prefered the abstract conceptual fields of life than the practical.

The only reason why the move you play would have a lower winrate is if at your skill level it would make the game fairly complicated but to AI it's still simple so it loses points for no benfit.

Some positions are still complicated for bots, i.e. require hundreds of thousands of playouts.

Yes! And a bot may think a move that creates such a position which loses points also increase the winrate by making the game complicated and unpredictable even by bot standards

[RobertJasiek]

Who (other than pros) still looks at AI evaluation by winrate since KataGo started offering evaluation by score

I look at both AI percentages and AI scores as they do not correlate proportionally but, I suppose, KataGo considers both. By doing so, I learn well from this empirical information. However, it is often good to wait for the order of candidate moves to stabilise. Below 100k per top candidate move during the opening, I trust little. If necessary, I wait for (many) more visits for value stabilisation.

Some positions are still complicated for bots, i.e. require hundreds of thousands of playouts.

Quite a few positions do! In fact, I have seen positions requiring 5 ~ 50 million visits per top candidate move!

On one such occasion, the early "correct" candidate after ca. 200k per top candidate move turned out to be a mistake after some 10 million visits per top candidate move. With my human theory knowledge, I expected it to be a mistake so let the AI run for two or three hours.

On another occasion, the correct move had bad AI continuations (my judgement; AI claimed a close game but it was a fight with one player having a huge moyo advantage and the opponent already a weak adjacent group) until AI justified it by aforementioned many visits to find good early contination moves and avoid the bad fight.

[Yakago]

If you say that a position is complicated for an AI based on whether or not the 'best-move candidate' can change given a few million extra playouts, then you probably include a rather large amount of all positions.

For example, if you're at move 50-100, then the complexity of your position includes all variations until the end of the game.

I would cut the AI a bit of slack and say that it is not a complicated position for the AI, if its "immediate" candidate plays reasonably well into the position and doesn't encounter a lot of problems say in the next 20 moves of the game tree.

At least this would differentiate it from situations where there is a misjudgement of a semeai, or a missed tesuji or similar.

But sure, it's also an argument of scale, and what is considered 'complicated' can be a subjective thing..