Verdict on AI verdicts on old pros

John Fairbairn · Post by **John Fairbairn** » Tue Aug 20, 2019 2:40 am

See my separate threads on AI verdicts on Jowa, Shusaku and Dosaku as compiled by Ohashi Hirofumi.

Ohashi has now added a very brief washing-up article on his mini series of using Golaxy to assess players of the past.

He admitted to great nervousness before the series began, expecting to demolish fans' respect for these legendary players. In fact it worked out the opposite way, and with the bonus that the bot actually seemed to confirm our traditional human perception of the strengths of each player.

But one traditional view that now has to be challenged is that the old pros were super strong in the endgame. Golaxy showed consistently that it was possible to make moves in the middle game and endgame that could impact on the game as a whole and which were therefore better than just making territory-settling boundary plays.

This not only calls into question the long favoured Japanese style of soba go, or compromise go or quid pro quo go. Ohashi believes it is in this area that Japanese players need to make changes in order to challenge Chinese and Korean players.

In fact I recall Wang Xi making this point in a long and detailed article in Weiqi Tiandi long before AI bots were even a gleam in the eye. He specifically trashed the concept of soba go and specifically said the Chinese and Koreans now went for the percentage play. Recent history seems to have justified that view. It seems that greed is good in go.

Of course playing the percentages only works if you know how to work out the percentages...

gowan · Post by **gowan** » Tue Aug 20, 2019 6:20 am

When I was first learning to play go I was, as probably many of us were, told that greedy play was bad. When weak players play greedily against a stronger player, the greedy player often loses. I was also told that go is a game of balance and if you try to keep your opponent from getting anything you will lose. The kind of greedy play often exhibited by weak amateurs is not the same as John Fairbairn cited above. John described the Korean and Chinese approach to go as playing the percentages. I believe that the win rates given by AI bots are not quite exactly the percentage chances of winning. I'm not sure what they are exactly, maybe the percentage of winning in the playouts? Could the percentage play John mentioned might be just playing the most efficient moves? I'm sure that's what pros in general try to do.

Bill Spight · Post by **Bill Spight** » Tue Aug 20, 2019 7:58 am

John Fairbairn wrote:But one traditional view that now has to be challenged is that the old pros were super strong in the endgame. Golaxy showed consistently that it was possible to make moves in the middle game and endgame that could impact on the game as a whole and which were therefore better than just making territory-settling boundary plays.

I thought that the idea that top players played nearly perfect endgames was confined to the 19th century, when games could take forever.

It may well be that Golaxy, which has an idea of territory or area, plays a better endgame than other top bots, but the question of impacting the game as a whole is not very precise. And "just making territory-settling boundary plays" is an unfair characterization of endgame technique. Few of my endgame problems, even though all of the plays are "territory-settling boundary plays", and the problems are pretty easy, deserve that description, IMHO.

I wonder how Golaxy would do with this problem, my revision of one of Berlekamp and Wolfe's problems in Mathematical Go. No komi, but Golaxy can handle that, right?

Click Here To Show Diagram Code: [go]$$Wc White to play and win $$ --------------------------------------- $$ | . X . . . O . . . . . . . X . . . O . | $$ | . X . . X O O O O X X . X X O . . O . | $$ | X X X X X O . X O . X . O X O O O . . | $$ | . O O O O O O X X , . X . X X X O O . | $$ | . O . O . . O O X X . . X . X X X X X | $$ | . O O . O . . . . X . . X O X . . . . | $$ | . . . . O X O O O X X . X . . . X X . | $$ | O O O . O . . O . X . X X X X X X O O | $$ | X X O O O O O . O X . . X . O O O . . | $$ | . X X X X . O O O X X X O X O , . O . | $$ | . . X . X . X X O O O O O O O O O . . | $$ | . . . X X . . . X X O . . . . . O O O | $$ | . X X . . X X X . X O . . O O . . . . | $$ | . X . X X O X . X X O O . O . O O O . | $$ | . X X X . . X X X O O . O O O O X X X | $$ | O O O O . X X . X X O . . O . , . X . | $$ | . . . O . X . . . . . O . . O X X X X | $$ | . O . O O X X X X X X O O . O O X . . | $$ | . . O . . O . . . . X X O . . O . . . | $$ ---------------------------------------[/go]

You need to read to depth 29 to reach an "obviously" won position, depth 69 to where only dame are left, but the first few plays are obvious and after that there are only a few plays worth considering at each turn. And all of the plays are just territory-settling boundary plays. Piece of cake for a superhuman bot, no?

John Fairbairn · Post by **John Fairbairn** » Tue Aug 20, 2019 10:02 am

gowan: the kind of percentages I'm talking about as characteristic of soba go are (as far as I know) nothing to do with win rates.

In golf each hole has a par score, which represents something like an estimate of how many strokes the generality of golf pros would, on average, take to complete that hole.

Soba go is like playing par golf.

But pros who want to actually win tournaments know they have to go beyond par and make birdies. That means they calculate the odds, for themselves, of overhitting a drive or taking a risky line to the path, so that they can get a birdie. They know there is a risk involved. They might end up with a bogie. But the best players can feel confident in their own play and convince themselves that the chances of making a birdie - the percentages - are better than making a bogie. They will further refine that gamble on the basis of the situation in the tournament and how their opponents are doing.

In go, Chinese and Korean players, full of confidence, are making the same sort of attempts to get birdies. Too many Japanese players are stuck playing for par. These percentages (not precise figures, of course, so maybe not the ideal term) apply to local situations, not the overall game. A game can thus seen as a series of "holes." A series of birdie opportunities. If you get more birdies than pars you can win easily against a typical Japanese player.

I doubt that AI can tell us much about all that. It's to do with humans' constant efforts to strike a balance between safety and risk taking. You can improve your chances of taking risks successfully by improving technique, fitness, experience, etc. But it still boil down to a gamble at the OK Corral.

Bill: you've changed the topic, really. My fault for using a sloppy term like territory-settling. Maybe I should have said aji-fixing or whatever, but I was trying to avoid the word 'thick' for reasons you are aware of more than most. And also, I did say middle game and endgame. We are talking about live games (in every sense), not puzzles.

Knotwilg · Post by **Knotwilg** » Tue Aug 20, 2019 10:23 am

John Fairbairn wrote: (...)
This not only calls into question the long favoured Japanese style of soba go, or compromise go or quid pro quo go. Ohashi believes it is in this area that Japanese players need to make changes in order to challenge Chinese and Korean players.

In fact I recall Wang Xi making this point in a long and detailed article in Weiqi Tiandi long before AI bots were even a gleam in the eye. He specifically trashed the concept of soba go and specifically said the Chinese and Koreans now went for the percentage play. Recent history seems to have justified that view. It seems that greed is good in go.

(...)

I went looking up https://senseis.xmp.net/?Souba to understand the point made here.

If a player makes a mistake leading to a bad position the pro may correct the mistake and show a natural resulting position and call this souba. There is an implication that this is the best both players can do in the situation

"Soba go" then means, "to play the best sequence in each position" and the traditional assumption is that this is also the best thing to do in the situation. If we undermine that assumption - and call this "playing the percentages" - this means that in many situations it is preferable to choose a more adventurous line than the "best sequence in that position" which may have implications beyond that position (more aji, a source of ko threats, a ko itself, ...) but overall that extra risk is offset by the extra gain.

Is this what you mean with "greedy go pays off"?

Bill Spight · Post by **Bill Spight** » Tue Aug 20, 2019 10:38 am

John Fairbairn wrote:Bill: you've changed the topic, really.

Well, you did start off talking about the presumed excellence of pros of yore in the endgame.

BTW, I did check out Elf's commentary on Shusai's famous 8 hour tank to read out a 2 pt. win in the endgame. Elf suggested other plays from time to time, no surprise there. Most of them, I think, did not alter the result, but a couple might have been improvements.

My fault for using a sloppy term like territory-settling. Maybe I should have said aji-fixing or whatever, but I was trying to avoid the word 'thick' for reasons you are aware of more than most.

Truth to say, the first thing I thought of when I read your post was Sakata's discussion of thick plays in the endgame. But I regarded them as plays that "impacted the game as a whole".

Ferran · Post by **Ferran** » Tue Aug 20, 2019 10:40 am

Knotwilg wrote:"Soba go" then means,[...]

I really thought it was the same kind of phrase as "spaghetti code"...

Take care

Bill Spight · Post by **Bill Spight** » Tue Aug 20, 2019 11:08 am

John Fairbairn wrote:gowan: the kind of percentages I'm talking about as characteristic of soba go are (as far as I know) nothing to do with win rates.

In golf each hole has a par score, which represents something like an estimate of how many strokes the generality of golf pros would, on average, take to complete that hole.

Soba go is like playing par golf.

But pros who want to actually win tournaments know they have to go beyond par and make birdies. That means they calculate the odds, for themselves, of overhitting a drive or taking a risky line to the path, so that they can get a birdie. They know there is a risk involved. They might end up with a bogie. But the best players can feel confident in their own play and convince themselves that the chances of making a birdie - the percentages - are better than making a bogie. They will further refine that gamble on the basis of the situation in the tournament and how their opponents are doing.

As the great, and very witty bridge author, Victor Mollo wrote, "Par is an unworthy goal."

Kirby · Post by **Kirby** » Tue Aug 20, 2019 11:24 am

Bill Spight wrote: As the great, and very witty bridge author, Victor Mollo wrote, "Par is an unworthy goal."

Just don't play subpar in your attempts to overcome par

Knotwilg · Post by **Knotwilg** » Tue Aug 20, 2019 12:15 pm

John Fairbairn wrote:I was trying to avoid the word 'thick' for reasons you are aware of more than most.

Maybe ... atsui?

No really, I love this post - so back to my question about souba vs greedy go.

Uberdude · Post by **Uberdude** » Tue Aug 20, 2019 12:41 pm

Knotwilg wrote:
John Fairbairn wrote: (...)
This not only calls into question the long favoured Japanese style of soba go, or compromise go or quid pro quo go. Ohashi believes it is in this area that Japanese players need to make changes in order to challenge Chinese and Korean players.

In fact I recall Wang Xi making this point in a long and detailed article in Weiqi Tiandi long before AI bots were even a gleam in the eye. He specifically trashed the concept of soba go and specifically said the Chinese and Koreans now went for the percentage play. Recent history seems to have justified that view. It seems that greed is good in go.
I went looking up https://senseis.xmp.net/?Souba to understand the point made here.

If a player makes a mistake leading to a bad position the pro may correct the mistake and show a natural resulting position and call this souba. There is an implication that this is the best both players can do in the situation
"Soba go" then means, "to play the best sequence in each position" and the traditional assumption is that this is also the best thing to do in the situation.

This is not my understanding of soba. I see it more as "good enough" or "reasonable" and a fairly standard and equitable exchange. Soba is never going to win move of the year. I think I read some article by John in GoGoD alluding it to accounting, lose a bit here, gain an equivalent bit there, so the end position is as good as the start position. So you maintain a balance and then if your opponent makes a mistake you take a lead, and then you keep playing soba to maintain the lead. You are not pushing the boundary of unreasonableness or overplay seeking to proactively take the lead which comes with the risk of getting punished if your move was indeed too much (which you probably don't even know, just have a feeling). The traditional Japanese style is the risk-averse soba, whereas someone like Lee Sedol is the Korean risk-taking trying to win, not just trying not to lose.

I also recall some quote from Hane Naoki about how he tries to play the 80% move (number not guaranteed, but it was a fair bit less than 100), where 80% doesn't refer to a win%, but 100% would be the most efficient and totally optimal move (so super-strong-bot win % remains unchanged at whatever it is between 0 and 100) but possibly really complicated and chances to backfire if you make the slightest mistake. Trying to find the boundary of the 100% move, and not trying to get even more, let's call it the 110% move if your opponent doesn't punish it but actually it is an overplay so it could end up as a 20% move, is very hard, so Hane is content to stay well away from it and play "good enough", whereas the Korean/Chinese (and Go Seigen) and now AI style is going for more efficiency and getting closer to that 100% line.

There was an interesting example of this on reddit recently, with someone asking for clarification about this position and explanation in Kageyama's "Lessons in the Fundamentals of Go" (taken from one of his games).

Click Here To Show Diagram Code: [go]$$B $$ +---------------------------------------+ $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X X X . 1 . . . . . . . . . O . . | $$ | . . O X O O . a . , . . . . O , X . . | $$ | . . O O X O . b . . . . . . . . . . . | $$ | . . O X X X X . . . . . . . . . . . . | $$ | . . O X . . . . . . . . . . . . . . . | $$ | . . . O . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . X . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . O . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ +---------------------------------------+[/go]

Kageyama writes:

... A stronger amateur would glare at the position and play black 'a', for a larger capture. A professional, however, would find the threat of White 'c' after Black 'a' disquieting, regardless of whether it works immediately or not. To him Black 1 would be the natural and proper move, the only move to make.
Black 1 or Black 'a'? Only an amateur would ask himself this question. A professional would simply dismiss the issue. Neither the intuitive school nor the profit school would give it a second thought. Here we can see another difference between amateur and professional.

Even before you ask an AI and it shreds Kageyama's argument, I find it rather dubious. First of all I now find the endless partition of go thinking and skill into ama vs pro a bit tiresome and inaccurate as many amateurs are stronger than pros (and in thinking not just oh this pro is old and can't read anymore but has a superior way of thinking to this strong at reading crude amateur 7d), but as the book was original called Ama vs Pro or whatever in Japanese it's understandable. I bet there would be some Japanese pros even back in 1978 who wouldn't like 1 and prefer the greedy but bad aji 'a', maybe Sakata?

Unsurprisngly, many bots (I asked LZ 234, LZ 15b, MiniGo cormorant, Elf v2) strongly prefer Kageyama's bad 'a' over his 'only move' 1, and I am strongly inclined to believe them over him that is objectively a better move. This seems to me to be a typical example of striving for optimial efficiency, rather than accepting a slightly slack result that minimises bad aji and the chance of you messing up later. If 1 really is the best move, Kageyama-agreeing pros can smugly say "I didn't even need to read the bad aji sequences which shows that 'a' is too much, just based on my experience/intuition and judgement I can discern that the future problems it leaves are not worth the extra points" (and less good forcing moves! very important negative of the net is white b next). However, I suspect a lot of Korean pros would want to play 'a' and the philosophy is "I want this better result, and I tried reading a way it is bad but couldn't find it, so go on, you have to prove to me it's not good or else I get more". So Kageyam's net is like Hane's 80%, avoid risk for a result he judges 'good enough'. The bots say it's not good enough because they can detect a e.g. 0.3 points loss on move 40, and want 100% efficiency.

P.S. on a personal note I too like to strive for 100% efficiency, and spend a lot of time thinking about such interesting finer points. So if I am successful in this I'm quite often leading by move 70. But it's tiring and leaves me short of time so I mess up later and lose, as in the first game of the 2015 British Championship. I was very dispirited by that loss, so the next game I played more 'going through the motions', not really trying to win, just play some moves, try not to lose (somewhat soba-ish), and against my fellow 4d opponent that was good enough to win. So although soba's not going to win any international pro tournaments these days, if us weakies can learn to emulate it it's good enough to win all of the games we play. And rationing your expenditure of mental resources so you can play Hane's 80% all game instead of 100% to start and 0% blunders at the end surely increases you overall chances to win the game.

Bill Spight · Post by **Bill Spight** » Tue Aug 20, 2019 2:45 pm

Uberdude wrote:Trying to find the boundary of the 100% move, and not trying to get even more, let's call it the 110% move if your opponent doesn't punish it but actually it is an overplay so it could end up as a 20% move, is very hard, so Hane is content to stay well away from it and play "good enough", whereas the Korean/Chinese (and Go Seigen) and now AI style is going for more efficiency and getting closer to that 100% line.

FWIW, my impression about bot style in general vs. human style in general is that the bots play more flexibly than humans. They may play more efficiently because they play better, but I think that's a different question. And if you are trying to win, well, good enough is good enough.

It may be better in some sense to offer a possible furikawari where the results are unclear than to nail down territory, but if nailing down territory nails down a 5 pt. win, why not? Remember bots are trained on self play, where both players may be drawn to the furikawari, so they may not have learned to play against opponents who nail things down.

(To be clear, I actually think that human play is not flexible enough, but I also would have more confidence in bots' play if they trained against a variety of styles.

)

There was an interesting example of this on reddit recently, with someone asking for clarification about this position and explanation in Kageyama's "Lessons in the Fundamentals of Go" (taken from one of his games).

$$B
$$ +---------------------------------------+
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X X X c 1 . . . . . . . . . O . . |
$$ | . . O X O O . a . , . . . . O , X . . |
$$ | . . O O X O . b . . . . . . . . . . . |
$$ | . . O X X X X . . . . . . . . . . . . |
$$ | . . O X . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ +---------------------------------------+
Click Here To Show Diagram Code
[go]$$B $$ +---------------------------------------+ $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X X X c 1 . . . . . . . . . O . . | $$ | . . O X O O . a . , . . . . O , X . . | $$ | . . O O X O . b . . . . . . . . . . . | $$ | . . O X X X X . . . . . . . . . . . . | $$ | . . O X . . . . . . . . . . . . . . . | $$ | . . . O . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . X . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . O . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ +---------------------------------------+[/go]
Kageyama writes:
... A stronger amateur would glare at the position and play black 'a', for a larger capture. A professional, however, would find the threat of White 'c' after Black 'a' disquieting, regardless of whether it works immediately or not. To him Black 1 would be the natural and proper move, the only move to make.
Black 1 or Black 'a'? Only an amateur would ask himself this question. A professional would simply dismiss the issue. Neither the intuitive school nor the profit school would give it a second thought. Here we can see another difference between amateur and professional.
Even before you ask an AI and it shreds Kageyama's argument, I find it rather dubious.

Indeed.

Unsurprisngly, many bots (I asked LZ 234, LZ 15b, MiniGo cormorant, Elf v2) strongly prefer Kageyama's bad 'a' over his 'only move' 1, and I am strongly inclined to believe them over him that is objectively a better move.

How strong is their preference? 7% or more? How many playouts? At least 100k for each of these two plays?

Kageyam's net is like Hane's 80%, avoid risk for a result he judges 'good enough'. The bots say it's not good enough because they can detect a e.g. 0.3 points loss on move 40, and want 100% efficiency.

If you mean 0.3% points winrate difference, I agree that the bots should pick the best play they can, but I would not claim that the higher rated play is more efficient. There is too much uncertainty for that. But it's a good enough play.

dfan · Post by **dfan** » Tue Aug 20, 2019 4:22 pm

Bill Spight wrote:
Uberdude wrote:
Unsurprisngly, many bots (I asked LZ 234, LZ 15b, MiniGo cormorant, Elf v2) strongly prefer Kageyama's bad 'a' over his 'only move' 1, and I am strongly inclined to believe them over him that is objectively a better move.
How strong is their preference? 7% or more? How many playouts? At least 100k for each of these two plays?

With 100k playouts, KataGo thinks that a has a winrate of 43.7% and an estimated score of -1.8 (with komi at 7.5, I think), while

has a winrate of 40.0% and an estimated score of -2.9.

Bill Spight · Post by **Bill Spight** » Tue Aug 20, 2019 5:10 pm

dfan wrote:With 100k playouts, KataGo thinks that a has a winrate of 43.7% and an estimated score of -1.8 (with komi at 7.5, I think), while has a winrate of 40.0% and an estimated score of -2.9.

Interesting. Thanks.

jann · Post by **jann** » Tue Aug 20, 2019 6:50 pm

I'm not sure it is correct to generalize from this one position. I saw many cases of the opposite as well, where bots happily traded (in open games) a few point of immediate profit for less aji (thus future flexibility). IMO it's just that in this particular case they see the risk smaller than the gain (not the extra points only I guess but more central control maybe).

Life In 19x19

Verdict on AI verdicts on old pros

Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros

Re: Verdict on AI verdicts on old pros