Takemiya's experiment

Higher level discussions, analysis of professional games, etc., go here.
John Fairbairn
Oza
Posts: 3724
Joined: Wed Apr 21, 2010 3:09 am
Has thanked: 20 times
Been thanked: 4672 times

Takemiya's experiment

Post by John Fairbairn »

I have just read an interesting experiment involving Takemiya Masaki, which perhaps indicates how young pros are adapting to AI.

Way back in 1985, the now defunct Igo Club, which used to be my favourite magazine, presented an article which asked where to play an erasing move in the following position. It was from a game by Takemiya Masaki which I either haven't got or yet done for the GoGoD database. I'm too old/lazy to climb into the loft and rummage through lots of boxes of magazines to refer to the original article (the game is apparently from 1985, but the opponent is not specified).



It may have been just a next-move problem then, but a new article now - part of a series coordinated by Hirata Tomoya - asks Takemiya and young pros how far they would dare enter White's moyo as an erasing move. Obviously the focus was quite different in 1985 - go AI wasn't even a gleam in the eye - and the article raises the interesting point whether simple "next-move" questions are always appropriate. There are plenty of positions that suggest more than move and the way to discriminate depends on the flow or the "story" of the game. (The Japanese text uses the English word "story" but "narrative" is probably better English.)

But Hirata now asked several young players where to play with a view to seeing whether there were differences between Showa and Reiwa (i.e. pre-A and AI generation players). As I say, I have declined to hunt up the original, but Takemiya's own Showa sequence was Black A, White B, Black C. We all know that this vague sort of move is the sort that gets highlighted in commentaries, so we may want to infer that Takemiya was more a than a little pleased with it. But that was then - his Showa intuition. His Reiwa intuiton has changed.

Before you read on you may wish to make your own guess, but make sure it's backed by a positional judgement first. Maybe you'd want to choose both a Showa and a Reiwa move :)

There may be a flaw in the methodology in that there is no mention of komi. The 1985 game would have been played with, at most, 5.5 komi. But Hirata spotted, just like Lizzie, that Black is in dire straits and so needs to wrench control of the game in some way. A quiet erasure is not enough, but one that is too deep (an invasion, really) in White's strong area is courting disaster. It seems to be taken for granted that Black's move does have to be inside this moyo, and Lizzie certainly agrees that this is the main area but she also gives close scrutiny to several moves on the left side.



Reiwa Takemiya now prefers the forceful shoulder hit at A, as does young player Fukuoka Kotaro 1-dan (aged 13). Fukuoka's view was that things are easy for Black if White answers on the third line as Black keeps pressing on the fourth line, and if White caps instead (a standard response) Black can achieve shinogi. Ah, the blithe confidence of youth! (He had his eye on F as a key move.)

Takemiya only mentioned White's third-line response, but instead of Fukuoka's forceful subsequent play, he showed only a vague jump out to the centre. He was actually somewhat surprised to be told he'd played as in the first diagram 34 years ago!

Ueno Asami 3-dan got the evaluation right - she'd prefer to be White - and suggested B for Black. Her strategic thinking seemed to be that w White surrounding move on the right side would out the game out of sight for White, so an erasing move tout court was required. But in intuition terms her first thought was the shoulder hit at A. She eschewed that because a White cap would make her too nervous, especially because of the White wall (but which wall? - answer at the end).

Onishi Ryuhei, 4-dan and rising fast (youngest ever Shinjin-O; he's 19), took a rather different strategic tack. He too obviously sensed Black was n trouble and so wanted to cause confusion, so he opted for Black C, expecting then White D, Black E. He felt this gave Black options of messing around on the upper right side while also aiming at the cut at G. But actually, his initial intuition was the same as Asami's - shoulder hit at A but spooked by a White cap.

My version of Lizzie preferred the shoulder hit. It didn't even list Takemiya's Showa move as a candidate. No surprise there. But what might surprise some is that when this Showa move was forced upon Lizzie, she didn't bat an eyelid and gave the same win rate as before!

This latter phenomenon (rating unconsidered moves highly) is too common - troubling even - to ignore. It needs a name so we can talk about it more. I think it was Bill Spight who first noticed it, and is certainly pointing it out most often, so I propose we call it Spight Analysis, or something like Spight Retrospective Analysis. When we use this tool, human pros can usually be shown to be performing very often only a whisker away from AI-bot level.

It also suggests to me that people like Taemiya may have allowed themselves to be swept along by the fashion for AI-type moves. Maybe he ought to have had more confidence in his original judgement?
Bill Spight
Honinbo
Posts: 10905
Joined: Wed Apr 21, 2010 1:24 pm
Has thanked: 3651 times
Been thanked: 3373 times

Re: Takemiya's experiment

Post by Bill Spight »

John Fairbairn wrote:I have just read an interesting experiment involving Takemiya Masaki, which perhaps indicates how young pros are adapting to AI.
Interesting comparison of the Ghost of Takemiya Past with the Ghost of Takemiya Present. :)
My version of Lizzie preferred the shoulder hit. It didn't even list Takemiya's Showa move as a candidate. No surprise there. But what might surprise some is that when this Showa move was forced upon Lizzie, she didn't bat an eyelid and gave the same win rate as before!

This latter phenomenon (rating unconsidered moves highly) is too common - troubling even - to ignore. It needs a name so we can talk about it more. I think it was Bill Spight who first noticed it, and is certainly pointing it out most often, so I propose we call it Spight Analysis, or something like Spight Retrospective Analysis. When we use this tool, human pros can usually be shown to be performing very often only a whisker away from AI-bot level.
Thanks for the vote, John, but I think that the phenomenon is well known in chess, where computers have ruled for some time. Still, it is something I expected because of training bots by self play. Obviously, that has been good enough to produce superhuman play and to improve current programs, but for analysis I would like a program that can discover such overlooked alternatives, and perhaps even improvements on the bot's first choice. OC, a program that searches too broadly would be too slow to analyze a whole game in a reasonable time. But maybe a human analyst could have the option of having the program do a broad search for selected positions. :)
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.
Uberdude
Judan
Posts: 6727
Joined: Thu Nov 24, 2011 11:35 am
Rank: UK 4 dan
GD Posts: 0
KGS: Uberdude 4d
OGS: Uberdude 7d
Location: Cambridge, UK
Has thanked: 436 times
Been thanked: 3718 times

Re: Takemiya's experiment

Post by Uberdude »

Is there a white stone missing at L16? (Black's shape is weird otherwise, and white only has 17 stones on the board to black's 19, assume 1 white captured top right). Also when you say Lizzie liked Takemiya's old move as much as its suggestions I think you need some qualification of network and playouts as I got quite different results: I just checked with LZ #226 with a few hundred playouts and it thought about 10% worse than shoulder hit. And that black was ahead (missing L16?)

P.S there's a review mode modification of the LZ engine that searches more widely.
John Fairbairn
Oza
Posts: 3724
Joined: Wed Apr 21, 2010 3:09 am
Has thanked: 20 times
Been thanked: 4672 times

Re: Takemiya's experiment

Post by John Fairbairn »

Is there a white stone missing at L16?
You are right. And White has also had one stone captured. Sorry.
User avatar
Knotwilg
Oza
Posts: 2432
Joined: Fri Jan 14, 2011 6:53 am
Rank: KGS 2d OGS 1d Fox 4d
GD Posts: 0
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
Location: Ghent, Belgium
Has thanked: 360 times
Been thanked: 1021 times
Contact:

Re: Takemiya's experiment

Post by Knotwilg »

This is my no-Lizzie but yes-cgoban analysis

kvasir
Lives in sente
Posts: 1040
Joined: Sat Jul 28, 2012 12:29 am
Rank: panda 5 dan
GD Posts: 0
IGS: kvasir
Has thanked: 25 times
Been thanked: 187 times

Re: Takemiya's experiment

Post by kvasir »

Not sure what this position has to do with AI. The shoulder hit is the most direct approach, maybe ideal if you are more worried about playing slack than making a mistake. Takemiya's original move looks super confident, maybe he was playing a weaker opponent? The rest of the game is not included so we can only guess what the plan was or the result.

Leela on my mobile phone still wants action on the left side after Takemiya's moves. Not sure what to make of that.
John Fairbairn wrote:This latter phenomenon (rating unconsidered moves highly) is too common - troubling even - to ignore. It needs a name so we can talk about it more.
I think that is called search bias, the AI has a bias to consider some moves and not others. We can call it blind spots when a good move is not found or the percentage changes dramatically when the variation is played out. It seems negative to talk about blind spots if the AI can find some other good move, just not the one we want it to find, so I would just call it bias in that case.
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Takemiya's experiment

Post by Gomoto »

(KataGo Winrates attached to Knotwilgs file)
John Fairbairn
Oza
Posts: 3724
Joined: Wed Apr 21, 2010 3:09 am
Has thanked: 20 times
Been thanked: 4672 times

Re: Takemiya's experiment

Post by John Fairbairn »

Here's another example from the series:



Takemiya played A in 1960 (Showa; again an unknown game - presumably this was to avoid the other players knowing the original move) but in Reiwa would now play B. His "feeling" was essentially that the Reiwa choice changed the direction of play somewhat. Perhaps this is more of a centre-facing "fighting shape" as recommended by Murashima (see Shuei and AI thread).

The male teenyboppers Fukuoka Kotaro and Onishi Ryuhei both chose C but for quite different reasons and envisaged quite different follow-ups. Fukuoka saw it as the "positive" way to play. Onishi, who correctly (if we rely on Lizzie) saw that Black was already a little behind, was influenced by the fact that some shapes had already been settled and so he reasoned that the upper side would be split 50-50. That in turn would make Black strong in the upper right corner, and he wanted to take advantage of that by mapping out a moyo on the right side. For him, C was a stake in his fence rather than an attack on White. Fukuoka in contrast had his eyes on an invasion at the lower 3-3 point.

Ueno Asami needed to spend just 5 seconds to decide on D - "the only move" - simply because allowing a White approach up there was unconscionable.

Hirata Tomoya had a similar mindset about not allowing the opponent to play first in his chosen area, but chose the lower right (like Takemiya) with E.

All these moves except Takemiya's were Lizzie picks (and she added some others on the lower left side), but using "Spight analysis" (looking at the winrate after playing Takemiya's moves anyway) just confirmed it was a confusing position for both her and the humans. Each move was within a very narrow band, well within the margin of error Bill has talked about. Furthermore, Lizzie's first choice for a long time was C and it was only after a deep search that she switched to E (54.5% over C's 53.9%).

My cynical conclusion is that, at least in this sort of position, it all seems to be a matter of style. Otherwise we have to conclude that pros don't just not know why bots choose their moves. They don't even know why other pros choose their moves!
Bill Spight
Honinbo
Posts: 10905
Joined: Wed Apr 21, 2010 1:24 pm
Has thanked: 3651 times
Been thanked: 3373 times

Re: Takemiya's experiment

Post by Bill Spight »

Gomoto wrote:(KataGo Winrates attached to Knotwilgs file)
Thanks. :)

Do all of these numbers come from analyzing a single position, or are some moves in the variations input by a human and a new analysis made from there?
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.
User avatar
ez4u
Oza
Posts: 2414
Joined: Wed Feb 23, 2011 10:15 pm
Rank: Jp 6 dan
GD Posts: 0
KGS: ez4u
Location: Tokyo, Japan
Has thanked: 2351 times
Been thanked: 1332 times

Re: Takemiya's experiment

Post by ez4u »

John Fairbairn wrote:Here's another example from the series:

...
Takemiya played A in 1960 (Showa; again an unknown game - presumably this was to avoid the other players knowing the original move) but in Reiwa would now play B....
Takemiya was 9 years old in 1960. Is the date incorrect?
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Takemiya's experiment

Post by Gomoto »

Do all of these numbers come from analyzing a single position, or are some moves in the variations input by a human and a new analysis made from there?
I load the sgf file into Lizzie and walk through all variations. I did not enter any moves additional to the file.

(If I would like to explore a move more deeply with more playouts I would enter the move by hand to force KataGo to have a deeper look. But for quick first impressions I just browse the file.)
John Fairbairn
Oza
Posts: 3724
Joined: Wed Apr 21, 2010 3:09 am
Has thanked: 20 times
Been thanked: 4672 times

Re: Takemiya's experiment

Post by John Fairbairn »

Takemiya was 9 years old in 1960. Is the date incorrect
My booboo. It was Showa 60 - 1985.
Tryss
Lives in gote
Posts: 502
Joined: Tue May 24, 2011 1:07 pm
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
Has thanked: 1 time
Been thanked: 153 times

Re: Takemiya's experiment

Post by Tryss »

John Fairbairn wrote:All these moves except Takemiya's were Lizzie picks (and she added some others on the lower left side), but using "Spight analysis" (looking at the winrate after playing Takemiya's moves anyway) just confirmed it was a confusing position for both her and the humans. Each move was within a very narrow band, well within the margin of error Bill has talked about. Furthermore, Lizzie's first choice for a long time was C and it was only after a deep search that she switched to E (54.5% over C's 53.9%).

My cynical conclusion is that, at least in this sort of position, it all seems to be a matter of style. Otherwise we have to conclude that pros don't just not know why bots choose their moves. They don't even know why other pros choose their moves!
I wouldn't call this a "confusing position", but rather an ambiguous position : several choices are possible.

Katago analysis with 30k playouts :
Katago-Takemiya.png
Katago-Takemiya.png (900.2 KiB) Viewed 13354 times
And for exhaustivity :

- A has 716 visits, wr 50.2% and 0.8pts
- D has 130 visits, wr 50.2% and 0.8pts
User avatar
ez4u
Oza
Posts: 2414
Joined: Wed Feb 23, 2011 10:15 pm
Rank: Jp 6 dan
GD Posts: 0
KGS: ez4u
Location: Tokyo, Japan
Has thanked: 2351 times
Been thanked: 1332 times

Re: Takemiya's experiment

Post by ez4u »

Tryss wrote:...
I wouldn't call this a "confusing position", but rather an ambiguous position : several choices are possible.

Katago analysis with 30k playouts :

...
I think you left the komi at the default 7.5 points. When I change the komi to 5.5 points, I get an interesting result. Katago thinks Black is ahead by 4 to 5 points with a winrate around 60%. In it's searching, it solidly favors M4 (above E and right of B). It never invests much time on A or B (1K+ in each). At the time I snapped this picture, it had about 7K visits in C and 4K in D. ALL of these points are evaluated between 60.1% and 60.7%, so flip a coin!
Takemiya 2 2019-10-15_9-39-34.jpg
Takemiya 2 2019-10-15_9-39-34.jpg (365.49 KiB) Viewed 13337 times
Edit:
I forgot and left Lizzie running after writing this post. After about 225K visits, katago switched to slightly preferring Ueno's D! If you look closely, you can see that it estimates the score at +4.6 points for M4 at the bottom vs +4.4 points for D at the top. However, on winrate it gives the tip to D over M4.
Takemiya 2b 2019-10-15_10-23-44.jpg
Takemiya 2b 2019-10-15_10-23-44.jpg (366.08 KiB) Viewed 13333 times
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21
Bill Spight
Honinbo
Posts: 10905
Joined: Wed Apr 21, 2010 1:24 pm
Has thanked: 3651 times
Been thanked: 3373 times

Re: Takemiya's experiment

Post by Bill Spight »

ez4u wrote:Edit:
I forgot and left Lizzie running after writing this post.
Maybe that's the key! :mrgreen:
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.
Post Reply