White has made a placement. How to follow-up?
Leela Zero prefers one of A/B by 15% but she's wrong about it. She misses a tesuji, even after 21k playouts
Five moves down the sequence, she suddenly finds the tesuji and the game is reversed.
KataGo makes a similar mistake but I haven't let it play out as many games.
So, which is best? And which mistake does LZ/KG make - as me and my opponent in the game.
LZ misses a corner tesuji
-
Knotwilg
- Oza
- Posts: 2432
- Joined: Fri Jan 14, 2011 6:53 am
- Rank: KGS 2d OGS 1d Fox 4d
- GD Posts: 0
- KGS: Artevelde
- OGS: Knotwilg
- Online playing schedule: UTC 18:00 - 22:00
- Location: Ghent, Belgium
- Has thanked: 360 times
- Been thanked: 1021 times
- Contact:
-
SoDesuNe
- Gosei
- Posts: 1810
- Joined: Wed Apr 21, 2010 1:57 am
- Rank: KGS 1-dan
- GD Posts: 0
- Has thanked: 490 times
- Been thanked: 365 times
Re: LZ misses a corner tesuji
I just checked with KataGo but due to the close game it very much depends on the komi value.
With 6.5 komi, White is ahead any way and KataGo even sacrificed the placement stone.
With 0.5 komi, Black wins any way according to KataGo and recommends neither 'a' or 'b' but the first line kosumi. Black lets White connect up.
With 6.5 komi, White is ahead any way and KataGo even sacrificed the placement stone.
With 0.5 komi, Black wins any way according to KataGo and recommends neither 'a' or 'b' but the first line kosumi. Black lets White connect up.
-
Bill Spight
- Honinbo
- Posts: 10905
- Joined: Wed Apr 21, 2010 1:24 pm
- Has thanked: 3651 times
- Been thanked: 3373 times
Re: LZ misses a corner tesuji
My first thought.
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
-
Bill Spight
- Honinbo
- Posts: 10905
- Joined: Wed Apr 21, 2010 1:24 pm
- Has thanked: 3651 times
- Been thanked: 3373 times
Re: LZ misses a corner tesuji
jlt wrote:
connects, we may get a smaller ko like this, at the cost of a humungous ko threat for Black. How to play this?
Last edited by Bill Spight on Sun May 17, 2020 11:35 am, edited 2 times in total.
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
-
Knotwilg
- Oza
- Posts: 2432
- Joined: Fri Jan 14, 2011 6:53 am
- Rank: KGS 2d OGS 1d Fox 4d
- GD Posts: 0
- KGS: Artevelde
- OGS: Knotwilg
- Online playing schedule: UTC 18:00 - 22:00
- Location: Ghent, Belgium
- Has thanked: 360 times
- Been thanked: 1021 times
- Contact:
Re: LZ misses a corner tesuji
Haha, you guys make it even better. I hadn't noticed the ko.
jlt's move was what I spotted indeed. Kata/LZ didn't take it consider it until the Black descent.
It's an interesting position!
I'll add a full endgame analysis, where this position plays an even bigger role, as there are many more threats available. Go to 154
jlt's move was what I spotted indeed. Kata/LZ didn't take it consider it until the Black descent.
It's an interesting position!
I'll add a full endgame analysis, where this position plays an even bigger role, as there are many more threats available. Go to 154
-
lightvector
- Lives in sente
- Posts: 759
- Joined: Sat Jun 19, 2010 10:11 pm
- Rank: maybe 2d
- GD Posts: 0
- Has thanked: 114 times
- Been thanked: 916 times
Re: LZ misses a corner tesuji
I'm curious what version of KataGo you were using, and how many playouts / what komi you tried. Similarly LZ.
Did the bots actually miss jlt's sequence, or did you interpret them evaluating most of jlt's sequence as good for white (except for the very last bit) as missing jlt's sequence? Because in fact most of that line *is* good for white. It's possible that they hadn't missed it at all, but evaluated it as good for white because they went on to read the ko as the counter to jlt's sequence and evaluated the ko as winning!
I think the summary of the situation is that "b" and "c" are about the same and both work simply and easily. "a" is one point better, but is very complicated because if black resists, it requires a messy ko fight.
I'm playing around with the latest public 40 block KataGo net, mid thousands to low tens of thousands of playouts clearing the cache a few times to see what it thinks on fresh retries ("clear_cache" in GTP console) on Japanese rules ("kata-set-rules japanese" in GTP console, Lizzie will not do this automatically for you) and I get the following results:
Next white cuts!
White has time to cut here because it gains white a liberty via the cut at "a". If black tries "b" for example, white will play "c" and kill the big black center group, winning 4 liberties to 3. Black can play "d" or "e" to make it a ko instead... but even though this isn't the ko white is "expecting", it's just as winning for white as the other expected one.
So in response to the cut, black must protect right away. The result of being able to interpose this cut of course, is that now the ko is extremely heavy for black because the center group is at stake.
Then white initiates 3-4-5. At this point, if black immediately begins fighting the ko with "a", it's immediately losing, so KataGo starts suggesting moves all around the board to try to generate bigger threats, but KataGo isn't optimistic about it. (80% for white).
So instead we get this:
And this leads into a very, very close endgame (at Japanese 5.5 komi) where white has the barest advantage and is hoping to win by 0.5.
Did the bots actually miss jlt's sequence, or did you interpret them evaluating most of jlt's sequence as good for white (except for the very last bit) as missing jlt's sequence? Because in fact most of that line *is* good for white. It's possible that they hadn't missed it at all, but evaluated it as good for white because they went on to read the ko as the counter to jlt's sequence and evaluated the ko as winning!
I think the summary of the situation is that "b" and "c" are about the same and both work simply and easily. "a" is one point better, but is very complicated because if black resists, it requires a messy ko fight.
I'm playing around with the latest public 40 block KataGo net, mid thousands to low tens of thousands of playouts clearing the cache a few times to see what it thinks on fresh retries ("clear_cache" in GTP console) on Japanese rules ("kata-set-rules japanese" in GTP console, Lizzie will not do this automatically for you) and I get the following results:
- Japanese rules, 7.5 komi or higher: White is clearly winning (by bot standards) and can just play "b" or "c", which is good enough to win: 85%ish. Trading 1 point for simplicity. Clearing the cache and repeating seems to result in it picking b with decent reliability for me, although not 100% consistent.
- Japanese rules, 6.5 komi: White is favored (by bot standards) with "b" or "c", but it's less clearly winning. 65%ish. Clearing the cache and repeating seems to result in it picking "b" or "c" most of the time. Here KataGo has a little bit of "blindspotness" by failing to play "a". Based on the variations it gives, I think sees jlt's line easily, and furthermore sees that white can counter with a ko, but initially doesn't see quite right way to start the ko and/or feels it's messy enough that it would rather go with the 65%ish moves. If you actually play "a" on the board then the winrate quickly goes up to 75% to 80%, since now it's found the right way to start the ko and is reading it deeply enough to see that white can fight it well, in which case missing "a" is clearly a mistake. But if "b" or "c" still win by 0.5, then at least maybe it's not a mistake that costs the game yet.
- Japanese rules, 5.5 komi: Now white is actually losing if playing b or c, probably by half a point. So KataGo seems to consistently choose "a" now. Again, it's already seen past jlt's issue to seeing that white can counter with ko, and however messy the ko initially appears, it's better than playing "b" or "c" and heading into a losing endgame. And with several thousands of playouts, it thinks the ko is eventually winning for white by a bunch, so black must not descend and instead must fight the normal endgame, where maybe white has a bare lead, by 0.5. (60%).
Next white cuts!
White has time to cut here because it gains white a liberty via the cut at "a". If black tries "b" for example, white will play "c" and kill the big black center group, winning 4 liberties to 3. Black can play "d" or "e" to make it a ko instead... but even though this isn't the ko white is "expecting", it's just as winning for white as the other expected one.
So in response to the cut, black must protect right away. The result of being able to interpose this cut of course, is that now the ko is extremely heavy for black because the center group is at stake.
Then white initiates 3-4-5. At this point, if black immediately begins fighting the ko with "a", it's immediately losing, so KataGo starts suggesting moves all around the board to try to generate bigger threats, but KataGo isn't optimistic about it. (80% for white).
So instead we get this:
And this leads into a very, very close endgame (at Japanese 5.5 komi) where white has the barest advantage and is hoping to win by 0.5.
-
Knotwilg
- Oza
- Posts: 2432
- Joined: Fri Jan 14, 2011 6:53 am
- Rank: KGS 2d OGS 1d Fox 4d
- GD Posts: 0
- KGS: Artevelde
- OGS: Knotwilg
- Online playing schedule: UTC 18:00 - 22:00
- Location: Ghent, Belgium
- Has thanked: 360 times
- Been thanked: 1021 times
- Contact:
Re: LZ misses a corner tesuji
Thanks lightvector. Since I hadn't spotted the ko at the time of publishing the problem, I may obviously have misinterpreted LZ's or KG's analyses.
What struck me and remains unexplained, is that (my version of) Lizzie, including both engines, recommends the move that gains one point, probably missing out on the turn at the 1-2 point. Then when played out, it discovers the turn and changes the winning rate drastically.
Perhaps the newer versions of the engines make a perfect analysis of the situation.
What struck me and remains unexplained, is that (my version of) Lizzie, including both engines, recommends the move that gains one point, probably missing out on the turn at the 1-2 point. Then when played out, it discovers the turn and changes the winning rate drastically.
Perhaps the newer versions of the engines make a perfect analysis of the situation.
-
Bill Spight
- Honinbo
- Posts: 10905
- Joined: Wed Apr 21, 2010 1:24 pm
- Has thanked: 3651 times
- Been thanked: 3373 times
Re: LZ misses a corner tesuji
I think we still have to keep in mind the question of what we mean by a perfect, or near perfect, analysis. Winrate estimates assume errors. Errors at a high level of play, but errors nonetheless. For humans, the question is psychological. Which play is more likely to induce an error by my opponent? With apologies to Arthur C. Clarke, we are not quite ready to speak of the psychology of computer programs.Knotwilg wrote:Thanks lightvector. Since I hadn't spotted the ko at the time of publishing the problem, I may obviously have misinterpreted LZ's or KG's analyses.
What struck me and remains unexplained, is that (my version of) Lizzie, including both engines, recommends the move that gains one point, probably missing out on the turn at the 1-2 point. Then when played out, it discovers the turn and changes the winning rate drastically.
Perhaps the newer versions of the engines make a perfect analysis of the situation.
The changing winrate estimates further down in the game tree suggest a horizon effect. I don't know when, if ever, we will be rid of those.
Currently, I think, humans and bots have different strengths and weaknesses. The best analysis, I think, comes from combining the two. IOW, from the human interacting with the program. As, I believe, lightvector has suggested elsewhere.
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
-
RobertJasiek
- Judan
- Posts: 6273
- Joined: Tue Apr 27, 2010 8:54 pm
- GD Posts: 0
- Been thanked: 797 times
- Contact:
Re: LZ misses a corner tesuji
No. AI needs a major redesign (such as always using dynamic komi as a start of better analysis, then assess different move candidates if they are related to different winners for adjacent komi) before they are reliable (let alone "perfect") sources for the endgame. AI only optimises winning percentages and ignores endgame counts.Knotwilg wrote:Perhaps the newer versions of the engines make a perfect analysis of the situation.
-
jlt
- Gosei
- Posts: 1786
- Joined: Wed Dec 14, 2016 3:59 am
- GD Posts: 0
- Has thanked: 185 times
- Been thanked: 495 times
Re: LZ misses a corner tesuji
KataGo does not only take into account winning percentages but also the number of points. KataGo may not play perfect endgame, but may be superhuman (i.e. makes fewer errors than humans)?
-
lightvector
- Lives in sente
- Posts: 759
- Joined: Sat Jun 19, 2010 10:11 pm
- Rank: maybe 2d
- GD Posts: 0
- Has thanked: 114 times
- Been thanked: 916 times
Re: LZ misses a corner tesuji
Are you using LZ272 or something near that and/or KataGo's latest nets (such as g170e b20 s438M)? Also, my winrate estimates to fluctuate a lot in the first few thousand playouts as reading it all out happens, so if you are, maybe it's not enough playouts? Different runs will also vary a little.Knotwilg wrote:Thanks lightvector. Since I hadn't spotted the ko at the time of publishing the problem, I may obviously have misinterpreted LZ's or KG's analyses.
What struck me and remains unexplained, is that (my version of) Lizzie, including both engines, recommends the move that gains one point, probably missing out on the turn at the 1-2 point. Then when played out, it discovers the turn and changes the winning rate drastically.
Perhaps the newer versions of the engines make a perfect analysis of the situation.
For example, here's the evaluation with the latest 20 block net instead of the 40 block net, on 5.5 komi (which low enough to force KataGo to want the ko, since everything else loses):
- 20ba.png (157.54 KiB) Viewed 12659 times
- 20bb.png (162.01 KiB) Viewed 12659 times
-
lightvector
- Lives in sente
- Posts: 759
- Joined: Sat Jun 19, 2010 10:11 pm
- Rank: maybe 2d
- GD Posts: 0
- Has thanked: 114 times
- Been thanked: 916 times
Re: LZ misses a corner tesuji
And here's the same for LZ272: also easily discovered black's resistance and well on to reading out the ko:
LZ can only use 7.5 komi Chinese and doesn't care about score, but ironically, it seems on average more inclined than KataGo to play the score maximizing move even at 7.5 komi. Probably just random differences in policy and things like that. Both nets agree that even at 7.5 komi, if you read out the ko line deeply enough, it's better, it's just a question of pushing through the mess to read it deeply enough when the simple moves look better at first.
But although sometimes LZ272 settles on the ko line, in many runs it often also behaves like KataGo and prefers giving up 1 point since it doesn't read the ko line deeply enough. For example in this run it's discovered the ko line, but still prefers the hane. Here, it's a little less confident in the hane than KataGo would be (77.3% for white vs ~85%) which makes some sense because usually KataGo is a bit stronger than LZ at the endgame, but 77% is still consistent with it being enough to win "comfortably" if the rest of the endgame is mistake-free.
- lz272.png (162.19 KiB) Viewed 12658 times
But although sometimes LZ272 settles on the ko line, in many runs it often also behaves like KataGo and prefers giving up 1 point since it doesn't read the ko line deeply enough. For example in this run it's discovered the ko line, but still prefers the hane. Here, it's a little less confident in the hane than KataGo would be (77.3% for white vs ~85%) which makes some sense because usually KataGo is a bit stronger than LZ at the endgame, but 77% is still consistent with it being enough to win "comfortably" if the rest of the endgame is mistake-free.
- lz272b.png (174.65 KiB) Viewed 12658 times