Guanzi-pu - Lizzie unreliable?

For discussing go computing, software announcements, etc.
John Fairbairn
Oza
Posts: 3724
Joined: Wed Apr 21, 2010 3:09 am
Has thanked: 20 times
Been thanked: 4672 times

Guanzi-pu - Lizzie unreliable?

Post by John Fairbairn »

I idly tested Lizzie with a problem from Guanzi-pu. She seemed to fluff it in a way. Maybe other bots can do better?

The problem I tried is below. You need to know that GZP is not a book of life & death but rather a book of boundary-play problems. Although L&D aspects can crop up, you are actually meant to be concerned with eking out extra profit, which you can do in various way - cutting off tails, fattening the slaughtered calf, encroaching, and so on. A tesuji (or two, or three - this is a hard book) is generally required rather than mere counting.



It is White to move, and a throw-in A makes a profitable capture. He cuts off the tail. But he can do better by fattening the calf.

With the position as shown, Lizzie gave the correct first move A (according to the original and backed by Go Seigen et al.), as below:



Lizzie also gave the correct follow-up as Black B. And then plumped for White C (after B was actually played). But the humans believe White D is the correct move.

I tried a variant position in which the rest of the board was filled up with absolutely secure territory for both sides, so that the only plays left were in the problem position. That altered Lizzie's perception but just seemed to make things worse initially. However, she did find the right answer after a few minutes.

That seems to suggest to me that bots can't be trusted in such positions unless you allow them to play slowly (much as for humans, I suppose :)), and the degree of trust may be much less early in the game. Also, how much time do you have to give them before you can trust them? If you have no human solution as a yardstick, we amateurs can be left high and dry. Pure L&D may be different because the stakes are higher and the outcome easier to understand.

The full human solution here is embedded in the position below.

Kirby
Honinbo
Posts: 9553
Joined: Wed Feb 24, 2010 6:04 pm
GD Posts: 0
KGS: Kirby
Tygem: 커비라고해
Has thanked: 1583 times
Been thanked: 1707 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Kirby »

Just a side note - lizzie is not a bot - it's a graphical interface for analyzing games using Leela Zero. You can use various networks (e.g. different versions of Leela Zero) with Lizzie, and different versions will give different move recommendations.
be immersed
dfan
Gosei
Posts: 1599
Joined: Wed Apr 21, 2010 8:49 am
Rank: AGA 2k Fox 3d
GD Posts: 61
KGS: dfan
Has thanked: 891 times
Been thanked: 534 times
Contact:

Re: Guanzi-pu - Lizzie unreliable?

Post by dfan »

Leela Zero (which is the engine I assume you ran using Lizzie as a graphical interface) does not care about the final score of the game, just who won. This is a common issue when using AlphaZero-inspired bots for analyzing - their endgame tends to be slack. So if you fill up the rest of the board, you have to take care to ensure that the correct sequence really is the difference between winning and losing.

Using Lizzie, I created a stable whole-board situation and asked KataGo, which does care about the margin of victory, rather than Leela Zero. I actually seem to be lucky in that the whole-board situation was such that the the correct sequence was the difference between winning and losing, and KataGo did find it immediately. It did also find it when I adjusted the komi so that playing at C and D both led to wins, which I imagine Leela Zero would not.
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

Go is a game about spooky actions at a distance. The local situations are always entangled to the whole board position.

The current bots are not simulating rationalizing about the optimal move in a local position, they are simulating intuition about the best way to win a game with the resources at hand (computing power in the AI case, reading abilities in the human case).

This is a post by me, so take it with a grain of salt as always.



(You could train a bot to get a good intuition about the "correct" solutions in these problems: The bot should not not play games against itself, but has to solve lots of such problems and compare the results to the known "optimal solution". Hope you have some millions of these problems with the "optimal" solutions available for this use case. Who decides what is the optimal solution with no info available about the entangled whole board position?)


It is possible to use current bots to analyze such problems, but I know some go related activities that are a hell of a lot more fun.
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

And now lets take this haphazard motion one daring step further:
We human beings developed by natural selection not only rational reasoning but also intuition, because our world is an entangled one as well like the cosmos we call goban ;-)

Let's go on!




And by the way analyzing the problem with KataGo did not add something usefull to the discussion in my opinion.
Last edited by Gomoto on Tue Oct 15, 2019 7:06 am, edited 1 time in total.
Tryss
Lives in gote
Posts: 502
Joined: Tue May 24, 2011 1:07 pm
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
Has thanked: 1 time
Been thanked: 153 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Tryss »

Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer.

Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

Care to show KataGo thinking? My results differ :-)

(Edit: Perhaps, just a matter of letting it run for a while again ;-) see further down)
Last edited by Gomoto on Tue Oct 15, 2019 7:07 am, edited 2 times in total.
User avatar
jlt
Gosei
Posts: 1786
Joined: Wed Dec 14, 2016 3:59 am
GD Posts: 0
Has thanked: 185 times
Been thanked: 495 times

Re: Guanzi-pu - Lizzie unreliable?

Post by jlt »

Unreliability of bots in some situations has been discussed already

viewtopic.php?p=249425#p249425

P.S.
Tryss wrote:Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer.
(What an ugly board...)
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

A: (I do not care about the win rate, blue is blue :-) )
Attachments
kat1.JPG
kat1.JPG (109.28 KiB) Viewed 11546 times
Last edited by Gomoto on Tue Oct 15, 2019 7:03 am, edited 1 time in total.
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

B:
kat2.JPG
kat2.JPG (197.93 KiB) Viewed 11545 times
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

C:
Kat3.jpg
Kat3.jpg (171.25 KiB) Viewed 11550 times
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

slightly off topic, or should I say slightly unreliable
Unreliability of bots in some situations has been discussed already
Unreliability of bots opposed to what?

I am not used to anything reliable in my world. I take precautions to survive every day, allthough I know I will fail in the end without doubt :twisted: :D
User avatar
jlt
Gosei
Posts: 1786
Joined: Wed Dec 14, 2016 3:59 am
GD Posts: 0
Has thanked: 185 times
Been thanked: 495 times

Re: Guanzi-pu - Lizzie unreliable?

Post by jlt »

@Gomoto: of course no entity, whether human or electronic, is 100% reliable. When I speak about unreliability, I am comparing bots with strong human players. While LeelaZero's moves are very often better than human ones, there are rare situations where an overwhelming majority of strong human players find a better move than LeelaZero.
Gomoto
Gosei
Posts: 1733
Joined: Sun Nov 06, 2016 6:56 am
GD Posts: 0
Location: Earth
Has thanked: 621 times
Been thanked: 310 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Gomoto »

Agreed. In every day situations I put more trust in bots. KataGo > LZ in every day situations aka ladder related positions. > as in reliability not as in strength.

(And jlt, thank you for explaining your view point in detail to me.)
Bill Spight
Honinbo
Posts: 10905
Joined: Wed Apr 21, 2010 1:24 pm
Has thanked: 3651 times
Been thanked: 3373 times

Re: Guanzi-pu - Lizzie unreliable?

Post by Bill Spight »

I tried a different way, which may well suck, to create a board for such a problem. Here is a diagram. Does White make the Guanzi-pu play?
Click Here To Show Diagram Code
[go]$$Wc White to play
$$ ---------------------------------------
$$ | . . . X X . . . . . . . . . . . . . . |
$$ | . . O X O X X . X O . X . . . . . . . |
$$ | . . O X O O X X X O . . . . O O . . . |
$$ | . . . O O . O O X O . X . X X , O O O |
$$ | . . . . . . . O O X X . X O . . X X O |
$$ | . O O . . . . . . . . . O X X X X O X |
$$ | . X X O O X . X O . . . . O O O O O . |
$$ | . . . X O O X X O O O O . . . . . O . |
$$ | . . . . O . O X O X X X X . . . X X . |
$$ | . X X O O O O X . , . . . . . X . . . |
$$ | O O O X X X X X X X X O X . . . . . . |
$$ | O X . X . . . O O . O O X . . . . . . |
$$ | O X X . . O O . O . O O X . X X . . . |
$$ | O . . . . O X O O X X O O . O X O . . |
$$ | . O O O . O X O X . X X O O X . X O . |
$$ | . X X , . . X X X X O X O X . X O O . |
$$ | . . . X . X . . . X O O O . X X X X . |
$$ | . . . . . . X . O O . . . . X . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.
Post Reply