Life In 19x19 http://www.lifein19x19.com/ |
|
Guanzi-pu - Lizzie unreliable? http://www.lifein19x19.com/viewtopic.php?f=18&t=16975 |
Page 1 of 1 |
Author: | John Fairbairn [ Tue Oct 15, 2019 3:07 am ] |
Post subject: | Guanzi-pu - Lizzie unreliable? |
I idly tested Lizzie with a problem from Guanzi-pu. She seemed to fluff it in a way. Maybe other bots can do better? The problem I tried is below. You need to know that GZP is not a book of life & death but rather a book of boundary-play problems. Although L&D aspects can crop up, you are actually meant to be concerned with eking out extra profit, which you can do in various way - cutting off tails, fattening the slaughtered calf, encroaching, and so on. A tesuji (or two, or three - this is a hard book) is generally required rather than mere counting. It is White to move, and a throw-in A makes a profitable capture. He cuts off the tail. But he can do better by fattening the calf. With the position as shown, Lizzie gave the correct first move A (according to the original and backed by Go Seigen et al.), as below: Lizzie also gave the correct follow-up as Black B. And then plumped for White C (after B was actually played). But the humans believe White D is the correct move. I tried a variant position in which the rest of the board was filled up with absolutely secure territory for both sides, so that the only plays left were in the problem position. That altered Lizzie's perception but just seemed to make things worse initially. However, she did find the right answer after a few minutes. That seems to suggest to me that bots can't be trusted in such positions unless you allow them to play slowly (much as for humans, I suppose ![]() The full human solution here is embedded in the position below. |
Author: | Kirby [ Tue Oct 15, 2019 4:54 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Just a side note - lizzie is not a bot - it's a graphical interface for analyzing games using Leela Zero. You can use various networks (e.g. different versions of Leela Zero) with Lizzie, and different versions will give different move recommendations. |
Author: | dfan [ Tue Oct 15, 2019 5:06 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Leela Zero (which is the engine I assume you ran using Lizzie as a graphical interface) does not care about the final score of the game, just who won. This is a common issue when using AlphaZero-inspired bots for analyzing - their endgame tends to be slack. So if you fill up the rest of the board, you have to take care to ensure that the correct sequence really is the difference between winning and losing. Using Lizzie, I created a stable whole-board situation and asked KataGo, which does care about the margin of victory, rather than Leela Zero. I actually seem to be lucky in that the whole-board situation was such that the the correct sequence was the difference between winning and losing, and KataGo did find it immediately. It did also find it when I adjusted the komi so that playing at C and D both led to wins, which I imagine Leela Zero would not. |
Author: | Gomoto [ Tue Oct 15, 2019 5:42 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Go is a game about spooky actions at a distance. The local situations are always entangled to the whole board position. The current bots are not simulating rationalizing about the optimal move in a local position, they are simulating intuition about the best way to win a game with the resources at hand (computing power in the AI case, reading abilities in the human case). This is a post by me, so take it with a grain of salt as always. (You could train a bot to get a good intuition about the "correct" solutions in these problems: The bot should not not play games against itself, but has to solve lots of such problems and compare the results to the known "optimal solution". Hope you have some millions of these problems with the "optimal" solutions available for this use case. Who decides what is the optimal solution with no info available about the entangled whole board position?) It is possible to use current bots to analyze such problems, but I know some go related activities that are a hell of a lot more fun. |
Author: | Gomoto [ Tue Oct 15, 2019 5:52 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
And now lets take this haphazard motion one daring step further: |
Author: | Tryss [ Tue Oct 15, 2019 6:28 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer. |
Author: | Gomoto [ Tue Oct 15, 2019 6:50 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Care to show KataGo thinking? My results differ ![]() (Edit: Perhaps, just a matter of letting it run for a while again ![]() |
Author: | jlt [ Tue Oct 15, 2019 6:52 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Unreliability of bots in some situations has been discussed already https://lifein19x19.com/viewtopic.php?p=249425#p249425 P.S. Tryss wrote: Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer. (What an ugly board...) |
Author: | Gomoto [ Tue Oct 15, 2019 6:52 am ] | ||
Post subject: | Re: Guanzi-pu - Lizzie unreliable? | ||
A: (I do not care about the win rate, blue is blue ![]()
|
Author: | Gomoto [ Tue Oct 15, 2019 6:54 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
B: |
Author: | Gomoto [ Tue Oct 15, 2019 6:56 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
C: |
Author: | Gomoto [ Tue Oct 15, 2019 7:01 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
slightly off topic, or should I say slightly unreliable |
Author: | jlt [ Tue Oct 15, 2019 7:11 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
@Gomoto: of course no entity, whether human or electronic, is 100% reliable. When I speak about unreliability, I am comparing bots with strong human players. While LeelaZero's moves are very often better than human ones, there are rare situations where an overwhelming majority of strong human players find a better move than LeelaZero. |
Author: | Gomoto [ Tue Oct 15, 2019 7:30 am ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
Agreed. In every day situations I put more trust in bots. KataGo > LZ in every day situations aka ladder related positions. > as in reliability not as in strength. (And jlt, thank you for explaining your view point in detail to me.) |
Author: | Bill Spight [ Sun Oct 20, 2019 8:36 pm ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
I tried a different way, which may well suck, to create a board for such a problem. Here is a diagram. Does White make the Guanzi-pu play? |
Author: | Gomoto [ Sun Oct 20, 2019 9:11 pm ] |
Post subject: | Re: Guanzi-pu - Lizzie unreliable? |
KataGo: |
Page 1 of 1 | All times are UTC - 8 hours [ DST ] |
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |