AlphaGo vs. AlphaGo: 50 Self-Play Games (May 2017)

[Schachus]

Here is a problem that an amateur dan player should be able to solve, if she has read Mathematical Go. In fact, White's first non-sente move should be obvious.

I'm not an amateur dan and have not read mathematical go, so the first non-sente move is not obvious to me at all, but here is my attempt at getting some sort of an order amongst those moves:

I see two sente moves (e19 and r1) and 15 others.
Amongst those others, p19 and t16 are 1pt reverse sente,
o11 and k19 are ordinary 2pt gote(thus, they are miai, or maybe k19 a bit smaller, bacause you can choose to ignore it, although I dont know what that is good for).
L3, F10 and E4 seem very similar to one another(if not the same) and I would rate them slightly smaller than o11.
A16, J14 and T16 seem to be worse than a 1pt reverse sente, because the reverse moves are not sente, but they have a slight sentish touch to them. Among them I think, J14 is best.
Q4 has slight benefits over a 2pt gote, but I'm not sure they matter at all
T13, K17 and A5 have a slight sentish touch to them, best of those seems to be K17, closely followed by a5.
about g1, I'm not sure at all.

All in all, I think the first non-sente move must be either a reverse sente or one of the moves with a sentish touch, unless I'm missing something great about g1. I have not counted out tedomari(nor have any idea how to do it), so I have no clue, whether 1 pt reverse is better or worse than 2pt gote here. But since there are 2 1pt reverse moves(though p19 seems clearly better..), maybe the first move than wouldnt be obvious...

So my guess is k17, but is could also be p19 or even g1, as far as I know..

I hope you enlighten me, Bill

[Bill Spight]

Can AlphaGo solve it in 45 seconds? Maybe so, but I'll believe it when I see it.

I am quite sure AlphaGo can do that. Remember those unofficial games played online in the name of "Master" this January?

All 60 games except one were fast paced games with three 20 or 30 seconds byo-yomi. Master offered to extend the byo-yomi to one minute when playing with Nie Weiping in consideration of his age.

The whole board game tree has a depth of 90 or more, with a branching factor of up to 17. And that's for humans, who can eliminate a lot of stupid moves. Monte Carlo playouts, which cannot eliminate as many stupid moves, will be almost useless, and I doubt if the value network will help much, either, but I could be wrong about that. I expect that the policy network will eliminate as many, or almost as many stupid plays as humans. That leaves AlphaGo with a search of the game tree, which is humungous. At nearly every point in the solution, to a certain depth, White has only one correct play. Can AlphaGo play correctly at the rate of play that it used most recently? (About 45 sec./move). Maybe so, but, as I said, I'll believe it when I see it.

[Uberdude]

Bill: I've not read Mathematical Go, but presumably the key issue here is which corridors do you push into in which order given that black's best move will be to ignore some of them, and when do such pushes become sente (and sente for how big a group)? That does seem like the kind of position where the meta/abstract thought that humans are good at will help a lot over monte-carlo tree exploration, so share your scepticism of the ability of AlphaGo to solve this based on its ability as Master to play game-typical endgames quickly.

[John Fairbairn]

Can AlphaGo play correctly at the rate of play that it used most recently? (About 45 sec./move). Maybe so, but, as I said, I'll believe it when I see it.

Oh. come now, Bill. You must have realised by now from this forum that AlphaGo can do anything, even chewing gum and peeling bananas. Yose is a mere frippery.

[gowan]

This is perhaps a bit OT for this thread but I find it annoying to see discussions about "AlphaGo" or "Master" as if they were people. Of course they aren't people but what are they? And it isn't perfect. Can it determine what is the best first move in a go game? I doubt it. And there seems to be some question about its endgame performance. Probably there are questionable things about its play in the middlegame, too. Alphago can't explain why its moves are good and perhaps we should be cautious about imitating its moves, just as we advise weaker players to play moves they understand.

[Bill Spight]

Here is a problem that an amateur dan player should be able to solve, if she has read Mathematical Go. In fact, White's first non-sente move should be obvious.

I'm not an amateur dan and have not read mathematical go, so the first non-sente move is not obvious to me at all, but here is my attempt at getting some sort of an order amongst those moves:

I see two sente moves (e19 and r1) and 15 others.
Amongst those others, p19 and t16 are 1pt reverse sente,
o11 and k19 are ordinary 2pt gote(thus, they are miai, or maybe k19 a bit smaller, bacause you can choose to ignore it, although I dont know what that is good for).
L3, F10 and E4 seem very similar to one another(if not the same) and I would rate them slightly smaller than o11.
A16, J14 and T16 seem to be worse than a 1pt reverse sente, because the reverse moves are not sente, but they have a slight sentish touch to them. Among them I think, J14 is best.
Q4 has slight benefits over a 2pt gote, but I'm not sure they matter at all
T13, K17 and A5 have a slight sentish touch to them, best of those seems to be K17, closely followed by a5.
about g1, I'm not sure at all.

All in all, I think the first non-sente move must be either a reverse sente or one of the moves with a sentish touch, unless I'm missing something great about g1. I have not counted out tedomari(nor have any idea how to do it), so I have no clue, whether 1 pt reverse is better or worse than 2pt gote here. But since there are 2 1pt reverse moves(though p19 seems clearly better..), maybe the first move than wouldnt be obvious...

So my guess is k17, but is could also be p19 or even g1, as far as I know..

I hope you enlighten me, Bill

Nice analysis. Identifying the sente and miai is very important in reducing the complexity of any search. (Something that AlphaGo does not do, IIUC. )

May I suggest a separate thread to continue this discussion?

Here. https://www.lifein19x19.com/forum/viewt ... 56#p220156

[Uberdude]

This is perhaps a bit OT for this thread but I find it annoying to see discussions about "AlphaGo" or "Master" as if they were people. Of course they aren't people but what are they?

I don't mind the anthropomorphising of AlphaGo (and indeed do it myself): it makes language use more natural/concise. For example I prefer saying "Master likes to press if you ignore its low approach to 3-4" to "The current iteration of the neural network weight coefficients give rise to a strong likelihood to play the press ...".

And it isn't perfect.

Indeed. I don't think I or anyone claims it is. If someone takes me using a phrase talking about "winning probability" to mean an objective win probability based on perfect play, rather than "AlphaGo's estimation of winning probability (which is good but not perfect, though actually is more like a score than an actual probability)" then again that's making concise language.

Can it determine what is the best first move in a go game? I doubt it.

Determine as in provide a rigourous mathematical proof? No. Just "This very strong player likes [there I go anthropomorphising again] to start here". I did find it interesting though when Michael Redmond asked about the first moves David Silver said AlphaGo likes to start with the standard human corner moves on the 3rd and 4th lines, not crazy centre moves. I wonder how much of that is a leftover bias from the human training, versus it learning on its own what moves work best (what percentage of self-play games didn't start in the corners, I suspect very low).

And there seems to be some question about its endgame performance.

The much talked about "problem" of losing points when it's winning doesn't concern me much, because it is just following its objective function of maximising win probability. When Diana Koszegi 1p from BIBA raised this issue on facebook I replied:

It's really hard to believe that playing bad in the end game gives Alphago a better percentage to win the game....
So actually it feels like he was set to win by 1.5 point or half a point.... (well, as Black, maybe 0.5 or 2.5 since they use Chinese rules..)
Actually Lee Sedol just commented on Baduk TV, that he feels like they set this up on purpose to make amateur players believe that it was a close game

I can believe about the giving up points to increase win percentage and don't think it's a trick: it's a natural consequence of Monte-Carlo tree search. For example with the team game and that unnecessary capture of the 3 stones at the end then imagine in the game tree with the variation where it cuts off the 2-2 then in some playouts of plausible moves from the policy network it won't make the throw in and thus lose the semeai on the left and lose the game. But with its move there's no way that loses. As it's a probabilistic system this makes it choose the safer but point-losing route. That's not to say they couldn't put in effort to fix this perceived problem, either by bolting on some "give extra komi and find move that still wins" approach or remaking the whole program with a different objective function, but it's likely that will have unintended consequences (as neural networks are essentially black boxes of magic) and make the program weaker in other areas, and be a lot of work. So unless AlphaGo actually loses a game following slack endgame in which it misjudged the status of something (like DeepZen vs Park Junghwan) it's simply not a priority to change this aspect of AlphaGo.

Probably there are questionable things about its play in the middlegame, too.

Yup, let's try to find them! It will be hard though as I suspect a lot will come down to positional judgement, which it seems to be better at than top humans. Guo Juan did recommend reviewing pro games with the aim of finding their mistakes as a way to focus study. It might sound arrogant, but you have each player's opponent to help you.

Alphago can't explain why its moves are good and perhaps we should be cautious about imitating its moves, just as we advise weaker players to play moves they understand.

Yes, imitating moves you don't understand can make you lose. But experimenting and losing and learning is also a good way to improve in the long run. (If I just wanted to win rather than have fun and play interesting games I'd always play the Kobayashi opening as it seems to be very effective against low-mid dans, but I don't as I find it overused and deathly dull). That pros are willing to experiment with its new ideas like the early 3-3 invasions, even if they sometimes don't work out well, is a good thing to me for adding creativity and variety (though perhaps you could argue such imitation is not creative if they only play moves given the seal of approval by the authority of AlphaGo, though I think we are also seeing more willingness to experiment).

[Baywa]

Edit: Actually, in this series of selfplays it could be interesting to see how hard and close the endgames were fought. The close final score may not tell the whole story.

Michael Redmond is going to make a video-series about the selfplays in a couple of weeks. He's going to look at the middlegame and endgame mostly. For the opening - especially the early 3-3 invasions - he thinks, that the 50 games may not be enough to make a good judgement. With regard to endgames he'll also look at the, rather obvious, question I posed.
See here: http://www.youtube.com/watch?v=00x7h8Lc4po The discussion about the selfplays starts about midway through the video.

[Baywa]

Right now I'm checking the outcome of the games starting with #50 and going up. The scores seem to be all very close (mostly +0.5). But I'm having problems with #46. The outcome is B+R. There are only a couple dame-points left. My count says B+1 on the board. So I must have missed something either obvious or subtle. Please somebody check!

Edit: That's a seki in the l.l. corner?? That would explain it.
Edit2: Yeah, that's it. Final score: B184, W175 -> B+1.5

[kwhyte]

I just watched Redmond's analysis of the first four games and they are even more impressive and confusing than I expected. Unlike the 60 "master" games, I'm not even sure I learned anything I could try to apply. Still a lot of fun to watch. It sounds like he plans to go through all 50 which is a massive project - I'm definitely looking forward to it.