AlphaGo vs. AlphaGo: 50 Self-Play Games (May 2017)

Uberdude · **#41**

Ah yes, my mistake, not AlphaGo's ;-)

. I missed White's problem in the middle so white loses a point there and black gets the last dame to win by half.

Cassandra · **#42**

Uberdude wrote:

billyswong wrote:

Hi, I am new here. May anyone help me on the game 20? Its official result is B+resign. However, when I try to play out the end game moves and see how close things are (inside the deepmind website), it ends up at "White wins by 0.5 points" showing in front of me. Is it that I played some moves wrong?

Very good point. I think in fact it should be White wins by 1.5, because he can get 2 of the 3 available gote dame: the 2 on the left are simple sente atari dame, and on the right side black needs 2 defensive moves inside so white gets both those dame. Did white AlphaGo not realise this and thus resigned a won game?! I also note there are a lot of captures and white has more than black, another possible source of bugs.

None of this ...

There are only two dame that are not sente (E5, L3), so each side will get one of these.

When applying CHINESE rules (as all the selfplay-games are played under), Black wins by 0.5 points, as he has played the last move.

It seems to me that the interface on deepmind's website uses JAPANESE style for counting (= territory plus prisoners), thus it has a white win by 0.5 points, not considering the dame on the board.

Baywa · **#43**

johnsmith wrote:

Black wins by 0.5. That's why white resigned and you can see quite often in these 50 games that one resigns because the opponent was leading by a very small margin of 0.5 or 1.5.

I noticed that, too. For example game #39 is very similar in this regard.

Quote:

Edit: AlphaGO does NEVER make mistakes!

Seems like that, at least for the endgame. We've seen several examples (e.g. game #1 against Ke Jie). Although AlphaGo "throws away" several points in exchange for higher win probability (the way it defines it) it has complete control about the outcome from several moves back.

Edit: Actually, in this series of selfplays it could be interesting to see how hard and close the endgames were fought. The close final score may not tell the whole story.

Bill Spight · **#44**

Baywa wrote:

johnsmith wrote:

AlphaGO does NEVER make mistakes!

Seems like that, at least for the endgame. We've seen several examples (e.g. game #1 against Ke Jie). Although AlphaGo "throws away" several points in exchange for higher win probability (the way it defines it) it has complete control about the outcome from several moves back.

I doubt if AlphaGo will drop a point in the late endgame, because of its reading ability. However, what it means by win probability is, AFAIK, unknown, even to its developers. (Because it depends in large part on what the evaluation network has learned.)

One advantage that humans have in the endgame is that the game tends to divide up into independent regions of play. Humans can analyze each independent region separately, which can greatly simplify the challenge of reading. We still have to combine play in all regions, but playing in the hottest region is nearly always correct. AlphaGo, by contrast, always builds a whole board game tree, and must explore more branches than humans.

Here is a problem that an amateur dan player should be able to solve, if she has read Mathematical Go. In fact, White's first non-sente move should be obvious.

Can AlphaGo solve it in 45 seconds? Maybe so, but I'll believe it when I see it.

Quote:

Actually, in this series of selfplays it could be interesting to see how hard and close the endgames were fought. The close final score may not tell the whole story.

I have taken a look at the end of game 31. Neither player dropped a point (that I found), and the play of the approach ko in the top left corner was impressive.

However, the game came down to the final 4/3 pt. ko (by area scoring). Go to move B305. Each player dropped a ko threat, and Black allowed White to get a virtual ko threat (W310). Since White won the ko, anyway, none of this affected the result. Black's play may be explained as playing for an error, but I fail to see how White's play (W308) could do anything but lower the probability of winning. (If it had any effect on that at all, OC.

)

Since AlphaGo has shown no sign of plateauing, I expect that it still makes mistakes in the opening and middle game. But are humans good enough to find them? But it would not surprise me for humans to find AlphaGo occasionally dropping a point or two 80-100 moves from the end.

billyswong · **#45**

Bill Spight wrote:

Can AlphaGo solve it in 45 seconds? Maybe so, but I'll believe it when I see it.

I am quite sure AlphaGo can do that. Remember those unofficial games played online in the name of "Master" this January? ;-)

Wikipedia wrote:

All 60 games except one were fast paced games with three 20 or 30 seconds byo-yomi. Master offered to extend the byo-yomi to one minute when playing with Nie Weiping in consideration of his age.

Schachus · **#46**

Bill Spight wrote:

Here is a problem that an amateur dan player should be able to solve, if she has read Mathematical Go. In fact, White's first non-sente move should be obvious.

I'm not an amateur dan and have not read mathematical go, so the first non-sente move is not obvious to me at all, but here is my attempt at getting some sort of an order amongst those moves:

Bill Spight · **#47**

billyswong wrote:

Bill Spight wrote:

Can AlphaGo solve it in 45 seconds? Maybe so, but I'll believe it when I see it.

I am quite sure AlphaGo can do that. Remember those unofficial games played online in the name of "Master" this January? ;-)

Wikipedia wrote:

All 60 games except one were fast paced games with three 20 or 30 seconds byo-yomi. Master offered to extend the byo-yomi to one minute when playing with Nie Weiping in consideration of his age.

The whole board game tree has a depth of 90 or more, with a branching factor of up to 17. And that's for humans, who can eliminate a lot of stupid moves. Monte Carlo playouts, which cannot eliminate as many stupid moves, will be almost useless, and I doubt if the value network will help much, either, but I could be wrong about that. I expect that the policy network will eliminate as many, or almost as many stupid plays as humans. That leaves AlphaGo with a search of the game tree, which is humungous. At nearly every point in the solution, to a certain depth, White has only one correct play. Can AlphaGo play correctly at the rate of play that it used most recently? (About 45 sec./move). Maybe so, but, as I said, I'll believe it when I see it.

Uberdude · **#48**

Bill: I've not read Mathematical Go, but presumably the key issue here is which corridors do you push into in which order given that black's best move will be to ignore some of them, and when do such pushes become sente (and sente for how big a group)? That does seem like the kind of position where the meta/abstract thought that humans are good at will help a lot over monte-carlo tree exploration, so share your scepticism of the ability of AlphaGo to solve this based on its ability as Master to play game-typical endgames quickly.

John Fairbairn · **#49**

Quote:

Can AlphaGo play correctly at the rate of play that it used most recently? (About 45 sec./move). Maybe so, but, as I said, I'll believe it when I see it.

Oh. come now, Bill. You must have realised by now from this forum that AlphaGo can do anything, even chewing gum and peeling bananas. Yose is a mere frippery.

gowan · **#50**

This is perhaps a bit OT for this thread but I find it annoying to see discussions about "AlphaGo" or "Master" as if they were people. Of course they aren't people but what are they? And it isn't perfect. Can it determine what is the best first move in a go game? I doubt it. And there seems to be some question about its endgame performance. Probably there are questionable things about its play in the middlegame, too. Alphago can't explain why its moves are good and perhaps we should be cautious about imitating its moves, just as we advise weaker players to play moves they understand.

Bill Spight · **#51**

Schachus wrote:

Bill Spight wrote:

Here is a problem that an amateur dan player should be able to solve, if she has read Mathematical Go. In fact, White's first non-sente move should be obvious.

I'm not an amateur dan and have not read mathematical go, so the first non-sente move is not obvious to me at all, but here is my attempt at getting some sort of an order amongst those moves:

Nice analysis. Identifying the sente and miai is very important in reducing the complexity of any search. (Something that AlphaGo does not do, IIUC.

)

May I suggest a separate thread to continue this discussion?

Here. forum/viewtopic.php?p=220156#p220156

Uberdude · **#52**

gowan wrote:

This is perhaps a bit OT for this thread but I find it annoying to see discussions about "AlphaGo" or "Master" as if they were people. Of course they aren't people but what are they?

I don't mind the anthropomorphising of AlphaGo (and indeed do it myself): it makes language use more natural/concise. For example I prefer saying "Master likes to press if you ignore its low approach to 3-4" to "The current iteration of the neural network weight coefficients give rise to a strong likelihood to play the press ...".

gowan wrote:

And it isn't perfect.

Indeed. I don't think I or anyone claims it is. If someone takes me using a phrase talking about "winning probability" to mean an objective win probability based on perfect play, rather than "AlphaGo's estimation of winning probability (which is good but not perfect, though actually is more like a score than an actual probability)" then again that's making concise language.

gowan wrote:

Can it determine what is the best first move in a go game? I doubt it.

Determine as in provide a rigourous mathematical proof? No. Just "This very strong player likes [there I go anthropomorphising again] to start here". I did find it interesting though when Michael Redmond asked about the first moves David Silver said AlphaGo likes to start with the standard human corner moves on the 3rd and 4th lines, not crazy centre moves. I wonder how much of that is a leftover bias from the human training, versus it learning on its own what moves work best (what percentage of self-play games didn't start in the corners, I suspect very low).

gowan wrote:

And there seems to be some question about its endgame performance.

The much talked about "problem" of losing points when it's winning doesn't concern me much, because it is just following its objective function of maximising win probability. When Diana Koszegi 1p from BIBA raised this issue on facebook I replied:

Diana Koszegi wrote:

It's really hard to believe that playing bad in the end game gives Alphago a better percentage to win the game....
So actually it feels like he was set to win by 1.5 point or half a point.... (well, as Black, maybe 0.5 or 2.5 since they use Chinese rules..)
Actually Lee Sedol just commented on Baduk TV, that he feels like they set this up on purpose to make amateur players believe that it was a close game

I wrote:

I can believe about the giving up points to increase win percentage and don't think it's a trick: it's a natural consequence of Monte-Carlo tree search. For example with the team game and that unnecessary capture of the 3 stones at the end then imagine in the game tree with the variation where it cuts off the 2-2 then in some playouts of plausible moves from the policy network it won't make the throw in and thus lose the semeai on the left and lose the game. But with its move there's no way that loses. As it's a probabilistic system this makes it choose the safer but point-losing route. That's not to say they couldn't put in effort to fix this perceived problem, either by bolting on some "give extra komi and find move that still wins" approach or remaking the whole program with a different objective function, but it's likely that will have unintended consequences (as neural networks are essentially black boxes of magic) and make the program weaker in other areas, and be a lot of work. So unless AlphaGo actually loses a game following slack endgame in which it misjudged the status of something (like DeepZen vs Park Junghwan) it's simply not a priority to change this aspect of AlphaGo.

gowan wrote:

Probably there are questionable things about its play in the middlegame, too.

Yup, let's try to find them! It will be hard though as I suspect a lot will come down to positional judgement, which it seems to be better at than top humans. Guo Juan did recommend reviewing pro games with the aim of finding their mistakes as a way to focus study. It might sound arrogant, but you have each player's opponent to help you.

gowan wrote:

Alphago can't explain why its moves are good and perhaps we should be cautious about imitating its moves, just as we advise weaker players to play moves they understand.

Yes, imitating moves you don't understand can make you lose. But experimenting and losing and learning is also a good way to improve in the long run. (If I just wanted to win rather than have fun and play interesting games I'd always play the Kobayashi opening as it seems to be very effective against low-mid dans, but I don't as I find it overused and deathly dull). That pros are willing to experiment with its new ideas like the early 3-3 invasions, even if they sometimes don't work out well, is a good thing to me for adding creativity and variety (though perhaps you could argue such imitation is not creative if they only play moves given the seal of approval by the authority of AlphaGo, though I think we are also seeing more willingness to experiment).

Baywa · **#53**

Baywa wrote:

Edit: Actually, in this series of selfplays it could be interesting to see how hard and close the endgames were fought. The close final score may not tell the whole story.

Michael Redmond is going to make a video-series about the selfplays in a couple of weeks. He's going to look at the middlegame and endgame mostly. For the opening - especially the early 3-3 invasions - he thinks, that the 50 games may not be enough to make a good judgement. With regard to endgames he'll also look at the, rather obvious, question I posed.
See here: http://www.youtube.com/watch?v=00x7h8Lc4po The discussion about the selfplays starts about midway through the video.

Baywa · **#54**

Right now I'm checking the outcome of the games starting with #50 and going up. The scores seem to be all very close (mostly +0.5). But I'm having problems with #46. The outcome is B+R. There are only a couple dame-points left. My count says B+1 on the board. So I must have missed something either obvious or subtle. Please somebody check!

Edit: That's a seki in the l.l. corner?? That would explain it.
Edit2: Yeah, that's it. Final score: B184, W175 -> B+1.5

kwhyte · **#55**

I just watched Redmond's analysis of the first four games and they are even more impressive and confusing than I expected. Unlike the 60 "master" games, I'm not even sure I learned anything I could try to apply. Still a lot of fun to watch. It sounds like he plans to go through all 50 which is a massive project - I'm definitely looking forward to it.

AlphaGo vs. AlphaGo: 50 Self-Play Games (May 2017)

Who is online