AlphaGo Teach discussion (Go Tool from DeepMind)

Baywa · **#21**

For what it's worth: I think one can find the games that AlphaGo played against humans inside that tree of variations. For example (I tried that out) game No. 18 of Master (Black) against Ke Jie is contained and one can follow AlphaGo's evaluations. Interestingly, Master does not always choose the "best" continuation. OTOH, when Ke Jie played 22 M4 AlphaGo's winning percentage rose to 49.9 percent. The move suggested as alternative, which defended the l.l. corner would have kept Blacks winning percentage at 47.7.

So it might be worthwhile to follow these evaluations.

Bill Spight · **#22**

moha wrote:

Bill Spight wrote:

moha wrote:

I think they simply mean the MCTS effort / iterations / node expands. EDIT: Actually this was Master still so rollouts as well.

If so, the win rates should be perfect for SDKs, eh?

Master itself did OK, looking only at these numbers, didn't it?

No. Master had/has a value network and did not rely upon MCTS quasi-random rollouts alone. Also, Master built a game tree and did not rely solely upon "win rates".

Again I ask, what were they simulating?

yoyoma · **#23**

John Fairbairn wrote:

The wording of the announcement is unclear as regards selection of "major" variations or indeed what is meant by "opening" (fuseki or joseki - I infer the former).

But insofar as I can make sense of it, the results don't seem to equate with pro tournament practice and the human moves shown must therefore be from a mix of pro and ama games but mostly ama.

If I follow the first two recommended moves (R16 then Q4) DeepMind gives 11 human moves: C17, C16, C15, C4, D17, D16, D4, D3, E17, O3 and P17.

But doing a similar exercise on a pro tournament database (approx. 90,000 games) gives a quite different result with 21 moves tried by humans, Human pros did not try the O3 given above, but did try the rest plus C5, C3, D15, D14, D5, E16, E4, E3, O17, P16 and R6.

The AG win rates are potentially interesting (but how much different from pro win rates as shown by Kombilo?), but if they include many wins against amateurs (and/or internet blitz games) they may be somewhat spurious. It would be nice if Nature could be cajoled into including a properly versed go player as one of their referees.

Just offering users the chance to find plays they have never considered before and thus be "creative" (sounds like an AI definition of the word) is not much of a selling point given the wealth of pro games around, and specifically those that were experimental such as New Fuseki, that already do that.

My interpretation is the process was:
1) DeepMind picks a tree of interesting openings. From your analysis it seems maybe it included some amateur games. I view that as not a problem.
2) DeepMind uses AlphaGo Master to analyze every position in the tree, and puts the resulting winrates in the tree. This analysis has nothing to do with what winrates of pros or amateurs had from those positions. So your point about "wins against amateurs" doesn't apply. If there are some positions in there that are very bad (whether an pro or amateur got to that position is not relevant), Master will tell us so.

moha · **#24**

Bill Spight wrote:

moha wrote:

Bill Spight wrote:

If so, the win rates should be perfect for SDKs, eh?

Master itself did OK, looking only at these numbers, didn't it?

No. Master had/has a value network and did not rely upon MCTS quasi-random rollouts alone. Also, Master built a game tree and did not rely solely upon "win rates".

Again I ask, what were they simulating?

But why do you think these numbers are rollouts alone?

IMO these are Master's complete evaluations, 50% value net + 50% rollout (and Master is known to had weaker value net than Zero, weak enough that dropping rollouts would damage its strength in spite of the huge speedup - and I get the feeling that you equate NN policy-based rollouts to random rollouts of the past).

As for "simulations", I think this word is used for pure MCTS as well, even without rollouts. Such as X simulations = X leaf node expands (in the MCTS tree that Master does build, and which also contains estimated winrates, its only idea of value).

(The word may be a remnant of the idea that for a leaf node to be expanded, a new "simulation" is started from top, that takes a weighted-random branch at each node, until it reaches a leaf which is expanded.)

Bill Spight · **#25**

Bill Spight wrote:

If so, the win rates should be perfect for SDKs, eh?

moha wrote:

Master itself did OK, looking only at these numbers, didn't it?

Bill Spight wrote:

No. Master had/has a value network and did not rely upon MCTS quasi-random rollouts alone. Also, Master built a game tree and did not rely solely upon "win rates".

Again I ask, what were they simulating?

moha wrote:

But why do you think these numbers are rollouts alone?

I don't think that they are rollouts. They are "independent . . . simulations", according to the site. As such, they should actually be probability estimates.

Quote:

IMO these ARE Master's evaluations, 50% value net + 50% rollout (and Master is known to had weaker value net than Zero, weak enough that dropping rollouts would damage its strength in spite of the huge speedup - and I get the feeling that you equate NN policy-based rollouts to random rollouts of the past).

Maybe you are right.

But then they would not be "winning probabilities", as the site claims.

moha · **#26**

(Meanwhile I edited my post above to make it more clear.)

I don't think there are any other meaning behind these words than the amount of search done, and the resulting value estimate.

Uberdude · **#27**

Baywa wrote:

For what it's worth: I think one can find the games that AlphaGo played against humans inside that tree of variations. For example (I tried that out) game No. 18 of Master (Black) against Ke Jie is contained and one can follow AlphaGo's evaluations. Interestingly, Master does not always choose the "best" continuation. OTOH, when Ke Jie played 22 M4 AlphaGo's winning percentage rose to 49.9 percent. The move suggested as alternative, which defended the l.l. corner would have kept Blacks winning percentage at 47.7.

So it might be worthwhile to follow these evaluations.

Indeed, we can use this to find the mistakes of the pros who were losing by move 30 without making any obviously bad moves (actually I think some were fairly obviously bad once you looked at the games on mass, like premature hanging connection in the chinese opening 3-3 invasion and tenuki).

As for that Ke Jie game, I did think having to answer the slide at 3-3 was really painful, and quite probably more so that playing kosumi initially and letting black make whatever loose but not territory formation on the lower side.

I made a thread to highlight and discuss interesting moves/evaluations in this data: viewtopic.php?f=15&t=15310.

djhbrown · **#28**

Alfie says all black's first moves are <50%, so either black should resign before placing a stone, or komi is too much.

pookpooi · **#29**

djhbrown wrote:

Alfie says all black's first moves are <50%, so either black should resign before placing a stone, or komi is too much.

Unless the opponent is God, the resign threshold can be set well below 50%.

And I'm wondering what rule AlphaGo calculate in. From the <50% Black winrate I bet it's Chinese. But DeepZenGo also give <50% to black in Japanese rule too.

vier · **#30**

pookpooi wrote:

And I'm wondering what rule AlphaGo calculate in. From the <50% Black winrate I bet it's Chinese.

The opening book "book.sgf" has KM[7.5] in the header.

luigi · **#31**

djhbrown wrote:

Alfie says all black's first moves are <50%, so either black should resign before placing a stone, or komi is too much.

That's consistent with the reported 45% Black win rate in AlphaGo's selfplay games.

djhbrown · **#32**

pookpooi wrote:

Unless the opponent is God

Since Alfie, when playing with herself - which according to authority is a sin, but according to Woody Allen is engaging with someone he really loves - is happy to continue when she sees her chances are <50%, she is either the Devil or she doesn't think her opponent is God. If the latter, why then do so many people (including her own father) think she is?

Or could it be that as well as being unable to see eyes, she doesn't even know who she's playing against?

Besides, if God intended Man to play Go, She wouldn't have made it so bloody difficult.

Vesa · **#33**

luigi wrote:

djhbrown wrote:

Alfie says all black's first moves are <50%, so either black should resign before placing a stone, or komi is too much.

That's consistent with the reported 45% Black win rate in AlphaGo's selfplay games.

In every game taking black, it may ask itself: "Do I feel lucky?"

Cheers,
Vesa

Gomoto · **#34**

Life is unjust, so is Go.

Uberdude · **#35**

Uberdude wrote:

I actually have a bit of a concern in that rather encouraging creativity, this could make Go more like chess in terms of having a rigid opening book (this giant variation file, 7 MB sgf), will people want to keep playing openings that AlphaGo said was bad?

We are now seeing some effect of this AlphaGo opening book on professional tournaments, in his interview after winning the Shinyao cup Ke Jie seems to say he was frustrated by his opponent memorising the AG opening lines so tried to go off book so that he could get an advantage in the opening like he did in the old days.

Translation from http://sports.sina.com.cn/go/2017-12-26 ... 4407.shtml by reddit user u/gjchangmu:

Quote:

My opponent has remembered the variations in the tool throughtly by heart, and this give me headache. Before the publication of the tool, I think that my openning is good and ahead of the time. After the tool is published, the difference in strength of professional players has been shortened, so I can't get a lead in the openning anymore, which gives me some headache. (In the fifth match) I decided that I need to play differently than AlphaGo suggested, so I played moves from another AI, combined with my own thoughts. However I feel that my opening in the fifth match was not successful, and AIs probably would not rate it high.

Fadedsun · **#36**

Did Ke Jie win that match despite the opening? I hope Alpha Go doesn't deter people from playing different openings. I believe Go is a more flexible game where people don't have to rely on what Alpha Go says is the best result in the opening. Just because one results gives a .5% increase (choosing an arbitrary percent here) in win rate doesn't mean the other moves are "bad".

trickyness · **#37**

How are we going with a analysis of the master vs zero games? Also has anyone heard about a 2 stone game A.I vs Kie Jie
also does anyone know where to find this game? https://www.youtube.com/watch?v=zImt4aRlk1Y

pookpooi · **#38**

trickyness wrote:

Also has anyone heard about a 2 stone game A.I vs Kie Jie
also does anyone know where to find this game?

Please check Ke Jie thread

Maharani · **#39**

I've been having a lot of fun exploring AlphaGo Teach.

An OGS demo board (working on more) with all of AGT's variations after 1. R17 2. Q4 3. D16 4. C4: https://online-go.com/review/412363

AGT positions continued to the end with KataGo from ZBaduk. I played its favourite move after it had at least 12,000 playouts and was also at least 0.3 pp better than its second-favourite move both in terms of "winrate" and "decision" (which sometimes meant 100,000+ playouts).
Game 1: https://online-go.com/review/431592
Game 2 (equal throughout until black crashed and burned in the very endgame): https://online-go.com/review/432118

AlphaGo Teach discussion (Go Tool from DeepMind)

Who is online