Comparison of AlphaGo Zero and Modern Zero Algorithms

[SHKD19]

Comparison of AlphaGo Zero (both 20 & 40 residual blocks sized) with its Deep Reinforcement Learning Descendents, Based on Matching the 40 Samples of AlphaGo Zero’s Moves.
In this text Leela Zero, ELF OpenGo, PhoenixGo, MiniGo & KataGo being tested for capability to detect and match the moves of their RL Big Brother.
Are they good enough for this? Is AGZ still actual template for them?

https://docs.google.com/document/d/114h ... sp=sharing

If you have any critics or ideas about this subject, you are welcome to share it here

[xela]

Is this your own work or someone else's?

[jlt]

I didn't read in detail because it's a bit long and some sentences not very precise, but if I understand the idea correctly, you look at AlphagoZero selfplay games, and say that an AI is strong if it can find many AZ moves.

Some of AZ's moves are found by none of the AIs, for instance the last move of the main line of the following game:



I tested it with 15-block trained on 40b LeelaZero. Indeed it didn't find the move, and wants to play the move on the variation. Once played, LZ thinks that AZ's move is 0.3% better. I don't think it's conclusive. If several good moves are equivalent, and if LZ finds one of them and not the one of AZ, it doesn't mean that LZ is weaker than AZ.

[Bill Spight]

I didn't read in detail because it's a bit long and some sentences not very precise, but if I understand the idea correctly, you look at AlphagoZero selfplay games, and say that an AI is strong if it can find many AZ moves.

Some of AZ's moves are found by none of the AIs, for instance the last move of the main line of the following game:

I'm not sure what you mean by the main line. Is it a generated variation? If so, the last move could well be weak. So what if it was off the radar of another bot, or all of them?

[jlt]

Sorry, I may have used a wrong word, perhaps "main variation" or "main branch" was more correct? Anyway, the file above consists of the first 36 moves of an Alphago Zero selfplay match. was at N13. According to the document, it wasn't on the radar of any other bot. My version of Leelazero indeed didn't consider it at all, and it chose M12. But once played, LZ thinks that N13 is only 0.3% better than M12.

[Bill Spight]

Sorry, I may have used a wrong word, perhaps "main variation" or "main branch" was more correct? Anyway, the file above consists of the first 36 moves of an Alphago Zero selfplay match. was at N13. According to the document, it wasn't on the radar of any other bot. My version of Leelazero indeed didn't consider it at all, and it chose M12. But once played, LZ thinks that N13 is only 0.3% better than M12.

OK. Many thanks.

[jann]

You cannot answer the question whether AGZ is stronger than current bots by assuming it is - which expecting move selections to match its does.

At least you need control by comparing to how well the new bots can predict each other's moves. Verifying whether the test moves are actually the best in the position (like running a long eval in various bots after making each possible legal move in the position) also seems advisable.

[lightvector]

Additionally, in the reddit post on this (https://www.reddit.com/r/baduk/comments ... n_zero_ai/) one of the commenters suggests that the AlphaGo Zero selfplay games were actually played with quite large numbers of playouts, much larger than was used in any of the analysis in the document.

I have not verified this, but if true, then AGZ would of course appear to be stronger and/or find moves that other bots are not finding. Bots can and do sometimes change their minds about the best moves at very large numbers of playouts and occasionally even select moves that they've put literally 0 playouts into early on, even if this is not the most common outcome.

[SHKD19]

You cannot answer the question whether AGZ is stronger than current bots by assuming it is - which expecting move selections to match its does.

At least you need control by comparing to how well the new bots can predict each other's moves. Verifying whether the test moves are actually the best in the position (like running a long eval in various bots after making each possible legal move in the position) also seems advisable.

Agree. I had the same idea in the beginning. But I simply don't have so much free time to assess what every bot from the list is thinking about all the others. Even playing with AGZ moves took sometime to finish. Doing 20X20 AI's cross-assessment would quickly turn from fun into a headeche

[SHKD19]

Additionally, in the reddit post on this (https://www.reddit.com/r/baduk/comments ... n_zero_ai/) one of the commenters suggests that the AlphaGo Zero selfplay games were actually played with quite large numbers of playouts, much larger than was used in any of the analysis in the document.

I have not verified this, but if true, then AGZ would of course appear to be stronger and/or find moves that other bots are not finding. Bots can and do sometimes change their minds about the best moves at very large numbers of playouts and occasionally even select moves that they've put literally 0 playouts into early on, even if this is not the most common outcome.

Thank for both of your comments! I already found few mistakes in the document. Sorry for this This is not a kind of exculpation, but even deadly serious scientific publications may contain severe errors, while for me the "beta-test" of AI analytical comparison was just a fun. And I share it for same reason Every mistake can be fixed, if author is not blind and stupid

Anyway, the fact that older bots in lots of cases matched the AGZ moves better is a bit weird. All the bots from the list had an equal search limitations, so the playouts itself is not the reason of such a great performance from older AI's. I did not expect this in any numbers of simulations.

[xela]

Anyway, the fact that older bots in lots of cases matched the AGZ moves better is a bit weird. All the bots from the list had an equal search limitations, so the playouts itself is not the reason of such a great performance from older AI's.

Maybe the newer bots are finding different moves because they are actually stronger than AGZ now?

[SHKD19]

Anyway, the fact that older bots in lots of cases matched the AGZ moves better is a bit weird. All the bots from the list had an equal search limitations, so the playouts itself is not the reason of such a great performance from older AI's.

Maybe the newer bots are finding different moves because they are actually stronger than AGZ now?

Facebook thinks that their ELF v2 is arguably comparable to only AlphaGo Zero 20B, while AlphaGo Zero 40B is much stronger than its smaller Alpha brother. Who knows, maybe current Leela Zero #254 40B already is able to compete with same size AGZ, but I can't see any reason to expect this from MiniGo v14 or v15, or another 20 blocks Network.