Will AI ever beat humans at Go with energy parity?

[Mike Novack]

Neural nets have some interesting properties. That makes it difficult to discuss things like "if komi changes, wouldn't that mean retraining the net from scratch?".

For example --- suppose a neural net has been trained from scratch to evaluate a function. You then randomly destroy the cell values in some percentage of the cells << you give the neural net the equivalent of a "stroke" >> You then retrain the net to evaluate the function. It will NOT take anywhere as near as long (as much training) as it did before when starting from scratch.

Think about how the "komi" question might be related to that << because of then komi change, some percentage of the cell values are wrong >>

[Bohdan]

I disagree, and various statements from DeepMind people suggest they would too. The breakthrough and source of AlphaGo's power is the value network, a quick and strong board evaluation function that didn't exist in bots prior to AlphaGo.

Then why do they need 500 CPU and 200 GPU? Why do Zen need 44 cores? You cannot play without reading out variations. Value and policy networks only narrowed the number of candidate moves which helped to read variations more efficiently. In the Nature paper they wrote that network-only bot without MC searching scored 85% against Pachi 2d version.

Did you read what I wrote? Self-play games form the rules without any human expert games as initial training.

Last year DeepMind promised that they will train a bot only using the self-play without any human players. No success so far. Why do you think it will work? And remember that you need a millions of self-played games to make some relevant database.

That might be true for the value network (not policy). But it might no longer be true. When asked about Zen's problems with komi in its value network in the WGC Hideki Kato said AlphaGo had apparently solved the komi problem somehow. I don't know if that is by retraining or something cleverer. But as Demis said training only takes a week now (probably helped by lots of TPUs), 10 years seems rather an over-estimation.

Transfer the knowledge between neural networks with different architecture is quite a challenge. No success is secured.

[Uberdude]

I disagree, and various statements from DeepMind people suggest they would too. The breakthrough and source of AlphaGo's power is the value network, a quick and strong board evaluation function that didn't exist in bots prior to AlphaGo.

Then why do they need 500 CPU and 200 GPU? Why do Zen need 44 cores? You cannot play without reading out variations. Value and policy networks only narrowed the number of candidate moves which helped to read variations more efficiently. In the Nature paper they wrote that network-only bot without MC searching scored 85% against Pachi 2d version.

Yes, you still need to read, but as the policy and value networks get better you need to read less. For Lee Sedol they needed lots of processing power because the networks weren't as good as they are now. Demis said they only used one GPU for Master so, assuming he's correct, that suggests their networks are now even stronger (following lots of up-front training computations) so AlphaGo can play very strongly without hundreds of processors. Remember the Nature paper was for v13 vs Fan Hui, it got a lot stronger vs Lee Sedol and even stronger now. So whilst I expect the just policy network or just value network or just them together will still be weaker than plus MCTS, I expect the networks on their own are now a lot stronger than they were 18 months ago.

Did you read what I wrote? Self-play games form the rules without any human expert games as initial training.

Last year DeepMind promised that they will train a bot only using the self-play without any human players. No success so far. Why do you think it will work? And remember that you need a millions of self-played games to make some relevant database.

I'm not sure it will work, but I think it's very likely DeepMind pursued that avenue (it is very important to the more general AI goals of training systems in which you don't have a large corpus of expert examples like we do with Go) and were quite possibly successful, given Demis's response to a question about it. It's only Chinese media rumours which have been wrong before, but the recent Sina article about the rumoured upcoming match with Ke Jie mentioned about this trained from scratch aspect. Also their Atari game playing AIs did a similar thing of training from scratch and worked. Sure, training requires a lot of compute resource, but DeepMind is part of Google and has access to a lot of oomph. If training via human games used to take 3 months and now takes 1 week as Demis said, then let's say training from scratch takes 10 times as long, that's 10 weeks. To take 10 years it would need to take ~500 times longer, and for them to not make any improvements to their training algorithms. They are a clever bunch.

That might be true for the value network (not policy). But it might no longer be true. When asked about Zen's problems with komi in its value network in the WGC Hideki Kato said AlphaGo had apparently solved the komi problem somehow. I don't know if that is by retraining or something cleverer. But as Demis said training only takes a week now (probably helped by lots of TPUs), 10 years seems rather an over-estimation.

Transfer the knowledge between neural networks with different architecture is quite a challenge. No success is secured.

Yes, but they have successfully done some very challenging things few thought possible already haven't they! Before AlphaGo I'd made what I thought was a safe bet with a Chess-playing friend that no Go bot would beat a top pro in the next 5 years. How the landscape has changed in under 2 years, we now wonder if the humans can win with 2 handicaps. But even if such transfers aren't achieved, just retrain the network for 21x21: taking weeks or months, not 10 years, would be my bet. And that's for equivalent to AlphaGo top pro beating level, not average amateur (by which do you mean a high dan like yourself, or something around 4 kyu which is the (medium) average). I wouldn't be surprised if the existing AlphaGo policy network plus MCTS (no value network) with minor hacks to get it working on 21x21 would already beat a 4 kyu.

[pookpooi]

Bohdan, the free DCNN program Leela has option to let you play with 25x25 board. Can you tell us how much weaker it is compare to 19x19. You'll have no problem winning in both board size in even game since it has serious bug that prevent it to go beyond 6 dan KGS so you've to give it handicap stones.

[Bohdan]

Let us look at the different angle. The Crazy Stone bot without NN palys around 5 dan on an average laptop. It was built by only one man who was doing it mostly for fun having a lot of other stuff to do. Imagine how whole team of very skilled programers and scientists from DeepMind (read Google) could improve an algorithm from a Crazy Stone bot and multiply it by 500 CPU and 200 GPU power. I would never be surprised if optimized version of MC trees can easily beat an average pro.

Regarding neural networks training:
the training process always converges to some value. So a bot can jump from 30 kyu to 1 dan in one month. Then from 1 dan to 6 dan in 3 month. From 6 dan to 1p in 6 month. From 1p to top pro in 1 year. There always is a point when you cannot improve anymore with current algorithm. That's a basic math.
Guys from DeepMind were not the first who used self-training algorithm. It was used also in chess but at some point it just stoped improvement. For example https://www.technologyreview.com/s/541276/deep-learning-machine-teaches-itself-chess-in-72-hours-plays-at-international-master/ You can't just run a self-training and wait until it becomes a perfect player.

[Bohdan]

Bohdan, the free DCNN program Leela has option to let you play with 25x25 board. Can you tell us how much weaker it is compare to 19x19. You'll have no problem winning in both board size in even game since it has serious bug that prevent it to go beyond 6 dan KGS so you've to give it handicap stones.

Sure thing. I'll play it and upload games here. Nice challenge

[pookpooi]

Let us look at the different angle. The Crazy Stone bot without NN palys around 5 dan on an average laptop. It was built by only one man who was doing it mostly for fun having a lot of other stuff to do. Imagine how whole team of very skilled programers and scientists from DeepMind (read Google) could improve an algorithm from a Crazy Stone bot and multiply it by 500 CPU and 200 GPU power. I would never be surprised if optimized version of MC trees can easily beat an average pro.

This remind me of Mogo, French MCTS bot just like Crazy Stone. It famous around 2007-2011 when it challenge pro/top amateur players with super computer, like 512 cores of the Louhi Cray XT4/XT5 supercomputer (and still lose to Catalin Taranu 5p in 7 stones handicap), 200 node supercomputer with 4 cores per node (and lose to Myungwan Kim 8p, also in 7 stones handicap), 640 cores of the Huygens supercomputer in Amsterdam (and lose to Chun-Hsun Chou 9p and Shih Chin 2p in 7 stones handicap game)

[Bohdan]

Let us look at the different angle. The Crazy Stone bot without NN palys around 5 dan on an average laptop. It was built by only one man who was doing it mostly for fun having a lot of other stuff to do. Imagine how whole team of very skilled programers and scientists from DeepMind (read Google) could improve an algorithm from a Crazy Stone bot and multiply it by 500 CPU and 200 GPU power. I would never be surprised if optimized version of MC trees can easily beat an average pro.

This remind me of Mogo, French MCTS bot just like Crazy Stone. It famous around 2007-2011 when it challenge pro/top amateur players with super computer, like 512 cores of the Louhi Cray XT4/XT5 supercomputer (and still lose to Catalin Taranu 5p in 7 stones handicap), 200 node supercomputer with 4 cores per node (and lose to Myungwan Kim 8p, also in 7 stones handicap), 640 cores of the Huygens supercomputer in Amsterdam (and lose to Chun-Hsun Chou 9p and Shih Chin 2p in 7 stones handicap game)

Pardon me but it is nowhere near Crazy Stone. Mogo strength is kyu level.

[pookpooi]

Pardon me but it is nowhere near Crazy Stone. Mogo strength is kyu level.

At that time Crazy Stone is also Kyu level.
At 2007 Amsterdam Computer Olympiad, in 19x19 category, Mogo got Gold medal while Crazy Stone got Silver medal.

[Bohdan]

Bohdan, the free DCNN program Leela has option to let you play with 25x25 board. Can you tell us how much weaker it is compare to 19x19. You'll have no problem winning in both board size in even game since it has serious bug that prevent it to go beyond 6 dan KGS so you've to give it handicap stones.

You mean it fails to see a snapback?

[Bohdan]

Pardon me but it is nowhere near Crazy Stone. Mogo strength is kyu level.

At that time Crazy Stone is also Kyu level.
At 2007 Amsterdam Computer Olympiad, in 19x19 category, Mogo got Gold medal while Crazy Stone got Silver medal.

I see. Anyway I am sure that you can reach a pro level by only using a MC method.

[Uberdude]

Anyway I am sure that you can reach a pro level by only using a MC method.

I agree, if Zen kept on improving like it was before AlphaGo made neural nets all the rage (not that is was the first to use a policy network, only value network) then I would be fairly sure it would beat a low pro at most 5 years from back then (it was about European 5 to 6 dan in 2015: FJ Dickhut German 6d beat it 3-1) and quite possible sooner, and faster if it got more developers/processing power. But to go from that to beating Lee Sedol is quite a jump that I thought would take 10 years or more without the value network revolution.

[DrStraw]

The original poster seems to think that power consumption should be comparable for an equal match. I immediately thought of it a different way and read the entire thread look for a mention of it. No one else did, so here it is. What do you think?

Trying to compare robots with humans is like comparing apples to oranges. I was reminded of the character Data in the old Star Trek series. He was a humanoid robot, self contained although able to access some universal database. This is ultimately where computers are heading. So to me, instead of comparing power usage, how about comparing the human to a computer which is totally self contain and able to fit into a humanoid body? Could the computer sit down at a table opposite a human and, with no external power input, win the game. An external humanoid appearance would be window dressing - it is the size which counts.

Just an alternative perspective and one which many will take issue with.

[Krama]

The original poster seems to think that power consumption should be comparable for an equal match. I immediately thought of it a different way and read the entire thread look for a mention of it. No one else did, so here it is. What do you think?

Trying to compare robots with humans is like comparing apples to oranges. I was reminded of the character Data in the old Star Trek series. He was a humanoid robot, self contained although able to access some universal database. This is ultimately where computers are heading. So to me, instead of comparing power usage, how about comparing the human to a computer which is totally self contain and able to fit into a humanoid body? Could the computer sit down at a table opposite a human and, with no external power input, win the game. An external humanoid appearance would be window dressing - it is the size which counts.

Just an alternative perspective and one which many will take issue with.

Can we use batteries or is that considered an external source of energy? By that logic human player should be starved for days since our external energy source is food.

[DrStraw]

The original poster seems to think that power consumption should be comparable for an equal match. I immediately thought of it a different way and read the entire thread look for a mention of it. No one else did, so here it is. What do you think?

Trying to compare robots with humans is like comparing apples to oranges. I was reminded of the character Data in the old Star Trek series. He was a humanoid robot, self contained although able to access some universal database. This is ultimately where computers are heading. So to me, instead of comparing power usage, how about comparing the human to a computer which is totally self contain and able to fit into a humanoid body? Could the computer sit down at a table opposite a human and, with no external power input, win the game. An external humanoid appearance would be window dressing - it is the size which counts.

Just an alternative perspective and one which many will take issue with.

Can we use batteries or is that considered an external source of energy? By that logic human player should be starved for days since our external energy source is food.

You clearly did not watch Star Trek.