AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

Full paper at https://arxiv.org/abs/1712.01815







AlphaZero hasn't lose against Stockfish but is it because of so many draws that the Elo in the graph is very close
Meanwhile, AlphaZero training just 34 hours beats 60 out of 100 games against AlphaGo Zero 20 blocks 3 days that beat AlphaGo Lee 100 out of 100 games. But look like they're not gonna train for 40 days, not anymore.

Pookpooi, thanks.

Congrats to Deep Mind.

The paper includes 10 sample games for chess (v. Stockfish),
but none for shogi (v. Elmo) or Go (v. AGZ).

Elmo, maybe the final old-school (non-NN) shogi program,
still managed to beat AZ in 8% of the games.
(Re: game 4, AGL v. LSD)

Page 15: Pondering was disabled for all players. --
Does pondering mean thinking on the opponent's time ?

Ugh... the chess set is buried in 70+ storage boxes...

I've been quite interested in machine learning, and artificial intelligence in general, since college. But given the rate at which artificial intelligence is advancing, I have an odd combination of feelings. Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

Maybe it's something else - like emotion or the human experience... Or maybe the bots will tell us someday

_________________
be immersed

If we're so lucky.

Yes, at least in this paper's operational definition.


I'm pretty sure my cat will choose to be born as a cat again in the next life even though it knows that human are smarter than it.

Impressive again from DeepMind! At one of his talks I got the impression from Demis he wasn't so interested in making a chess program as basically a solved problem already (and maybe a bit of a cop out in case theirs wasn't as strong as stockfish et al), but they went and did it anyway!

I have a few questions for the chess experts here:
- looking at the chess openings is AlphaZero playing the long standard opening book lines or has it found a way to diverge early without playing bad moves? My impression of chess was human knowledge of the opening was closer to perfect play than in go and is sharper so there was less scope for novelty into unexplored areas without playing suboptimal moves.
- Is the play of stockfish near its peak strength, i.e if it has more time or resources does it get significantly better (anyone try at home) and not play the moves that let AZ beat it? I wonder if perhaps neural networks bots are better at blitz than tree search bots (in training you essentially transfer the skill from tree search into one huge function which is quick to compute).*

Edit: Now I read the paper the 100 game match was not 1 seconds a move like for the Elo evaluation in the graph, but 1 minute a move with 64 threads and 1 GB hash which sounds better but still I'm not clear how far from peak strength and diminishing returns that is (and could be a lot smaller than the 4 TPUs AZ got). Looking at the kibitz on the TCEC match many chess players are dismissive of the conditions, saying the specs for stockfish engine are unfair/small.

This may be easier to browse: https://lichess.org/study/EOddRjJ8

(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )

Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.

Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ?

Man, give them an inch...!

Yeah, DeepMind should publish some of the shogi games too. It would be interesting to see them, or rather read what other people think of them (seeing as I don't know much about shogi myself).


Nah, I just found the link in /u/Spreek's comment, as mentioned. I assume they're responsible, unless they too got the link from someone else.

I also thought Spreek's comment and the replies to it in the /r/chess discussion were interesting: it sounds like they think AlphaZero's moves feel different than previous computer chess, possibly more "natural".

EDIT: also found another viewer for the chess games via twitter: https://chess24.com/watch/live-tourname ... -stockfish

I'm wondering why DeepMind choose Shogi out of many board games beside Go (chess is understandable) and finally found some clues



Yoshiharu Habu is the only person ever to hold seven Shogi titles simultaneously in 1996, even right now he's still holding two of it, and the only person ever to has all seven lifetime titles. He's also Chess Master.



He also got to play with AlphaGo! But I'm not sure how strong he is in Go.

from http://www.nhk.or.jp/special/ai/article1.html

Another usual suspect: Xiangqi ; ...Chaturanga.

For diplomatic reason, choosing Xianqi to conquer by (non-Chinese) AI is a really bad choice.

Stockfish won the 2016 TCEC, but this year the match is currently in progress between the other 2 big names in computer chess, Houdini and Komodo (interestingly Stockfish was better against Komodo head-to-head in qualifying but didn't do so well beating up the weaker half of the 8-player field, not enough "contempt" in the lingo, I suppose this means it reads for both players so will play a draw line if it sees against itself that's the best it could do, but maybe it could overplay and go for a win and get away with it against weaker opposition, I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win). Here's an interesting interview with the creators of Houdini and Komodo, which includes GM Larry Kaufman who is also a strong shogi and decent Go player who wrote an interesting comparison of the games I've posted before (he thinks shogi is the best game).

http://www.chessdom.com/interview-with- ... y-kaufman/

Some famous last words on November 27th


5 years turned out to be 12 days

Nice if someone finds the clip when Letterman asked Kasparov, (paraphrasing) "Do you think the chess computer will ever beat the best humans?" and Kasparov said no. (This might be shortly after he'd beaten Deep Thought. ...Early 1990's?)

Famous last words, indeed.

Last word on Shogi world

Late last year, a film about shogi player Satoshi Murayama called Satoshi no Seishun hit Japanese cinemas. It shows Murayama, who died at 29 in 1997, delivering one of his most famous proclamations: “The day will never come when a computer defeats a pro shogi player.”

https://qz.com/906447/artificial-intell ... ese-chess/

no opening book, no endgame database, no heuristics, no nothing! full paper coming soon, will have things like early games.

https://twitter.com/demishassabis/statu ... 4462542849

So this is not the full paper yet? Maybe they'll come with the usual DeepMind blog post

This would be hard unless you had multiple games to go on (and no chess engines that I know of have a "meta-game" like that), since by the problem setup your opponent's moves so far in this game have been about as good as your own.

It continues to surprise me how every field thinks that their problem will be too difficult for AI to have good performance right up until it does. Go players were no different before AlphaGo, but I would have thought chess’s relationship with strong computer programs would have changed their outlook.

I'll try not to derail this thread into a philosophical discussion, so I'll keep my thoughts to this post (if you want to discuss further, feel free to PM me Kirby!):

All the artificial intelligence so far developed has intelligence that is directed. Human beings have the capacity to reflect on our aims and revise them in a very general way - this is evidenced both in the wide diversity of things human beings devote their lives to (not just careers or hobbies, look also at cultural values and practices) as well as the extremes to which humans will go to accomplish those (even to the point of death, consider martyrs of various stripes, such as Buddhist monks self-immolating in protest).

Maybe one day AI will develop this generalized reflective capacity, and begin making choices about whether it will devote its intelligence to mastering Go, or Shogi, or medical diagnosis, or what-have-you. But I give it as just one example of the rich array of cognitive abilities that human beings have that machines are nowhere near approaching.

That said, I think your other suggestions are important. I think tying the meaning of life to special cognitive capacities has its own pitfalls. Pookpoi mentioned cats, but we only have to look as far as humans with cognitive disabilities. An often overlooked implication of rooting the meaning of human life or the moral status of humans in our special intellectual capacities is that human beings lacking them have less meaningful (or non-meaningful) lives or do not have the moral status of unimpaired human beings. This is an implication that I certainly find problematic, and so it tends to undermine the idea that we should base our understanding of these concepts in the special intellectual capacities of human beings.