It is currently Thu Mar 28, 2024 8:05 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 82 posts ]  Go to page 1, 2, 3, 4, 5  Next
Author Message
Offline
 Post subject: AlphaZero paper discussion (Mastering Go, Chess, and Shogi)
Post #1 Posted: Tue Dec 05, 2017 9:28 pm 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

Full paper at https://arxiv.org/abs/1712.01815

Image

Image

Image

AlphaZero hasn't lose against Stockfish but is it because of so many draws that the Elo in the graph is very close
Meanwhile, AlphaZero training just 34 hours beats 60 out of 100 games against AlphaGo Zero 20 blocks 3 days that beat AlphaGo Lee 100 out of 100 games. But look like they're not gonna train for 40 days, not anymore.


Last edited by pookpooi on Wed Dec 06, 2017 8:10 am, edited 2 times in total.

This post by pookpooi was liked by 3 people: Bonobo, EdLee, Waylon
Top
 Profile  
 
Offline
 Post subject:
Post #2 Posted: Tue Dec 05, 2017 9:55 pm 
Honinbo
User avatar

Posts: 8859
Location: Santa Barbara, CA
Liked others: 349
Was liked: 2076
GD Posts: 312
Pookpooi, thanks.

Congrats to Deep Mind.

The paper includes 10 sample games for chess (v. Stockfish),
but none for shogi (v. Elmo) or Go (v. AGZ). :)

Elmo, maybe the final old-school (non-NN) shogi program,
still managed to beat AZ in 8% of the games.
(Re: game 4, AGL v. LSD)

Page 15: Pondering was disabled for all players. --
Does pondering mean thinking on the opponent's time ?

Ugh... the chess set is buried in 70+ storage boxes... :oops:

Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #3 Posted: Tue Dec 05, 2017 10:23 pm 
Honinbo

Posts: 9545
Liked others: 1600
Was liked: 1711
KGS: Kirby
Tygem: 커비라고해
I've been quite interested in machine learning, and artificial intelligence in general, since college. But given the rate at which artificial intelligence is advancing, I have an odd combination of feelings. Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

Maybe it's something else - like emotion or the human experience... Or maybe the bots will tell us someday :grumpy:

_________________
be immersed

Top
 Profile  
 
Offline
 Post subject:
Post #4 Posted: Tue Dec 05, 2017 10:31 pm 
Honinbo
User avatar

Posts: 8859
Location: Santa Barbara, CA
Liked others: 349
Was liked: 2076
GD Posts: 312
Quote:
maybe the bots will tell us someday
If we're so lucky.
Attachment:
T.jpg
T.jpg [ 31.48 KiB | Viewed 18012 times ]

Top
 Profile  
 
Offline
 Post subject: Re:
Post #5 Posted: Tue Dec 05, 2017 10:37 pm 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
EdLee wrote:
Does pondering mean thinking on the opponent's time ?

Yes, at least in this paper's operational definition.

Kirby wrote:
Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

I'm pretty sure my cat will choose to be born as a cat again in the next life even though it knows that human are smarter than it.

Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #6 Posted: Wed Dec 06, 2017 12:20 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Impressive again from DeepMind! At one of his talks I got the impression from Demis he wasn't so interested in making a chess program as basically a solved problem already (and maybe a bit of a cop out in case theirs wasn't as strong as stockfish et al), but they went and did it anyway!

I have a few questions for the chess experts here:
- looking at the chess openings is AlphaZero playing the long standard opening book lines or has it found a way to diverge early without playing bad moves? My impression of chess was human knowledge of the opening was closer to perfect play than in go and is sharper so there was less scope for novelty into unexplored areas without playing suboptimal moves.
- Is the play of stockfish near its peak strength, i.e if it has more time or resources does it get significantly better (anyone try at home) and not play the moves that let AZ beat it? I wonder if perhaps neural networks bots are better at blitz than tree search bots (in training you essentially transfer the skill from tree search into one huge function which is quick to compute).*

Edit: Now I read the paper the 100 game match was not 1 seconds a move like for the Elo evaluation in the graph, but 1 minute a move with 64 threads and 1 GB hash which sounds better but still I'm not clear how far from peak strength and diminishing returns that is (and could be a lot smaller than the 4 TPUs AZ got). Looking at the kibitz on the TCEC match many chess players are dismissive of the conditions, saying the specs for stockfish engine are unfair/small.

Top
 Profile  
 
Offline
 Post subject: Re:
Post #7 Posted: Wed Dec 06, 2017 12:38 am 
Beginner
User avatar

Posts: 7
Liked others: 0
Was liked: 7
EdLee wrote:
Ugh... the chess set is buried in 70+ storage boxes... :oops:

This may be easier to browse: https://lichess.org/study/EOddRjJ8

(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )

Top
 Profile  
 
Offline
 Post subject: Re: Re:
Post #8 Posted: Wed Dec 06, 2017 12:50 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
mistakenot wrote:

Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.

Top
 Profile  
 
Offline
 Post subject:
Post #9 Posted: Wed Dec 06, 2017 1:20 am 
Honinbo
User avatar

Posts: 8859
Location: Santa Barbara, CA
Liked others: 349
Was liked: 2076
GD Posts: 312
Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ? :)

Quote:
they're not impressed...
Man, give them an inch...! :-?

Top
 Profile  
 
Offline
 Post subject:
Post #10 Posted: Wed Dec 06, 2017 2:08 am 
Beginner
User avatar

Posts: 7
Liked others: 0
Was liked: 7
pookpooi wrote:
mistakenot wrote:

Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.

Yeah, DeepMind should publish some of the shogi games too. It would be interesting to see them, or rather read what other people think of them (seeing as I don't know much about shogi myself).

EdLee wrote:
Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ? :)

Nah, I just found the link in /u/Spreek's comment, as mentioned. I assume they're responsible, unless they too got the link from someone else.

I also thought Spreek's comment and the replies to it in the /r/chess discussion were interesting: it sounds like they think AlphaZero's moves feel different than previous computer chess, possibly more "natural".

EDIT: also found another viewer for the chess games via twitter: https://chess24.com/watch/live-tourname ... -stockfish


Last edited by mistakenot on Wed Dec 06, 2017 2:28 am, edited 3 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #11 Posted: Wed Dec 06, 2017 2:14 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
I'm wondering why DeepMind choose Shogi out of many board games beside Go (chess is understandable) and finally found some clues

Image

Yoshiharu Habu is the only person ever to hold seven Shogi titles simultaneously in 1996, even right now he's still holding two of it, and the only person ever to has all seven lifetime titles. He's also Chess Master.

Image

He also got to play with AlphaGo! But I'm not sure how strong he is in Go.

from http://www.nhk.or.jp/special/ai/article1.html


Last edited by pookpooi on Wed Dec 06, 2017 2:28 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject:
Post #12 Posted: Wed Dec 06, 2017 2:20 am 
Honinbo
User avatar

Posts: 8859
Location: Santa Barbara, CA
Liked others: 349
Was liked: 2076
GD Posts: 312
Quote:
I'm wondering why DeepMind choose Shogi out of many board games beside Go
Another usual suspect: Xiangqi ; ...Chaturanga.

Top
 Profile  
 
Offline
 Post subject: Re:
Post #13 Posted: Wed Dec 06, 2017 2:22 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
EdLee wrote:
Quote:
I'm wondering why DeepMind choose Shogi out of many board games beside Go
Another usual suspect: Xiangqi ; ...Chaturanga.

For diplomatic reason, choosing Xianqi to conquer by (non-Chinese) AI is a really bad choice.


This post by pookpooi was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #14 Posted: Wed Dec 06, 2017 3:00 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Stockfish won the 2016 TCEC, but this year the match is currently in progress between the other 2 big names in computer chess, Houdini and Komodo (interestingly Stockfish was better against Komodo head-to-head in qualifying but didn't do so well beating up the weaker half of the 8-player field, not enough "contempt" in the lingo, I suppose this means it reads for both players so will play a draw line if it sees against itself that's the best it could do, but maybe it could overplay and go for a win and get away with it against weaker opposition, I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win). Here's an interesting interview with the creators of Houdini and Komodo, which includes GM Larry Kaufman who is also a strong shogi and decent Go player who wrote an interesting comparison of the games I've posted before (he thinks shogi is the best game).

http://www.chessdom.com/interview-with- ... y-kaufman/

Some famous last words on November 27th

Quote:
Robert (Houdini developer): Well, I think we are all waiting for artificial intelligence to pop up in chess after having seen the success of the artificial intelligence approach of Google for the Go game. And so basically what I would expect if some of these giant corporations would be interested is that in the next five years chess also might see that kind of development. For example the artificial intelligence for the evaluation of a position, it could produce some very surprising results in chess. And so, we’re probably waiting for that and then we can retire our old engines. Look at the AlphaChess engine that will be 4000 Elo. [chuckles]

Nelson (moderator): Yep, at that point we can all fade back into history. Larry, anything to add?

Larry (GM and Komodo developer): Well, I also followed closely the AlphaGo situation. The guy who is the head of it at Google Mind is a chess master himself, Demis Hassabis. Although Go is thought to be a much harder game than chess to beat the best humans at, and they have certainly proven that they can do that, it is so far yet to be proven that a learning program such as the latest one from DeepMind [can replicate that in chess]. Their latest learning program beat the pants off all other, previous Go programs. But that does not apply to chess. Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy. Maybe that’s the challenge, to get Google to prove that it applies to chess too. But who knows.

5 years turned out to be 12 days :D


This post by Uberdude was liked by 2 people: Bonobo, mumps
Top
 Profile  
 
Offline
 Post subject:
Post #15 Posted: Wed Dec 06, 2017 3:42 am 
Honinbo
User avatar

Posts: 8859
Location: Santa Barbara, CA
Liked others: 349
Was liked: 2076
GD Posts: 312
Quote:
Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy.
Nice if someone finds the clip when Letterman asked Kasparov, (paraphrasing) "Do you think the chess computer will ever beat the best humans?" and Kasparov said no. (This might be shortly after he'd beaten Deep Thought. ...Early 1990's?)

Famous last words, indeed.

Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #16 Posted: Wed Dec 06, 2017 3:53 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
Last word on Shogi world

Late last year, a film about shogi player Satoshi Murayama called Satoshi no Seishun hit Japanese cinemas. It shows Murayama, who died at 29 in 1997, delivering one of his most famous proclamations: “The day will never come when a computer defeats a pro shogi player.”

https://qz.com/906447/artificial-intell ... ese-chess/


This post by pookpooi was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #17 Posted: Wed Dec 06, 2017 4:26 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
no opening book, no endgame database, no heuristics, no nothing! full paper coming soon, will have things like early games.

https://twitter.com/demishassabis/statu ... 4462542849

So this is not the full paper yet? Maybe they'll come with the usual DeepMind blog post

Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #18 Posted: Wed Dec 06, 2017 7:53 am 
Gosei

Posts: 1590
Liked others: 886
Was liked: 527
Rank: AGA 3k Fox 3d
GD Posts: 61
KGS: dfan
Uberdude wrote:
[re contempt] I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win.

This would be hard unless you had multiple games to go on (and no chess engines that I know of have a "meta-game" like that), since by the problem setup your opponent's moves so far in this game have been about as good as your own.

Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #19 Posted: Wed Dec 06, 2017 8:10 am 
Lives in sente

Posts: 902
Location: Fort Collins, CO
Liked others: 319
Was liked: 287
Rank: AGA 3k
Universal go server handle: jeromie
Uberdude wrote:
Stockfish won the 2016 TCEC, but this year the match is currently in progress between the other 2 big names in computer chess, Houdini and Komodo (interestingly Stockfish was better against Komodo head-to-head in qualifying but didn't do so well beating up the weaker half of the 8-player field, not enough "contempt" in the lingo, I suppose this means it reads for both players so will play a draw line if it sees against itself that's the best it could do, but maybe it could overplay and go for a win and get away with it against weaker opposition, I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win). Here's an interesting interview with the creators of Houdini and Komodo, which includes GM Larry Kaufman who is also a strong shogi and decent Go player who wrote an interesting comparison of the games I've posted before (he thinks shogi is the best game).

http://www.chessdom.com/interview-with- ... y-kaufman/

Some famous last words on November 27th

Quote:
Robert (Houdini developer): Well, I think we are all waiting for artificial intelligence to pop up in chess after having seen the success of the artificial intelligence approach of Google for the Go game. And so basically what I would expect if some of these giant corporations would be interested is that in the next five years chess also might see that kind of development. For example the artificial intelligence for the evaluation of a position, it could produce some very surprising results in chess. And so, we’re probably waiting for that and then we can retire our old engines. Look at the AlphaChess engine that will be 4000 Elo. [chuckles]

Nelson (moderator): Yep, at that point we can all fade back into history. Larry, anything to add?

Larry (GM and Komodo developer): Well, I also followed closely the AlphaGo situation. The guy who is the head of it at Google Mind is a chess master himself, Demis Hassabis. Although Go is thought to be a much harder game than chess to beat the best humans at, and they have certainly proven that they can do that, it is so far yet to be proven that a learning program such as the latest one from DeepMind [can replicate that in chess]. Their latest learning program beat the pants off all other, previous Go programs. But that does not apply to chess. Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy. Maybe that’s the challenge, to get Google to prove that it applies to chess too. But who knows.

5 years turned out to be 12 days :D


It continues to surprise me how every field thinks that their problem will be too difficult for AI to have good performance right up until it does. Go players were no different before AlphaGo, but I would have thought chess’s relationship with strong computer programs would have changed their outlook.

Top
 Profile  
 
Offline
 Post subject: Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Post #20 Posted: Wed Dec 06, 2017 10:12 am 
Lives in gote
User avatar

Posts: 388
Location: Riverside CA
Liked others: 246
Was liked: 79
Rank: KGS 7 kyu
KGS: Krill
OGS: Krill
Kirby wrote:
I've been quite interested in machine learning, and artificial intelligence in general, since college. But given the rate at which artificial intelligence is advancing, I have an odd combination of feelings. Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

Maybe it's something else - like emotion or the human experience... Or maybe the bots will tell us someday :grumpy:


I'll try not to derail this thread into a philosophical discussion, so I'll keep my thoughts to this post (if you want to discuss further, feel free to PM me Kirby!):

All the artificial intelligence so far developed has intelligence that is directed. Human beings have the capacity to reflect on our aims and revise them in a very general way - this is evidenced both in the wide diversity of things human beings devote their lives to (not just careers or hobbies, look also at cultural values and practices) as well as the extremes to which humans will go to accomplish those (even to the point of death, consider martyrs of various stripes, such as Buddhist monks self-immolating in protest).

Maybe one day AI will develop this generalized reflective capacity, and begin making choices about whether it will devote its intelligence to mastering Go, or Shogi, or medical diagnosis, or what-have-you. But I give it as just one example of the rich array of cognitive abilities that human beings have that machines are nowhere near approaching.

That said, I think your other suggestions are important. I think tying the meaning of life to special cognitive capacities has its own pitfalls. Pookpoi mentioned cats, but we only have to look as far as humans with cognitive disabilities. An often overlooked implication of rooting the meaning of human life or the moral status of humans in our special intellectual capacities is that human beings lacking them have less meaningful (or non-meaningful) lives or do not have the moral status of unimpaired human beings. This is an implication that I certainly find problematic, and so it tends to undermine the idea that we should base our understanding of these concepts in the special intellectual capacities of human beings.


This post by Monadology was liked by: Kirby
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 82 posts ]  Go to page 1, 2, 3, 4, 5  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group