Databases again

[Pippen]

Obviously deep learning needs deep databases of good quality games (better than 5dKGS/8dTygem I guess). Anyone a clou where to find those or where those programmers get them? I could need a database with 1.000.000 quality games .

[hyperpape]

I was about to say it's not obvious, since Google used one version of the neural net to play games which were then fed back into their system for reinforcement learning.

But if there was an open database of sufficient size, you could avoid the step of recreating games while training your system. Does anyone know how much of the resources that accounts for?

[djhbrown]

deep learning does not need deep databases because if you already have a deep database you don't need to deep learn

[Pio2001]

In the paper published by Deepmind in Nature, they say

We trained the policy network (...) to classify positions according to expert moves played in the KGS data set. This data set contains 29.4 million positions from 160,000 games played by KGS 6 to 9 dan human players; 35.4% of the games are handicap games.

In the press conference after match 5, they said that Demis thought that AlphaGo could learn the game by itself, but they all the same started the deep learning with a database of human games.

I think that it would be interesting to see what fuseki would AlphaGo play if it had not seen any human game at all. But I'm afraid that it would miss some important concepts, like a human repeating the same mistakes against a weak software because the mistakes are never punished.
As they said in the interviews, neural networks have "blind spots". The more they are trained, the smaller the blind spots.

[Tryss]

but since pros are only human (cogito ergo err) they make mistakes, and a database full of mistakes is no use to man or beast.

You don't need perfect play for a database to be usefull... I even think that a database of games devoid of of mistakes would be less usefull


By the way, your joseki database is probably full of mistakes (the concept of joseki, is, in the absolute, a mistake. But in practice, at human level, it's verery usefull)

[djhbrown]

the concept of joseki, is, in the absolute, a mistake.

[DrStraw]

the concept of joseki, is, in the absolute, a mistake.

oh, really? fair play is a mistake, eh? ...

Why do you say that? Fair play is a mistake to any political party in any country in the world. If it weren't they would never get elected.