Prediction for the upcoming tournament?

[Mike Novack]

Next week, at the European Congress. I don't know whose idea this was, but the competing programs are required to use the same cpu (i5-4590, machines being provided) and this is something rather affordable. So not only do we get to see which program strongest under this limitation, but should give us an idea of the strength of the programs on our own machines.

My guess is that aya and oakfoam will place higher than they have been doing. The machines those two have been running on in recent tournaments not much more powerful than what they will be running on in this tournament. But several of the other programs have been running on much more powerful machines.

[EdLee]

required to use the same cpu, machines being provided

[ez4u]

required to use the same cpu, machines being provided

Second that!

[DrStraw]

Isn't that a little bit like asking Einstein to use the brain of an average physics professor?

[Bantari]

Isn't that a little bit like asking Einstein to use the brain of an average physics professor?

Nope.

[Mike Novack]

No for several reasons:

1) Even for the programs using MCTS there are issues related to proper modification of the basic algorithm to optimize for the time control + crunch power situation. Ideally the designers of the programs should have that adjustable. But maybe some of then aren't readily adjustable.

For the marketers of such software, I'd call it an honesty issue (remember, some of these are free, others not). The buyers (or free users) of the software don't have machines more powerful (or not much more powerful) than the machines that will be used in this tournament. They may even have portable devices this strong. So it is of great interest how strong go can be played by bots running on that amount of hardware.

2) Not all of these programs are MCTS. For example, the oakfoam engine is a neural net. Simply doesn't have such high computational requirements. I think we all might want to see how this compares to MCTS limited to the same amount of crunch because one of the suggestions I have seen made is to have the neural net return not one but a small set of moves and then apply MCTS to this narrowed tree. The difference between the small machine used here and the most powerful machines otherwise used in these tournaments would be available for that.

3) Limiting some factor in contests is fairly common and does not prevent development of all the other factors. For example, limits on engine size, fuel used, etc. in auto racing simply means that the other factors determine who is fastest.

[Krama]

No for several reasons:

1) Even for the programs using MCTS there are issues related to proper modification of the basic algorithm to optimize for the time control + crunch power situation. Ideally the designers of the programs should have that adjustable. But maybe some of then aren't readily adjustable.

For the marketers of such software, I'd call it an honesty issue (remember, some of these are free, others not). The buyers (or free users) of the software don't have machines more powerful (or not much more powerful) than the machines that will be used in this tournament. They may even have portable devices this strong. So it is of great interest how strong go can be played by bots running on that amount of hardware.

2) Not all of these programs are MCTS. For example, the oakfoam engine is a neural net. Simply doesn't have such high computational requirements. I think we all might want to see how this compares to MCTS limited to the same amount of crunch because one of the suggestions I have seen made is to have the neural net return not one but a small set of moves and then apply MCTS to this narrowed tree. The difference between the small machine used here and the most powerful machines otherwise used in these tournaments would be available for that.

3) Limiting some factor in contests is fairly common and does not prevent development of all the other factors. For example, limits on engine size, fuel used, etc. in auto racing simply means that the other factors determine who is fastest.

Any more info on this neural network + MCTS ?

[Mike Novack]

Any more info on this neural network + MCTS ?

I've seen it mentioned but don't know if it's being worked on yet.

[ericf]

Any more info on this neural network + MCTS ?

I've seen it mentioned but don't know if it's being worked on yet.

This looked interesting:

http://leafcloud.com/research/decision- ... -learning/

Decision Forests for Computer Go Feature Learning
Author: Francois van Niekerk
Supervisor: Dr. Steve Kroon
Thesis for MSc (Computer Science) at Stellenbosch University

Abstract
In computer Go, moves are typically selected with the aid of a tree search algorithm. Monte-Carlo tree search (MCTS) is currently the dominant algorithm in computer Go. It has been shown that the inclusion of domain knowledge in MCTS is able to vastly improve the strength of MCTS engines. A successful approach to representing domain knowledge in computer Go is the use of appropriately weighted tactical features and pattern features, which are comprised of a number of hand-crafted heuristics and a collection of patterns respectively. However, tactical features are hand-crafted specifically for Go, and pattern features are Go-specific, making it unclear how they can be easily transferred to other domains.

As such, this work proposes a new approach to representing domain knowledge, decision tree features. These features evaluate a state-action pair by descending a decision tree, with queries recursively partitioning the state-action pair input space, and returning a weight corresponding to the partition element represented by the resultant leaf node. In this work, decision tree features are applied to computer Go, in order to determine their feasibility in comparison to state-of-the-art use of tactical and pattern features. In this application of decision tree features, each query in the decision tree descent path refines information about the board position surrounding a candidate move.

The results of this work showed that a feature instance with decision tree features is a feasible alternative to the state-of-the-art use of tactical and pattern features in computer Go, in terms of move prediction and playing strength, even though computer Go is a relatively well-developed research area. A move prediction rate of 35.9% was achieved with tactical and decision tree features, and they showed comparable performance to the state of the art when integrated into an MCTS engine with progressive widening.

We conclude that the decision tree feature approach shows potential as a method for automatically extracting domain knowledge in new domains. These features can be used to evaluate state-action pairs for guiding search-based techniques, such as MCTS, or for action-prediction tasks.

[quantumf]

I chatted to Francois yesterday to get some insight into this. He confirmed that neural nets have been added to Oakfoam, but that he's not actively working on it anymore. The work is being done by Detlef Schmicker, who has forked Oakfoam. You can see his fork here: https://bitbucket.org/dsmic/oakfoam

Oakfoam with neural nets played in the recent computer olympiad (coming last): http://www.grappa.univ-lille3.fr/icga/t ... php?id=290

[Mike Novack]

Only some of the usual programs took part. My guess here is that with the constraint of having to play on a machine much weaker than they usually played on, the teams for some of the programs didn't think they could adjust correctly to compensate for that, didn't want a poor result for that reason. It was not a surprise to me that Aya, which normally runs on a machine of approximately this power, did comparatively better than usual.

EGC2015 Computer Go Tournament
Players Position Name # Results Score Notes1234

1 AyaMC [-] X 1/1 1/1 1/1 3 Winner
2 ManyFaces1 [-] 0/1 X 1/1 1/1 2
3 pachi [2d] 0/1 0/1 X 1/1 1
4 jimmy [-] 0/1 0/1 0/1 X 0

So it was Aya that got to play the human

2015-07-29 Hajin Lee 3p vs AyaMC 19×19 Five stones 0-1 2015 European Go Congress
Played in Liberec, Czech Republic. The time-setting was 60 minutes each plus byoyomi. Japanese rules were used.


OOPS -- apparently I don't understand how to get an sgf file in here properly. Can somebody help with that so we can show the game.

[Mike Novack]

[ez4u]

[Mike Novack]

Thanks, though I would appreciate it if somebody explained what I was doing wrong.

Meanwhile, maybe somebody would put up the tournament game between AyaMC and ManyFaces1. Apparently a half point squeaker.

[oren]

Thanks, though I would appreciate it if somebody explained what I was doing wrong.

Hit "quote" on ez4u's post and you'll see.