Does computer intelligence tell us of Go skill?

[Toge]

There are several computer bots playing on KGS go server. Some of the quite strong ones, AyaBot3 [1k] and AyaBot08 [2k] run on Monte-Carlo simulation. AyaBot3 is a strong 1k player, about 90% of the way to first dan. AyaBot08 plays at an average 2k strength, about ~45% of the way to 1k. I'm making the assumption that their architecture is similar. The reported difference is that AyaBot3 runs 5000 simulations for each move and AyaBot08 runs 1200 simulations for each move.

Can we say that number of simulations represents Go skill for bots? All other things being equal, increasing your visualization power 4-fold would grant you one rank of Go skill? Perhaps the picture is not so simple. We don't know what number of simulations means for humans.

More thoughts of human cognition in hidden section.

My personal experiences about learning Go is that there's two kinds of knowledge. One is the pruning of bad thinking that causes mistakes. For example heeding the advice that best move does not depend on where the opponent played his move has the effect of pruning a bad branch. Beginner is overwhelmed at the big board - every move for him is equally valid.

Result of first knowledge is the rigid adherence to familiar shapes and tactics. Second knowledge is the partial un-doing of pruning. Learning new tesuji and shapes increases the player's repertoire of expression.

There's also a third form of ability, exhaustive reading, which our brains really would rather not spend their energy on. It's also pretty much everything computers are capable of. Generalizations are evolutionarily important means to reduce brain energy consumption, and the methods of last two paragraphs are just ways to do so. When it is said that someone is "good at reading", does it mean that he knows accurately the goals of his reading, gets the relevant follow-ups fast and can get a conclusion without failing? Or does it mean that the brain has better short-term memory and is able to do more work at the same given time?

[snorri]

The process that a Monte Carlo engine uses for selecting moves is very different from the human process, so I don't think it's easy to make analogies. That being said, I like the idea of comparing the strength of similar engine with differences in only the human knowledge parts of the algorithm, though. For example, I once modified gnugo to have an irresistible urge to peep at every one-point jump. I assumed it would make it much weaker, but actually it didn't, at least playing against an unmodified gnugo, which I know isn't the best test. I think such comparisons might be good ways to test objectively how much bad habits really cost. We may eventually gain some different views on which skills are most important.

[speedchase]

I wouldn't think to much about it in that way. That being said, if you are going to think about skill and knowledge like that, remember that distance traveled is proportional to the square of the number of random motions, when you are moving randomly.

[Rémi]

I wouldn't think to much about it in that way. That being said, if you are going to think about skill and knowledge like that, remember that distance traveled is proportional to the square of the number of random motions, when you are moving randomly.

You mean the square root.

[pasky]

See http://pasky.or.cz/go/pachi-tr.pdf Fig. 4 (look only at the black line with circle points) for one diagram that shows Elo differences (100 Elo == one rank) in performance of the same software with just different number of playouts (simulations).

A rough approximation is that single doubling of the number of simulations should increase the strength of the program by single stone.

In practice, it's more complicated. First, this strength increase will not happen indefinitely as each program has some inherent strength limits related to systematic mishandling of various specific situations. So you will see a plateau at some point.

Second, each program has a unique mix of heuristics and some heuristics will improve strength only with small amount of situations (covering some glaring gaps that more simulations will solve easily) while other heuristics will kick in more the more simulations are available. So some programs will scare better and some will scare worse, and the differences are actually very big; designing a program that will work well on weak hardware and a program that will work well on a supercomputer are different tasks and a single program may not be good at both. (E.g. of the opensource programs, Fuego is better on older computers but Pachi would likely beat it on top-end computers.)

[speedchase]

I wouldn't think to much about it in that way. That being said, if you are going to think about skill and knowledge like that, remember that distance traveled is proportional to the square of the number of random motions, when you are moving randomly.

You mean the square root.

Yeah... got that backwards