Are we humans or are we bots?

[Solomon]

So I hear a lot of people claim that bots play "differently" than humans, that they have a different style and just play like bots. But I wonder if such people can tell, so a small exercise to them and those interested in participating:

Below I've posted three games, non-blitz and all dan-level. The question is, can you tell how many of the players are bots and which ones are the bots? Games could be human vs. human, human vs. bot, or bot vs. bot. Should be easy right?

[billywoods]

My guess is going to be:

1. Black bot, white human
2. Black human, white human
3. Black bot, white bot

[badukJr]

I think when people talk about how 'bots' play differently, its that they have very specific flaws in their playing styles that can be exploited repeatedly until the author can update the code. This of course requires repeated playing that you won't see in this test.

You won't be able to derive any conclusions here then. A better test is to show a 10 or 20 game series and ask which player is bot or human.

[Ellyster]

I was going to say "All bots"... but in second thoughts:

1. Black bot, White bot
2. Black bot, White bot
3. Black human, White human

My twisted mind tells me that are the oposite that it looks

[Joaz Banbeck]

I only have two guesses about the six

1. Black = bot
2. White = human

This is a great thread.

[snorri]

I don't think it's that easy if you've actually included some bots here.

Game 1

Black is not a bot. Moves are are too fashionable. For the same reason, I think white indicates a non-bot player. No further analysis.

Game 2

The high enclosure strikes me as a very human thing to do. It smells of ego. I think maybe white is not a bot. Also, shows jealously, a very non-bot trait. The sequence to is odd, but I'm not sure I'd attribute it to irrationality? I will assume so, therefore, not a bot.

Game 3

Some funny tenukis lead me to suspect that both players are bots.

[gogameguru]

Haha, interesting idea

I think one factor that's still different is consistency (at whatever level). Even strong bots still seem to play a bit phrenetically.

It's been awhile since I've watched bots play though, so let's see if I'm right...

My guesses:

Game 1: Both players are human. There seems to be too much creativity and interplay between both players. As far as I know bots still don't play like that.

Game 2: Black seems like a bot, I think white's a human.

Game 3: White's a bot (or someone who's high). I'm going to go out on a limb here and guess that black is Sol pretending to be a bot (bonus points maybe?).

[hyperpape]

Answers:

The only one that says anything to me is White in the third game. This is hard.

Question:

How did you choose these?

[yoyoma]

1. Black=bot, White=human
2. No idea
3. Both bots.

[jts]

1. I'm confident white's not a bot... I don't think even bad bots screw up and then immediately peep the way this W does. B probably isn't a bot. There are a few moves in the UL that seem weird, but it's probably just a joseki I don't know.

2. I'm not confident here. I plead not enough data. W12 is a move that bots are much more likely to play than humans, but W smacked around B so badly that there's nothing stylistically notable for either side.

3. White is a really bad human - there are "following B around" moves that I wouldn't expect from a computer, in addition to whimsical, meaningless moves that remind me of gnugo. I'm fairly confident B is a human too; B has laser-vision when it comes to following through on threats he has made.

[tapir]

I agree with gogameguru.

What stands out to me as particularly bot-like moves are 33 in game 2 - Black could draw back and enclose White instead of letting the White group escape and move 34 in game 3 - tenuki from the unsettled lower right corner. Especially the second would not happen to any dan level player who played several moves with the intent to make life for the group to then suddenly tenuki with the group still unsettled. Also, White in 2nd and Black in 3rd game play as if they were confident to outread their opponents.

[Mike Novack]

Please note.

We were told "dan level" so that means MCTS based programs not more than a couple years old. Older experience with how bots play is not going to be relevant.

Don't expect fixed patterns of play from the bots. MCTS doesn't work that way.

Don't rule out moves because particularly "fashionable" or diabolically clever. Only those programs using a go knowledge AI to prepare a plausible move set going to be limited in this way. Straightforward MCTS can find any "best move".

Do expect some "style" consistencies though these not might help much determining bot or human but then again might under certain circumstances. I haven't yet met a modern bot that played a "territorially tight" style so if you see that, guess human.

[hyperpape]

Mike

Hidden, because I don't want to bias other players. The point about fashionable moves is actually accurate. You're right that what you don't have to worry about is an opening/joseki book that rules out fashionable plays. However, a MCTS bot is no more likely to play a move that is fashionable today than a move that was fashionable 100 years ago, unlike human players who follow trends. And of course the bot is more likely to play a move that was never fashionable. So it is a potential indicator.

[Bill Spight]

Humans or bots?

The two are not mutually contradictory.

[SmoothOper]

1) Both are human 2) white is a bot 3) white is a bot

It is difficult to tell without knowing which bots were selected since I suspect that they can have styles as varied as the humans that code them, for example Monte Carlo vs. Many Faces or GNU go.

One of the signs that I would expect of a bot is to do very well in life and death situations such as white in 2 where a search would yield good results.

The other is to have a fuseki with many tenuki and and a conventional handicap fuseki as in three.

If the bots were trained on databases of prior professional games then they could be indistinguishable from the pros by me.