On handling online cheating with AI

[HermanHiddema]

Using Zoom doesn't guarantee that an AI doesn't show moves on the screen (or on another screen).

Of course it doesn't. It's an aid. You can use it watch a player's eyes move, and so on. Humans are incredibly sensitive to anything that happens on the face.

It's not perfect but it's practical and free and it's here now.

And other one point the numbers guys keep overlooking is that a mathematical modelling solution, apart from being also imperfect while being impractical, expensive and not available yet, is that for people who don't understand the maths or statistics, the imposition of such a system creates an uncomfortable atmosphere - almost an invasion of privacy. It's a kind of Google/Facebook/Twitter approach to life. You have to trust something you don't really understand. Of course you have to do that sometimes in life - but for a mere board game? It's an anti-social solution to a social problem.

On the other hand, I must say I wouldn't always be comfortable with the human approach. Sure, humans are very sensitive to anything that happens on the face, but they are also consciously and subconsciously biased in a million subtle ways. People have their prejudices. People will reach conclusions, in part, based on race, gender, nationality, religion and appearance. One of the advantages of a technical solution based purely on the analysis of the moves is that players can trust that it has no such bias. That they are not more likely to be accused of cheating based on the colour of their skin.

Some may not trust computer algorithms they do not understand. Others may not trust the fallible human referee, whose judgement might also be impaired by being tired, tipsy or annoyed. Technological solutions may not be the be-all-end-all of preventing and detecting cheating, but they certainly have their place.

[bogiesan]

If I were playing my favorite partner online and he suddenly made two or three or four direct 3-3s, any of those weirdly unsupported mid-region invasions, or those funny-looking attachments on the third line I’d know three things: 1) he had an AI open on a separate device; 2) he was in way over his head; and 3) I would be resigning in a few dozen more moves.

What would differentiate me knowing he had the AI open instead of having grown five stones stronger? The follow-up moves would be too good, too precise, too inexplicably out of character.

[Bill Spight]

People have their prejudices. People will reach conclusions, in part, based on race, gender, nationality, religion and appearance. One of the advantages of a technical solution based purely on the analysis of the moves is that players can trust that it has no such bias. That they are not more likely to be accused of cheating based on the colour of their skin.

Or because they are Eurasian teenage males?

In the real world, technical algorithms have been found to enshrine prejudices. See the book, Weapons of Math Destruction, for more. The moves do not reveal cheating or not cheating. You have to have examples of cheating and non-cheating to compare. Or, given cheating by bot, comparable examples of bot vs. human play. I actually provided a few such examples, controlling for the quality of the plays. For that I have been roundly criticized.

In any event, technical solutions have to be validated, and that is not easy to do. There are many more examples of suspected cheating than of proven cheating. You can't use the examples of suspected cheating to validate cheating detection algorithms. That would enshrine current preconceptions and prejudices.

Technological solutions may not be the be-all-end-all of preventing and detecting cheating, but they certainly have their place.

True. But John's suggestion has merit, as well.

[Bill Spight]

If I were playing my favorite partner online and he suddenly made two or three or four direct 3-3s, any of those weirdly unsupported mid-region invasions, or those funny-looking attachments on the third line I’d know three things: 1) he had an AI open on a separate device; 2) he was in way over his head; and 3) I would be resigning in a few dozen more moves.

Or he watched lectures and reviews in the Yunguseng Dojang, where Hwang In-seong 8d talks about these things at length and encourages his students to try these moves in their games.

This illustrates something that I think Polama hinted at before.

Of course, we'll need to resample those players every couple years as our understanding of Go improves...

I suspect that within 10-15 years the median level of amateur go will have increased by one stone, maybe two. The reason is that we have learned and will continue to learn from bots. Which also means that we will play more like bots. We already see the effect in joseki and openings. It used to be that beginners played early 3-3 invasions, but now everybody does it. We also play those funny attachments. And even if we don't understand bot plays, we can imitate them, and learn from experience. The difference between human play and bot play is decreasing.

[SoDesuNe]

Since this thread has now seven pages and at best a hint of an idea of a solution... maybe we should start anew and align our conception of what the actual problem is?

My understanding is, the main and for the moment maybe only relevant problem is cheating while playing go online.

As others have pointed out (and Antti even wrote at length about it), playing against a bot or bot enhanced player is not fun. The most obvious reason being: Competition-wise all your efforts will be in vain because you can never win.

If cheating while playing online takes away the fun for (legit) players, every solution should be benchmarked against providing (at least the usual - pre bot -) fun for the majority of players.

But to go back one step: Even if you'll lose because your opponent used a bot, it only kills the fun when you think bot usage is likely in play (and also happens somewhat frequently, I guess). So at least in the beginning the goal should be to get rid of obvious cheaters. The ones the admins can (easily) discern themselves but which takes a lot of time and energy when doing in manually.

Possible indicators: Frequently over-performing. Fast and seemingly unstoppable rank-climbing. Highly unstable results (rank graph like a sine curve). Maybe the frequent creation of new accounts, when the old ones plowed through a certain rank range. See Adin's first post, too.

You don't need a highly complex algorithm to search the player base for these indicators. Decisions can still be made by the admins. Of course there still should be a strike-system and a courtesy-note to the suspected player informing her or him about possibilities to appeal the decision.

On the highest level the problem can even be (partially) solved by simple means. Look at chess and start using titles or indications for strong players who provided ID and can show their documented strength (EGF, pro dan, AGA database) - could also produce a nice cashflow for the federations... Of course the ID won't be made public without the players consent but other strong players will know that there is an actual strong person on the other side. In online chess you rarely have a high elo player without a title and to my knowledge there is no (relevant) problem of cheating in high level games and there are quite a few and regular online tournaments with cash prizes.

[Bill Spight]

As others have pointed out (and Antti even wrote at length about it), playing against a bot or bot enhanced player is not fun. The most obvious reason being: Competition-wise all your efforts will be in vain because you can never win.

Polama's idea addresses that problem.

If people are complaining that the 9-dan ranks are all bots, that's a perception of a widespread problem. But I think Spinal Tap can point us to a helpful counter-measure: make it go to 11! Bots are better than us at Go. Let the cheaters raise up to a rank above us humans and play amongst themselves.

So if cheating 9 dans are always defeating honest human 9 dans, promote them to 11 dan, or even higher. Let them play amongst themselves, if that's what they would like to do. (Probably not, given the motivation for such cheating.) If, then, the cheaters start losing games in order to keep their ratings down, then we have a type of familiar problem, that of sandbagging.

If cheating while playing online takes away the fun for (legit) players, every solution should be benchmarked against providing (at least the usual - pre bot -) fun for the majority of players

As a contract bridge director, I consider my responsibility to be to all players, even suspected cheaters. Earlier in this thread I have written about ACBL procedures, which I think are good. OC, every organization has its own procedures. I don't pretend to prescribe for anybody else.

[RobertJasiek]

Possible indicators: Frequently over-performing.

How do you identify this?

Fast and seemingly unstoppable rank-climbing.

Do not confuse with new player climbing to his rank or with anomalies of the rating system.

Highly unstable results (rank graph like a sine curve).

Sigh. Such has been frequent on KGS for many years before AI programs. You first need to fix the rating system!

Maybe the frequent creation of new accounts, when the old ones plowed through a certain rank range.

Very common, but no indication of cheating unless a server rule prohibits it.

[Matti]

It's the lesson of history. When the pilfering of prisoners became a bad enough problem, the ancient Chinese changed from territory to area counting.

Has this been a real problem? Pilfering is easy to dtetct. After filling prisoners, if the numenr of black and white stones on the board differ by more than (possibly number of handicap stones), pilfering has taken place.

[John Fairbairn]

Has this been a real problem? Pilfering is easy to dtetct. After filling prisoners, if the numenr of black and white stones on the board differ by more than (possibly number of handicap stones), pilfering has taken place.

I'm sorry but I am unsure what this sentence is supposed to imply.

The nearest I can get to making sense of it is that you may are suggesting a player dissatisfied with the count can demand a count of all the stones on the board. In what universe with go clubs does that ever happen? As The Incident Room shows, they don't even do that in professional games. What I have seen, in amateur play, are attempts to count prisoners by looking at positions on the board where captures were made. These have been very rarely successful. There's also the point that, until quite recently, it was not the norm to fill in dame points alternately, so a surplus of one colour at the count-up stage was usually almost inevitable.

Remember, too, that the commonest cause of pilfering might not even involve the actual players. Hence the old age "never sit next to a left-handed player at a tournament." He might nick your prisoners, or even add to them, depending on whether he's playing or capturing. So even if you discover a problem, you can't be sure who the culprit is.

As to how much of a problem it has been, I can't be sure of that, either. The classic text is Wang Siren's Yi Lu from Ming times. That had a jokey element, of course, but the frequent prohibitions on gambling suggest that he was not mining fool's gold.

[Bill Spight]

Remember, too, that the commonest cause of pilfering might not even involve the actual players. Hence the old age "never sit next to a left-handed player at a tournament." He might nick your prisoners, or even add to them, depending on whether he's playing or capturing. So even if you discover a problem, you can't be sure who the culprit is.

Never play cards with a man called Doc. Never eat at a place called Mom's. Never play go beside a man called Lefty.

Nelson Algren didn't really say that.

[Bojanic]

I'd posit there's two types of cheaters: people who just run the bot and destroy their opponent, who want to play the best move every move. And people who want to get away with cheating, who spot check for blunders or the occasional tricky move, but who are savvy enough to not blindly play clearly bot sequences.

I really don't think you'll catch the latter group with anything short of monitoring. And again, if you have an algorithm that detects bot moves, you can train a bot that doesn't play those moves.

In a lot of ways, though, I think it's the former group that's the bigger problem because it's more visible. If people are complaining that the 9-dan ranks are all bots, that's a perception of a widespread problem. But I think Spinal Tap can point us to a helpful counter-measure: make it go to 11! Bots are better than us at Go. Let the cheaters raise up to a rank above us humans and play amongst themselves.

That is very good point.

By simply checking your moves for something like more than -5% loss, and then correcting it if necessary, one can significantly improve his winning chances. And it can never be proven. Though current policy can not prove an AI cheater even if he uses all of the best moves.

Let's go back to -5%. Even top European and American players (pro level) very often make double digit % mistakes.
Now you don't even have to play best AI move to beat other amateur. It is just enough not to play too bad move, and wait for opponent to make a bigger mistake.
In total, most of the moves don't even have to be top AI choice.

[Matti]

Has this been a real problem? Pilfering is easy to dtetct. After filling prisoners, if the numenr of black and white stones on the board differ by more than (possibly number of handicap stones), pilfering has taken place.

I'm sorry but I am unsure what this sentence is supposed to imply.

The nearest I can get to making sense of it is that you may are suggesting a player dissatisfied with the count can demand a count of all the stones on the board. In what universe with go clubs does that ever happen? As The Incident Room shows, they don't even do that in professional games. What I have seen, in amateur play, are attempts to count prisoners by looking at positions on the board where captures were made. These have been very rarely successful. There's also the point that, until quite recently, it was not the norm to fill in dame points alternately, so a surplus of one colour at the count-up stage was usually almost inevitable.

You are right. Area counting gives the players an incentive to fill the dame alternately. It is more efficient than just trying to enforce filling dame as a rule.

Remember, too, that the commonest cause of pilfering might not even involve the actual players. Hence the old age "never sit next to a left-handed player at a tournament." He might nick your prisoners, or even add to them, depending on whether he's playing or capturing. So even if you discover a problem, you can't be sure who the culprit is.

I have played next to left handed players in tournaments, but I am left handed too.

[moha]

Careful first steps also seem possible instead of banning suspicious accounts. For example, OGS already does quick bot reviews of finished games. This could be used to collect error distribution statistics for each player similar to this.

If this is then presented as part of the player profile, next to its current rating/rank, the decision could be left to other players (to play him or not). The average distribution for all accounts of the same rank could also be shown as comparison.

[Bill Spight]

Antti wrote an article for the Nordic Go Dojo titled How to catch cheaters online, in response to this discussion.

There he also analyzes one of the games from the thread "Can you determine the use of AI based on 4 games?".

I looked through his analysis, finally skipping to the conclusion. My impression is similar to Antti's. The player is sandbagging, perhaps with the aid of AI.

I also agree, as came out in the discussion in that thread here, that a very high concordance rate between a human's choices and a bot's, is evidence of cheating. The concordance rate between bots appears to be somewhere around 80%, at first blush. So for a human to achieve greater than 90% concordance is more than suspicious. The point is not matching, per se, which is weak, confirmatory evidence, but the difference between the concordance rate for the human versus the concordance rate for top bots.

jlt checked the concordance rate for the suspected cheater and found a 37% rate, which suggests an amateur dan level of play, not a 5 kyu level of play.

[FinrodFelagund]

A first solution might be to simply use the AI to characterize the skill of players and the skill level of specific games.

1.) Find the average winrate change (i.e. move 1 deviates from the AI's top choice by some percentage, called the move's delta for players) in a specific rank by analyzing a high number of games.

2.) Create a histogram for all ranks, from 25k to pro level play. This might mean that the average pro level move is only -5% (an example, I have no idea), and that the average 1d move is -15%.

You can use this data to identify suspicious players. The statistics of any player could be displayed in a public way on the server, so that instead of forcing the admins to decide if a player is cheating, the decision would be up to the suspicious player's opponents. This would work even for high dan players, where it would be obvious if their average deviation put them on par with or above top pros. Matchmaking systems and challenges could also be customized, so that you can set your own limits on the suspicious-ness of your opponents.

Eventually, you could use this kind of data to replace traditional ranking systems. Instead of a 1dan being some arbitrary elo rating, you could assign the rank to an average move delta.

If the cheater was clever, and didn't always pick the best move, they would end up playing at some reasonable level. It would be more difficult to detect these cheaters, but they would also do less harm, since they wouldn't be playing above the strength of the account and so would not on average be frustrating for honest players.

The only really difficult part of this is choosing the number of playouts and the specific engine. My guess is that even a relatively modest level of playouts with any strong engine would work, because the most important thing is that you use the same engine with the same settings for everyone, like a measuring stick.