It is currently Thu Mar 28, 2024 5:43 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 147 posts ]  Go to page Previous  1 ... 4, 5, 6, 7, 8  Next
Author Message
Offline
 Post subject: Re: On handling online cheating with AI
Post #121 Posted: Fri Jun 05, 2020 6:10 am 
Gosei
User avatar

Posts: 2011
Location: Groningen, NL
Liked others: 202
Was liked: 1087
Rank: Dutch 4D
GD Posts: 645
Universal go server handle: herminator
John Fairbairn wrote:
Quote:
Using Zoom doesn't guarantee that an AI doesn't show moves on the screen (or on another screen).


Of course it doesn't. It's an aid. You can use it watch a player's eyes move, and so on. Humans are incredibly sensitive to anything that happens on the face.

It's not perfect but it's practical and free and it's here now.

And other one point the numbers guys keep overlooking is that a mathematical modelling solution, apart from being also imperfect while being impractical, expensive and not available yet, is that for people who don't understand the maths or statistics, the imposition of such a system creates an uncomfortable atmosphere - almost an invasion of privacy. It's a kind of Google/Facebook/Twitter approach to life. You have to trust something you don't really understand. Of course you have to do that sometimes in life - but for a mere board game? It's an anti-social solution to a social problem.


On the other hand, I must say I wouldn't always be comfortable with the human approach. Sure, humans are very sensitive to anything that happens on the face, but they are also consciously and subconsciously biased in a million subtle ways. People have their prejudices. People will reach conclusions, in part, based on race, gender, nationality, religion and appearance. One of the advantages of a technical solution based purely on the analysis of the moves is that players can trust that it has no such bias. That they are not more likely to be accused of cheating based on the colour of their skin.

Some may not trust computer algorithms they do not understand. Others may not trust the fallible human referee, whose judgement might also be impaired by being tired, tipsy or annoyed. Technological solutions may not be the be-all-end-all of preventing and detecting cheating, but they certainly have their place.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #122 Posted: Fri Jun 05, 2020 7:13 am 
Lives in gote

Posts: 532
Liked others: 1
Was liked: 136
Rank: 13k
If I were playing my favorite partner online and he suddenly made two or three or four direct 3-3s, any of those weirdly unsupported mid-region invasions, or those funny-looking attachments on the third line I’d know three things: 1) he had an AI open on a separate device; 2) he was in way over his head; and 3) I would be resigning in a few dozen more moves.

What would differentiate me knowing he had the AI open instead of having grown five stones stronger? The follow-up moves would be too good, too precise, too inexplicably out of character.

_________________
David Bogie, Boise ID
I play go, I ride a recumbent, of course I use Macintosh.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #123 Posted: Fri Jun 05, 2020 7:32 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
HermanHiddema wrote:
People have their prejudices. People will reach conclusions, in part, based on race, gender, nationality, religion and appearance. One of the advantages of a technical solution based purely on the analysis of the moves is that players can trust that it has no such bias. That they are not more likely to be accused of cheating based on the colour of their skin.


Or because they are Eurasian teenage males? ;)

In the real world, technical algorithms have been found to enshrine prejudices. See the book, Weapons of Math Destruction, for more. The moves do not reveal cheating or not cheating. You have to have examples of cheating and non-cheating to compare. Or, given cheating by bot, comparable examples of bot vs. human play. I actually provided a few such examples, controlling for the quality of the plays. For that I have been roundly criticized. ;)

In any event, technical solutions have to be validated, and that is not easy to do. There are many more examples of suspected cheating than of proven cheating. You can't use the examples of suspected cheating to validate cheating detection algorithms. That would enshrine current preconceptions and prejudices.

Quote:
Technological solutions may not be the be-all-end-all of preventing and detecting cheating, but they certainly have their place.


True. But John's suggestion has merit, as well. :)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by: Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #124 Posted: Fri Jun 05, 2020 7:51 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
bogiesan wrote:
If I were playing my favorite partner online and he suddenly made two or three or four direct 3-3s, any of those weirdly unsupported mid-region invasions, or those funny-looking attachments on the third line I’d know three things: 1) he had an AI open on a separate device; 2) he was in way over his head; and 3) I would be resigning in a few dozen more moves.


Marcel Grünauer wrote:
Or he watched lectures and reviews in the Yunguseng Dojang, where Hwang In-seong 8d talks about these things at length and encourages his students to try these moves in their games.


This illustrates something that I think Polama hinted at before.
Polama wrote:
Of course, we'll need to resample those players every couple years as our understanding of Go improves...


I suspect that within 10-15 years the median level of amateur go will have increased by one stone, maybe two. The reason is that we have learned and will continue to learn from bots. Which also means that we will play more like bots. We already see the effect in joseki and openings. It used to be that beginners played early 3-3 invasions, but now everybody does it. We also play those funny attachments. And even if we don't understand bot plays, we can imitate them, and learn from experience. The difference between human play and bot play is decreasing.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #125 Posted: Fri Jun 05, 2020 8:47 am 
Gosei
User avatar

Posts: 1810
Liked others: 490
Was liked: 365
Rank: KGS 1-dan
Since this thread has now seven pages and at best a hint of an idea of a solution... maybe we should start anew and align our conception of what the actual problem is?

My understanding is, the main and for the moment maybe only relevant problem is cheating while playing go online.

As others have pointed out (and Antti even wrote at length about it), playing against a bot or bot enhanced player is not fun. The most obvious reason being: Competition-wise all your efforts will be in vain because you can never win.

If cheating while playing online takes away the fun for (legit) players, every solution should be benchmarked against providing (at least the usual - pre bot -) fun for the majority of players.

But to go back one step: Even if you'll lose because your opponent used a bot, it only kills the fun when you think bot usage is likely in play (and also happens somewhat frequently, I guess). So at least in the beginning the goal should be to get rid of obvious cheaters. The ones the admins can (easily) discern themselves but which takes a lot of time and energy when doing in manually.

Possible indicators: Frequently over-performing. Fast and seemingly unstoppable rank-climbing. Highly unstable results (rank graph like a sine curve). Maybe the frequent creation of new accounts, when the old ones plowed through a certain rank range. See Adin's first post, too.

You don't need a highly complex algorithm to search the player base for these indicators. Decisions can still be made by the admins. Of course there still should be a strike-system and a courtesy-note to the suspected player informing her or him about possibilities to appeal the decision.

On the highest level the problem can even be (partially) solved by simple means. Look at chess and start using titles or indications for strong players who provided ID and can show their documented strength (EGF, pro dan, AGA database) - could also produce a nice cashflow for the federations... Of course the ID won't be made public without the players consent but other strong players will know that there is an actual strong person on the other side. In online chess you rarely have a high elo player without a title and to my knowledge there is no (relevant) problem of cheating in high level games and there are quite a few and regular online tournaments with cash prizes.

_________________
My "guide" to become stronger in Go

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #126 Posted: Fri Jun 05, 2020 9:19 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
SoDesuNe wrote:
As others have pointed out (and Antti even wrote at length about it), playing against a bot or bot enhanced player is not fun. The most obvious reason being: Competition-wise all your efforts will be in vain because you can never win.


Polama's idea addresses that problem.

Polama wrote:
If people are complaining that the 9-dan ranks are all bots, that's a perception of a widespread problem. But I think Spinal Tap can point us to a helpful counter-measure: make it go to 11! Bots are better than us at Go. Let the cheaters raise up to a rank above us humans and play amongst themselves.


So if cheating 9 dans are always defeating honest human 9 dans, promote them to 11 dan, or even higher. Let them play amongst themselves, if that's what they would like to do. (Probably not, given the motivation for such cheating.) If, then, the cheaters start losing games in order to keep their ratings down, then we have a type of familiar problem, that of sandbagging.

Quote:
If cheating while playing online takes away the fun for (legit) players, every solution should be benchmarked against providing (at least the usual - pre bot -) fun for the majority of players


As a contract bridge director, I consider my responsibility to be to all players, even suspected cheaters. Earlier in this thread I have written about ACBL procedures, which I think are good. :) OC, every organization has its own procedures. I don't pretend to prescribe for anybody else.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by: Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #127 Posted: Fri Jun 05, 2020 10:32 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
SoDesuNe wrote:
Possible indicators: Frequently over-performing.


How do you identify this?

Quote:
Fast and seemingly unstoppable rank-climbing.


Do not confuse with new player climbing to his rank or with anomalies of the rating system.

Quote:
Highly unstable results (rank graph like a sine curve).


Sigh. Such has been frequent on KGS for many years before AI programs. You first need to fix the rating system!

Quote:
Maybe the frequent creation of new accounts, when the old ones plowed through a certain rank range.


Very common, but no indication of cheating unless a server rule prohibits it.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #128 Posted: Sat Jun 06, 2020 4:42 am 
Lives in gote

Posts: 309
Liked others: 3
Was liked: 41
Rank: 5 dan
John Fairbairn wrote:
It's the lesson of history. When the pilfering of prisoners became a bad enough problem, the ancient Chinese changed from territory to area counting.

Has this been a real problem? Pilfering is easy to dtetct. After filling prisoners, if the numenr of black and white stones on the board differ by more than (possibly number of handicap stones), pilfering has taken place.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #129 Posted: Sat Jun 06, 2020 8:19 am 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
Quote:
Has this been a real problem? Pilfering is easy to dtetct. After filling prisoners, if the numenr of black and white stones on the board differ by more than (possibly number of handicap stones), pilfering has taken place.


I'm sorry but I am unsure what this sentence is supposed to imply.

The nearest I can get to making sense of it is that you may are suggesting a player dissatisfied with the count can demand a count of all the stones on the board. In what universe with go clubs does that ever happen? As The Incident Room shows, they don't even do that in professional games. What I have seen, in amateur play, are attempts to count prisoners by looking at positions on the board where captures were made. These have been very rarely successful. There's also the point that, until quite recently, it was not the norm to fill in dame points alternately, so a surplus of one colour at the count-up stage was usually almost inevitable.

Remember, too, that the commonest cause of pilfering might not even involve the actual players. Hence the old age "never sit next to a left-handed player at a tournament." He might nick your prisoners, or even add to them, depending on whether he's playing or capturing. So even if you discover a problem, you can't be sure who the culprit is.

As to how much of a problem it has been, I can't be sure of that, either. The classic text is Wang Siren's Yi Lu from Ming times. That had a jokey element, of course, but the frequent prohibitions on gambling suggest that he was not mining fool's gold.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #130 Posted: Sat Jun 06, 2020 9:11 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
John Fairbairn wrote:
Remember, too, that the commonest cause of pilfering might not even involve the actual players. Hence the old age "never sit next to a left-handed player at a tournament." He might nick your prisoners, or even add to them, depending on whether he's playing or capturing. So even if you discover a problem, you can't be sure who the culprit is.


Nelson Algren wrote:
Never play cards with a man called Doc. Never eat at a place called Mom's. Never play go beside a man called Lefty.


Nelson Algren didn't really say that. ;)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #131 Posted: Sun Jun 07, 2020 11:35 pm 
Lives with ko

Posts: 142
Liked others: 27
Was liked: 89
Rank: 5 dan
Polama wrote:
I'd posit there's two types of cheaters: people who just run the bot and destroy their opponent, who want to play the best move every move. And people who want to get away with cheating, who spot check for blunders or the occasional tricky move, but who are savvy enough to not blindly play clearly bot sequences.

I really don't think you'll catch the latter group with anything short of monitoring. And again, if you have an algorithm that detects bot moves, you can train a bot that doesn't play those moves.

In a lot of ways, though, I think it's the former group that's the bigger problem because it's more visible. If people are complaining that the 9-dan ranks are all bots, that's a perception of a widespread problem. But I think Spinal Tap can point us to a helpful counter-measure: make it go to 11! Bots are better than us at Go. Let the cheaters raise up to a rank above us humans and play amongst themselves.

That is very good point.

By simply checking your moves for something like more than -5% loss, and then correcting it if necessary, one can significantly improve his winning chances. And it can never be proven. Though current policy can not prove an AI cheater even if he uses all of the best moves.

Let's go back to -5%. Even top European and American players (pro level) very often make double digit % mistakes.
Now you don't even have to play best AI move to beat other amateur. It is just enough not to play too bad move, and wait for opponent to make a bigger mistake.
In total, most of the moves don't even have to be top AI choice.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #132 Posted: Mon Jun 08, 2020 8:27 am 
Lives in gote

Posts: 309
Liked others: 3
Was liked: 41
Rank: 5 dan
John Fairbairn wrote:
Quote:
Has this been a real problem? Pilfering is easy to dtetct. After filling prisoners, if the numenr of black and white stones on the board differ by more than (possibly number of handicap stones), pilfering has taken place.


I'm sorry but I am unsure what this sentence is supposed to imply.

The nearest I can get to making sense of it is that you may are suggesting a player dissatisfied with the count can demand a count of all the stones on the board. In what universe with go clubs does that ever happen? As The Incident Room shows, they don't even do that in professional games. What I have seen, in amateur play, are attempts to count prisoners by looking at positions on the board where captures were made. These have been very rarely successful. There's also the point that, until quite recently, it was not the norm to fill in dame points alternately, so a surplus of one colour at the count-up stage was usually almost inevitable.
You are right. Area counting gives the players an incentive to fill the dame alternately. It is more efficient than just trying to enforce filling dame as a rule.
Quote:
Remember, too, that the commonest cause of pilfering might not even involve the actual players. Hence the old age "never sit next to a left-handed player at a tournament." He might nick your prisoners, or even add to them, depending on whether he's playing or capturing. So even if you discover a problem, you can't be sure who the culprit is.

I have played next to left handed players in tournaments, but I am left handed too.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #133 Posted: Wed Jun 10, 2020 5:30 am 
Lives in gote

Posts: 311
Liked others: 0
Was liked: 45
Rank: 2d
Careful first steps also seem possible instead of banning suspicious accounts. For example, OGS already does quick bot reviews of finished games. This could be used to collect error distribution statistics for each player similar to this.

If this is then presented as part of the player profile, next to its current rating/rank, the decision could be left to other players (to play him or not). The average distribution for all accounts of the same rank could also be shown as comparison.


This post by moha was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #134 Posted: Wed Jun 10, 2020 7:43 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Marcel Grünauer wrote:
Antti wrote an article for the Nordic Go Dojo titled How to catch cheaters online, in response to this discussion.

There he also analyzes one of the games from the thread "Can you determine the use of AI based on 4 games?".


I looked through his analysis, finally skipping to the conclusion. My impression is similar to Antti's. The player is sandbagging, perhaps with the aid of AI.

I also agree, as came out in the discussion in that thread here, that a very high concordance rate between a human's choices and a bot's, is evidence of cheating. The concordance rate between bots appears to be somewhere around 80%, at first blush. So for a human to achieve greater than 90% concordance is more than suspicious. The point is not matching, per se, which is weak, confirmatory evidence, but the difference between the concordance rate for the human versus the concordance rate for top bots.

jlt checked the concordance rate for the suspected cheater and found a 37% rate, which suggests an amateur dan level of play, not a 5 kyu level of play.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #135 Posted: Wed Jun 10, 2020 8:56 am 
Beginner
User avatar

Posts: 18
Liked others: 0
Was liked: 1
Rank: OGS 2k
IGS: GilGalad
OGS: FinrodFelagund
A first solution might be to simply use the AI to characterize the skill of players and the skill level of specific games.

1.) Find the average winrate change (i.e. move 1 deviates from the AI's top choice by some percentage, called the move's delta for players) in a specific rank by analyzing a high number of games.

2.) Create a histogram for all ranks, from 25k to pro level play. This might mean that the average pro level move is only -5% (an example, I have no idea), and that the average 1d move is -15%.

You can use this data to identify suspicious players. The statistics of any player could be displayed in a public way on the server, so that instead of forcing the admins to decide if a player is cheating, the decision would be up to the suspicious player's opponents. This would work even for high dan players, where it would be obvious if their average deviation put them on par with or above top pros. Matchmaking systems and challenges could also be customized, so that you can set your own limits on the suspicious-ness of your opponents.

Eventually, you could use this kind of data to replace traditional ranking systems. Instead of a 1dan being some arbitrary elo rating, you could assign the rank to an average move delta.

If the cheater was clever, and didn't always pick the best move, they would end up playing at some reasonable level. It would be more difficult to detect these cheaters, but they would also do less harm, since they wouldn't be playing above the strength of the account and so would not on average be frustrating for honest players.

The only really difficult part of this is choosing the number of playouts and the specific engine. My guess is that even a relatively modest level of playouts with any strong engine would work, because the most important thing is that you use the same engine with the same settings for everyone, like a measuring stick.

_________________
Now men awoke and listened to Felagund as he harped and sang, and each thought that he was in some fair dream, until he saw that his fellows were awake also beside him; but they did not speak or stir while Felagund still played

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #136 Posted: Wed Jun 10, 2020 5:56 pm 
Beginner

Posts: 19
Liked others: 2
Was liked: 8
Rank: 1p
FinrodFelagund wrote:
A first solution might be to simply use the AI to characterize the skill of players and the skill level of specific games.

1.) Find the average winrate change (i.e. move 1 deviates from the AI's top choice by some percentage, called the move's delta for players) in a specific rank by analyzing a high number of games.

2.) Create a histogram for all ranks, from 25k to pro level play. This might mean that the average pro level move is only -5% (an example, I have no idea), and that the average 1d move is -15%.

You can use this data to identify suspicious players. The statistics of any player could be displayed in a public way on the server, so that instead of forcing the admins to decide if a player is cheating, the decision would be up to the suspicious player's opponents. This would work even for high dan players, where it would be obvious if their average deviation put them on par with or above top pros. Matchmaking systems and challenges could also be customized, so that you can set your own limits on the suspicious-ness of your opponents.

Eventually, you could use this kind of data to replace traditional ranking systems. Instead of a 1dan being some arbitrary elo rating, you could assign the rank to an average move delta.

If the cheater was clever, and didn't always pick the best move, they would end up playing at some reasonable level. It would be more difficult to detect these cheaters, but they would also do less harm, since they wouldn't be playing above the strength of the account and so would not on average be frustrating for honest players.

The only really difficult part of this is choosing the number of playouts and the specific engine. My guess is that even a relatively modest level of playouts with any strong engine would work, because the most important thing is that you use the same engine with the same settings for everyone, like a measuring stick.


I have looked into this, and unfortunately it does not look feasible.

I have analysed a considerable number of games with KataGo looking at the size of a player's 'average mistake' – not in terms of winrate-%, but points, as KataGo is able to do. I think this method is more robust than looking at the winrate, because even a very small mistake can cause a big winrate change when the game is close.

After analysing a few players, Ke Jie's average mistake seemed to be in the range of -0.5 points per move. For my own games, I got around -0.8 points; then for Shūsaku I got around -1.2 points, at which point I started to get suspicious. Then I checked a few European 6d players, who came to around -1.5 points; and then I came upon a game by Lukas Podpera 7d and Tanguy Le Calvé, which had an average mistake of only -0.3 points per move for both players.

Investigating further, I realised that the size of the average mistake depends on the 'nature' of the game: fighting-oriented games inevitable lead to higher average mistakes and peaceful games lead to lower average mistake. This is why Ke Jie's -0.5 points per move is impressive. Even if we analysed winrate-% rather than KataGo-points, I believe we would get the same conclusion.

In order to find a way to rank players by the size of their average mistake, it seems we need a way to quantify how 'complex', or 'error-prone', a particular game is. So far I have not thought of a way to accomplish this.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #137 Posted: Thu Jun 11, 2020 2:11 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
FinrodFelagund wrote:
A first solution might be to simply use the AI to characterize the skill of players and the skill level of specific games.

1.) Find the average winrate change (i.e. move 1 deviates from the AI's top choice by some percentage, called the move's delta for players) in a specific rank by analyzing a high number of games.

2.) Create a histogram for all ranks, from 25k to pro level play. This might mean that the average pro level move is only -5% (an example, I have no idea), and that the average 1d move is -15%.

You can use this data to identify suspicious players. The statistics of any player could be displayed in a public way on the server, so that instead of forcing the admins to decide if a player is cheating, the decision would be up to the suspicious player's opponents. This would work even for high dan players, where it would be obvious if their average deviation put them on par with or above top pros. Matchmaking systems and challenges could also be customized, so that you can set your own limits on the suspicious-ness of your opponents.

Eventually, you could use this kind of data to replace traditional ranking systems. Instead of a 1dan being some arbitrary elo rating, you could assign the rank to an average move delta.
...


Before you make this kind of system, you have to be very certain that 'average move delta' correlates where clearly with rank.

For instance if we have 2 1d players:
One player studies with AI a lot, and is for most of the moves able to play moves with low delta, but he is weaker in fighting and is prone to making big decisive blunders.

The other player employs a tricky creative style, and is stronger in fighting. He often has higher delta because he plays unorthodox moves / overplays, but overall he is more consistent due to his fighting ability.

Will these be measured similarly by the system?

And if it's only a 'suspiciousness' measure - I don't want to be judged by other players because of the style that I chose to play, before the game starts.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #138 Posted: Thu Jun 11, 2020 11:02 am 
Lives in sente
User avatar

Posts: 914
Liked others: 391
Was liked: 162
Rank: German 2 dan
Sorry, but I am quite unconvinced by Antti's analysis, especially because all moves that are counter-examples are disregarded as “then he didn't cheat for just those moves”.

The way to verify such a method would be to get a large enough sample size of known good/bad games, then doing a double-blind application of the method and determine false/true positive/negative. This would have to be done at different levels, too.

Obvious cases are easy to find, and easy to adjudicate. 100% bot choice: no problem. The difficulty is finding the threshold of obviousness. Where do we cross the line where there's just “strong suspicion”? How do we protect against bias of the judge?

_________________
A good system naturally covers all corner cases without further effort.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #139 Posted: Thu Jun 11, 2020 11:51 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Antti's argument is something like:

1) Many moves are blue, and some surprising moves are considered by the AI, so Black is a strong dan player.

2) However a few moves like :b61: are kyu-level mistakes.

It would be interesting to find an algorithm that determines if a mistake is DDK level, or 10k, or 5k level without relying on a human judgment but just on a bot analysis, but that doesn't seem an easy task.

Top
 Profile  
 
Offline
 Post subject: Re: On handling online cheating with AI
Post #140 Posted: Thu Jun 11, 2020 12:44 pm 
Oza
User avatar

Posts: 2408
Location: Ghent, Belgium
Liked others: 359
Was liked: 1019
Rank: KGS 2d OGS 1d Fox 4d
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
Yakago wrote:
For instance if we have 2 1d players:
One player studies with AI a lot, and is for most of the moves able to play moves with low delta, but he is weaker in fighting and is prone to making big decisive blunders.


There's an assumption here that "being weaker in fighting" would still allow you to stay close to AI play most of the time. That's true in the opening but not in the middle game, where the choice for a move depends on the understanding of a group's status and relationships between those positions. Although the AI choice is often on the territorial side, their "understanding" of fighting is supreme.

We must discard the opening from an investigation on cheating because indeed it's much harder to distinguish between learning from (in the mimicking sense) AI and using it. This is similar to amateurs who are strong in the opening because they mimic pro play. Then when the stones get into contact, the positions and the players' images crumble.

If you have learned so much that you can stay close to AI choices throughout the whole game, then you've reached professional strength. To do that in a very short time span is suspicious. To play mostly at AI level and make a few kyu level blunders, is also suspicious.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 147 posts ]  Go to page Previous  1 ... 4, 5, 6, 7, 8  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group