It is currently Fri Dec 03, 2021 10:37 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 32 posts ]  Go to page 1, 2  Next
Author Message
Offline
 Post subject: Will pay for real ai programmer/dev analysis for cheating
Post #1 Posted: Thu Apr 29, 2021 3:53 am 
Beginner

Posts: 1
Liked others: 0
Was liked: 0
Hi, our official state association has monthly tournament, this is mainly for selecting players represent our state to compete in national competition.

There's a beginner beat all the high dans and a few of the players suspected he cheated using ai. we need a solid proof before disqualifying him.

We are hiring ai developer to analyze his games, about 7 sgf

Please contact me for more info, and let me know how much you charge.
Will pay with Paypal.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #2 Posted: Fri Apr 30, 2021 12:33 am 
Lives in gote

Posts: 435
Location: Netherlands
Liked others: 232
Was liked: 125
Rank: EGF 3d
Universal go server handle: gennan
It sounds to me that you already know that the player cheated in these 7 games. So I don't really understand what you expect from this developer. Are you looking for someone to come up with some numbers to support your suspicion?
As you are even paying for this, there is a very high risk of confirmation bias, I would say.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #3 Posted: Fri Apr 30, 2021 1:20 am 
Gosei
User avatar

Posts: 1360
Liked others: 124
Was liked: 417
You may want to contact Antti Törmänen 1p, he has some experience with cheating detection and if I am not mistaken, he has a program based on this paper:

https://arxiv.org/pdf/2009.01606.pdf

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #4 Posted: Fri Apr 30, 2021 1:37 am 
Lives in gote

Posts: 435
Location: Netherlands
Liked others: 232
Was liked: 125
Rank: EGF 3d
Universal go server handle: gennan
It is not only important to successfully detect cheating (true positives), but is also important (perhaps even more) that the detection method gives very few false positives (leading to unjustful disqualifications and public discrediting of innocent players). You want to avoid things like the Prosecutor's Fallacy and Confirmation Bias.

I think what's actually needed for proper automatic cheating detection, is this procedure (which is quite common in machine learning and software classifier competitions):

[1] prepare a large (the larger, the better) collection of games where you know for certain whether there was cheating or not, because it includes games of various volunteers who cheated on purpose in various ways that they saw fit. This collection needs to have games from all levels of play and many different playing styles.
[2] with this annotated collection of games, developers can create and test classifiers (by machine learning or some other method).
[3] you can objectively compare the quality of various classifiers by (for example) their Matthews correlation coefficient.
[4] you could even create a competition between classifiers from different developers by using a (perhaps undisclosed) representative subcollection of the annotated games that were not used in the creation and testing of those classifiers.

I think that step [1] will be a lot of work, requiring a coordinated effort to create a high quality dataset to use in the next steps. Also, I think step [1] is typically a task for an organisation and not a task for the developers of classifiers.

But once you have this data set, a public competition with prize money could be quite a cheap way to get a very good classifier (see for example the CASP14 protein folding competition that was won by Deepmind's AlphaFold in 2020).


This post by gennan was liked by 2 people: Harleqin, mhlepore
Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #5 Posted: Fri Apr 30, 2021 2:12 am 
Judan

Posts: 5517
Liked others: 0
Was liked: 748
"we need a solid proof"

Then do not rely on programs' analysis to infer guilt. You might use programs to infer likely innocence. If programs do not suggest likely innocence, then establish evidence on objective facts, such as eye-witnesses.

If you have already held a tournament for which objective facts cannot be established, then the mistake is one of the organisers, which should have held the tournament in a manner that objective facts can be established.

Compromise solution: Suppose players in the tournament are expected to play at least amateur X dan real world level. Let the alleged cheater prove his innocence in the tournament game(s) by letting him play test games against (X-3) dans (3 ranks weaker dans). If he wins more than 50% of the test games played so that objective facts are established, assume his innocence in the earlier tournament game.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #6 Posted: Fri Apr 30, 2021 2:20 am 
Gosei

Posts: 1382
Liked others: 110
Was liked: 286
and then the sun rose

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #7 Posted: Fri Apr 30, 2021 2:41 am 
Oza

Posts: 3086
Liked others: 18
Was liked: 4112
Quote:
and then the sun rose


And then, Ian, we could all see the tournament organisers floating up the Lagan in bubbles?

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #8 Posted: Fri Apr 30, 2021 5:13 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3368
weiqiasia wrote:
We are hiring ai developer to analyze his games, about 7 sgf


Pardon me, but why an AI developer? (A rhetorical question.) A forensic statistician has a more relevant skill set. To catch Jack the Ripper you don't ask someone who makes knives. Although they may have relevant information.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by 2 people: gennan, PeterHB
Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #9 Posted: Fri Apr 30, 2021 11:18 am 
Lives with ko

Posts: 274
Liked others: 30
Was liked: 36
Rank: OGS 6K
GD Posts: 56
KGS: CDavis7M
OGS: CDavis7M
RobertJasiek wrote:
"we need a solid proof"

Then do not rely on programs' analysis to infer guilt. You might use programs to infer likely innocence. If programs do not suggest likely innocence, then establish evidence on objective facts, such as eye-witnesses.


If "solid proof" is needed then surely eye-witnesses are out -- too unreliable.

----------

My suggestion would be to hold a supervised playoff match between the suspected cheater and the runner up.

_________________
㋷ ㋣ ㋢

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #10 Posted: Fri Apr 30, 2021 11:49 am 
Gosei

Posts: 1382
Liked others: 110
Was liked: 286
John Fairbairn wrote:
Quote:
and then the sun rose


And then, Ian, we could all see the tournament organisers floating up the Lagan in bubbles?


That phrase was rather quaint when I was a child, what wonders TV holds, but rest assured that if you lived near Slieve Croob you really wouldn't want to take that trip.

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #11 Posted: Fri Apr 30, 2021 11:51 am 
Gosei
User avatar

Posts: 1360
Liked others: 124
Was liked: 417
P.S. A few months ago I read an email which suggests that the organizers of this competition https://m.facebook.com/2020WYAWT/about

have a cheating detection program, so you may want to ask them.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #12 Posted: Fri Apr 30, 2021 12:44 pm 
Judan

Posts: 5517
Liked others: 0
Was liked: 748
CDavis7M wrote:
If "solid proof" is needed then surely eye-witnesses are out -- too unreliable.


It depends on what they saw, how they report it and how the court judges that.

An eye-witness contradicting himself in central aspects cannot be trusted.

Several eye-withnesses reporting the same independently from each other can be trusted.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #13 Posted: Sat May 01, 2021 2:33 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3368
Here is a post on statistician Andrew Gelman's site about an accusation of cheating on chess.com.

https://statmodeling.stat.columbia.edu/ ... meone-off/

And here are a couple of links to the accused cheater's site.

https://lostontime.blogspot.com/2019/10 ... ed_67.html

https://lostontime.blogspot.com/2019/10 ... ed_89.html

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #14 Posted: Sat May 01, 2021 3:45 am 
Oza

Posts: 3086
Liked others: 18
Was liked: 4112
Thanks, Bill.

One of those links contained a point I think has been generally overlooked, and I certainly totally overlooked it.

Quote:
I gather from the complaint, though, that it’s the online chess equivalent of having one of your publications publicly flagged by an algorithm as reporting fake data.


I have forgotten the details, and even some of the bigger points, as it was many years ago and I wasn't directly involved, but as I recall there was a case of a British organisation being affected with a denial of service by a flood of e-mails by Stop the War protesters. The response was to treat 'stop the war' as a junk phrase at server level and so e-mails containing it were stopped getting through. That caused major problems as it blocked all sorts of legitimate and valuable e-mails from other people such as politicians (e.g. "should we actually stop the war?").

In that case, both sides forgot the maxim that actions have consequences, and further forgot that in most cases the consequences are the ones you least expect.

Seems like chess and go are following the same dangerous path. The aspect that would worry me most is that the anti-cheaters are unwittingly creating an atmosphere that leads to their beloved game being regarded as a vice. It's happened before. When I was young, there were many parents who saw cards as the devil's playthings and so banned their children from playing bridge or whist.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #15 Posted: Sat May 01, 2021 5:13 am 
Judan

Posts: 5517
Liked others: 0
Was liked: 748
I do not think that with "anti-cheaters" you mean everybody not cheating. You also do not even mean all organisers, referees, server admins investigating. Apparently, you only mean those officials being rash with identifying alleged cheating.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #16 Posted: Sat May 01, 2021 6:38 am 
Oza

Posts: 3086
Liked others: 18
Was liked: 4112
Quote:
I do not think that with "anti-cheaters" you mean everybody not cheating. You also do not even mean all organisers, referees, server admins investigating. Apparently, you only mean those officials being rash with identifying alleged cheating.


No, I mean ALL. Remember how keen you are to promote the idea of accurate reading. I wrote 'unwittingly'. maybe you didn't understand it and so skipped over it, but 'wit' has the same root as German wissen. And that unknowingness is what makes the whole business so nastily insidious.

I have no solutions, and don't believe anyone else has. So-called solutions just create more problems. That's why it's nasty.

It's not the fault of the good people you mention. It's the bad apples. But it takes just one bad apple to sour the barrel, just as it takes just one selfish idiot to spread the virus.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #17 Posted: Sat May 01, 2021 7:29 am 
Judan

Posts: 5517
Liked others: 0
Was liked: 748
I do not share your thought that every player who opposes cheating or anybody advocating strong skill, such as accurate resding, would contribute to the problem initiated by cheating. I only agree that bad solutions contribute to the problem.

Concerning the specific suggestion that advocating accurate reading would contribute to the problem, you need to explain why you think this instead of thinking that everybody should strive to reach this aspect of skill of the strongest players.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #18 Posted: Sat May 01, 2021 8:22 am 
Oza

Posts: 3086
Liked others: 18
Was liked: 4112
Quote:
Concerning the specific suggestion that advocating accurate reading would contribute to the problem, you need to explain why you think this instead of thinking that everybody should strive to reach this aspect of skill of the strongest players.


Your English is clearly too poor to have this conversation. I am referring to the fact that you did not read what I wrote accurately, which is ironic because you advocate careful reading, albeit of a different type. You (a) skipped the key word the first time and (b) ignored me when I pointed the keyword out to you. To repeat, my point is that good people who think they are providing good solutions can (and do) UNWITTINGLY (UNWISSENTLICH, UNBEWUSST, UNBEABSICHTIGT) cause problems. Good intentions are never enough. And bad solutions do not always reveal themselves at once.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #19 Posted: Sat May 01, 2021 12:08 pm 
Gosei

Posts: 1382
Liked others: 110
Was liked: 286
Bill Spight wrote:
Here is a post on statistician Andrew Gelman's site about an accusation of cheating on chess.com.


You should note that Chess.com rescinded the ban, which is about as close as you'll get to an admission that they were wrong.
This case was particularly amusing given that Mr Horton was a contributor to Kingpin and a foe of the dishonest writers/administrators/whatevers

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #20 Posted: Sat May 01, 2021 10:05 pm 
Judan

Posts: 5517
Liked others: 0
Was liked: 748
John Fairbairn wrote:
Your English is clearly too poor to have this conversation.


You repeatedly rely your meta-discussion on the false assumption that I would not have understood the following:

Quote:
good people who think they are providing good solutions can (and do) UNWITTINGLY (UNWISSENTLICH, UNBEWUSST, UNBEABSICHTIGT) cause problems.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 32 posts ]  Go to page 1, 2  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group