It is currently Thu Mar 28, 2024 7:56 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 32 posts ]  Go to page Previous  1, 2
Author Message
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #21 Posted: Sat May 01, 2021 10:47 pm 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
To summarize,

:b1: found a weakness
:w2: disagreed
:b3: is a double-purpose move, based on the assumption that :w2: misread
:w4: didn't see the double purpose
:b5: makes clear that :w4: was a bad move
:w6: says that :b3: makes no sense since :w2: didn't misread.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #22 Posted: Sun May 02, 2021 3:33 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Javaness2 wrote:
Bill Spight wrote:
Here is a post on statistician Andrew Gelman's site about an accusation of cheating on chess.com.


You should note that Chess.com rescinded the ban, which is about as close as you'll get to an admission that they were wrong.


That's mentioned in the comments of both Gelman's post and his link to Horton's post. :)

Since what you want is a statistician, I thought that the take of a top level statistician would be of interest.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #23 Posted: Sun May 02, 2021 3:45 am 
Gosei

Posts: 1494
Liked others: 111
Was liked: 315
Indeed, but since you didn't reference it as an false accusation of cheating I thought it proper to mention this explicitly.
I don't remember if Ken Regan's analysis of these games was made public or not.
I recently read that most online chess servers take no action against sandbagging. That is an interesting choice given their policies.

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #24 Posted: Sun May 02, 2021 4:47 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
Bill Spight wrote:
the take of a top level statistician would be of interest.


One should study very many human - human, computer - computer, human - computer games to know what has happened and therefore, related, what can happen. (More can happen than what has happened but everything new happening is expected to be somehow near to what has happened.) The problem is to assess what happens / has happened. The only absolutely clear date is the result of each game. For just one particular game, its result is insufficient data for deciding whether cheating has occurred.

Therefore, the idea occurs to also measure moves.

Who suggested that one could identify one player's cheating by only assessing his moves but ignoring the opponent's moves? The player's and the opponent's moves have an impact on how the player's moves are perceived! Besides, both players might cheat.

Next, for the purpose of measuring moves, some measurement criteria and some interpretation criteria must be chosen with justification for each criterion, the combination of all criteria, their choice and the ommission of other criteria. The chosen criteria must be checked against what can happen in comparison to what has happened in the very many analysed games.

A top level statistician won't hurt for applying all this:) Regardless, arbitration involves much more than statistics. E.g., an alleged verdict only based on statistics can be rendered void if the suspect could not have accessed any electronic device or related transmitted information during the game.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #25 Posted: Sun May 02, 2021 5:54 am 
Lives in gote

Posts: 389
Liked others: 81
Was liked: 128
KGS: lepore
I wonder about the value of a top statistician, given the original poster's interest: "There's a beginner beat all the high dans and a few of the players suspected he cheated using ai. we need a solid proof before disqualifying him."

It is literally impossible for a beginner to beat a high dan, let alone "all the high dans." It seems they are just looking for evidence to confirm what they already know. As others have said, if you create a world where cheaters can plausibly offer any other explanation for their behavior, they will.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #26 Posted: Sun May 02, 2021 6:26 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
It depends on what weiqiasia means by "beginner". If he means roughly what I understand (10 kyu or weaker), my initial compromise solution is a solid proof because a beginner wins 0% against ca. amateur 3 dan test opponents, let alone more than 50%. If, however, he means "beginner among want-to-be qualifiers", establishing evidence can be difficult.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #27 Posted: Sun May 02, 2021 9:57 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
RobertJasiek wrote:
It depends on what weiqiasia means by "beginner". If he means roughly what I understand (10 kyu or weaker), my initial compromise solution is a solid proof because a beginner wins 0% against ca. amateur 3 dan test opponents, let alone more than 50%. If, however, he means "beginner among want-to-be qualifiers", establishing evidence can be difficult.


Yes, if beginner means kyu player then his known rank or a few FTF games against high amateur dans should be enough to disqualify him from representing any region in national competition. Even if arranging such games requires the cost of travel and accommodation, that should be cheaper than hiring a programmer or statistician.

Besides, from a statistical and evidentiary point of view, comparing the difference between the player's online moves and FTF moves is more telling than comparing the similarity between the player's online moves and AI choices.

Besides, faced with the choice of having to prove his mettle over the board, the player may choose to withdraw, allowing the second place finisher to play in the national compotition.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #28 Posted: Sat May 08, 2021 3:21 am 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
I come back to this thread because of news from a totally different field in Britain. If you are not British you will perhaps need a bit of background.

We have a class of workers called sub-postmasters and sub-postmistresses, the latter predominating by far, I believe. They have a special place in British life. So much so that they were once a stock character in films and novels. They run post offices, sometimes in conjunction with a grocery store, in villages and other small communities, holding a sort of franchise for what used to be the nationalised Royal Mail, which was eventually turned over to the private sector.

Especially in villages, the sub-posties shops formed a focus for community interaction, not just because they may have been the only grocery shop around, but because the post-office was where people collected their pensions, child benefits and so on in pre-digital days. On top of that - and this is what made them a stock character in fiction - they used to either have the only telephone in the area, or they ran they only telephone switchboard and could listen in to calls. They knew all the gossip. And soon all the village knew all the gossip.

The typical sub-postie was just like your grannie. The last person you could ever imagine would commit a crime. Also the last kind of person who did actually feature on lists of criminals.

I'm sure every nation has a similar class of person. The widows of war veterans in France come to mind, although it's a different age. Whoever they are, we are shocked and disbelieving when even ONE turns out to have their collar felt.

But then the new boys with their MBA diplomas took over the Royal Mail, all these sweet old ladies magically turned out to be Moriarty types with skeletons in their cupboards and evil in their hearts. Not one or two but HUNDREDS were arrested because the fancy new computer program the bottom-liners had installed PROVED they were defrauding the new shareholders. For reasons that completely escape me, but at a guess must be something to do with blind, quasi-religious faith in the then new religion of computer programming, they were all found guilty. Very many were imprisoned, and quite a few died there. Others were fined, got criminal convictions so lost their livelihoods and couldn't replacement jobs except as cleaners and so on.

As the number of prosecutions began to mount very, very many people nationwide began to voice disbelief. But the prosecutions just kept on coming. The computer program showed more and more fraud, even though most of the apparent fraudsters were now in gaol!

A campaign to revisit the evidence was strongly resisted by the Post Office. The algorithm was secret. But the basic stupidity of the prosecutions and the resistance could not be hidden. Still, it took years and years just to prise open the merest chink. But once the dam was breached, a flood of overturned convictions followed. It turned out the algorithm was faulty. Today I read that the Post Office has agreed to pursue a review of even more cases after PM Boris declared this was already the biggest case of miscarriage of justice ever in Britain.

Yet here in go we have a group of self-appointed people working on a secret algorithm that will similarly be used to trash the reputation, unaccountably and without recourse, of fellow go players around the world (and in chess, too, of course). All because of religious faith in numbers spewed in an unknown way from somebody else's computer.

I would not characterise my fellow go players as sweet old grannies, but I've met thousands and I would seriously never believe that more than a tiny handful would stoop to cheating.

I recognise that there is some cheating, and agree that it is VERY annoying, but given my real-world experience I have to conclude that the anti-cheating programs are a prime example of treating a headache with a lobotomy.

Digesting the Post Office news in Britain today, however, I also have to conclude that we are in grave danger of letting our distaste for cheating and fraud make us overlook the far worse risk of allowing the lobotomists and MBAs to run riot.


This post by John Fairbairn was liked by: Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #29 Posted: Sat May 08, 2021 5:49 am 
Dies in gote

Posts: 39
Location: Slovenia
Liked others: 36
Was liked: 9
Rank: EGF 1d
Universal go server handle: schrody
Online playing schedule: usually Sat & Sun afternoon CET
John Fairbairn wrote:
Yet here in go we have a group of self-appointed people working on a secret algorithm that will similarly be used to trash the reputation, unaccountably and without recourse, of fellow go players around the world (and in chess, too, of course). All because of religious faith in numbers spewed in an unknown way from somebody else's computer.


Of course it has to be a secret algorithm. Otherwise cheaters would learn how to circumvent it and then someone else would have to develop another secret algorithm.

On a serious note, I feel this is one of those damned if you do, damned if you don't situations. Either we do nothing and risk players quitting online go due to cheaters or we start banning "cheaters" and risk losing players due to false positives. The situation isn't that bad yet in go and we can only hope that due to go community being much smaller than the chess community, it won't worsen. I'm not all that optimistic. :sad:

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #30 Posted: Sat May 08, 2021 6:23 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
schrody wrote:
Of course it has to be a secret algorithm. Otherwise cheaters would learn how to circumvent it and then someone else would have to develop another secret algorithm.


An algorithm meant to reveal truth is open so that everybody can judge whether the claim of truth is fulfilled. Algorithms weaker than establishing truth should be avoided for judgement. If this means that there is no such algorithm, do not use any for arbitration.

Cheating relies on the false so must avoid all algorithms that establish truth. This does not mean that cheaters get away with it but it means that arbitration must (also) use what it has always used or could have always used: means of judgement other than algorithms.

Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #31 Posted: Sat May 08, 2021 6:34 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
John Fairbairn wrote:
I come back to this thread because of news from a totally different field in Britain. If you are not British you will perhaps need a bit of background.


I am in sympathy with this post, but I have a somewhat different take on the problem, which I will get to below.

John Fairbairn wrote:
We have a class of workers called sub-postmasters and sub-postmistresses, the latter predominating by far, I believe. They have a special place in British life. So much so that they were once a stock character in films and novels. They run post offices, sometimes in conjunction with a grocery store, in villages and other small communities, holding a sort of franchise for what used to be the nationalised Royal Mail, which was eventually turned over to the private sector.

{snip}

The typical sub-postie was just like your grannie. The last person you could ever imagine would commit a crime. Also the last kind of person who did actually feature on lists of criminals.

{snip}

But then the new boys with their MBA diplomas took over the Royal Mail, all these sweet old ladies magically turned out to be Moriarty types with skeletons in their cupboards and evil in their hearts. Not one or two but HUNDREDS were arrested because the fancy new computer program the bottom-liners had installed PROVED they were defrauding the new shareholders. For reasons that completely escape me, but at a guess must be something to do with blind, quasi-religious faith in the then new religion of computer programming, they were all found guilty.


I cannot comment on any popular, quasi-religious faith in computer programs, as I learned to program 50 years ago, and find such faith laughable.

John Fairbairn wrote:
As the number of prosecutions began to mount very, very many people nationwide began to voice disbelief. But the prosecutions just kept on coming. The computer program showed more and more fraud, even though most of the apparent fraudsters were now in gaol!


IIUC, both the US and the UK have an adversarial legal system, but for criminal prosecutions prosecutors are supposed to seek justice, but have career incentives to seek successful prosecutions, in most cases. An adversarial legal system leads to motivated reasoning on the part of the lawyers.

John Fairbairn wrote:
A campaign to revisit the evidence was strongly resisted by the Post Office. The algorithm was secret. But the basic stupidity of the prosecutions and the resistance could not be hidden.


The algorithm was secret. How in the hell could you prosecute someone on the basis of a secret algorithm?

The reason the algorithms are secret is that they are considered commercial trade secrets, whose revelation could damage the commercial viability, or at least, profitability of the firms that own them. This is a problem that continues to arise in the US as public functions have been privatized in whole or in part. I was alive at the rise of Reaganism and Thatcherism, and I agree that government can be the problem. But the answer is not privatization. IMX, private bureaucracy is worse than public bureaucracy, because to the private bureaucrats we are often the enemy, but the public bureaucrats usually realize that they work for us.

See Weapons of Math Destruction by Cathy O'Neil.

John Fairbairn wrote:
Digesting the Post Office news in Britain today, however, I also have to conclude that we are in grave danger of letting our distaste for cheating and fraud make us overlook the far worse risk of allowing the lobotomists and MBAs to run riot.


Hear, hear!

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


Last edited by Bill Spight on Sat May 08, 2021 7:37 am, edited 1 time in total.

This post by Bill Spight was liked by: Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: Will pay for real ai programmer/dev analysis for cheatin
Post #32 Posted: Sat May 08, 2021 7:17 am 
Lives in gote

Posts: 389
Liked others: 81
Was liked: 128
KGS: lepore
John's post is an interesting read (as usual), but I too don't understand convictions with a secret algorithm. Here in The States, the Internal Revenue Service uses algorithms to flag tax filings as suspicious or not. If there is something in your tax filings that gets the attention of the algorithm, it can trigger an audit, and it is then up to the humans to uncover the specific violations.

The traditional literature on the economics of deterrence doesn't really apply to these sub-posties, since they probably could not have foreseen technology being created that would catch them. But in online Go, it has to be on the cheater's mind, so we should be doing what we can to flag cheaters. I just don't know what that is.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 32 posts ]  Go to page Previous  1, 2

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group