Based on only 4 games, is it possible to determine the use of AI?

Game 1:


Game2:

Game 3:

Game 4:

What do you mean, the use of AI?

What is the evidence of using AI in go, except becoming a better player?

As for using AI to cheat, I suppose the giveaway is relatively poor reading skill.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Online cheating is very difficult to detect, read for instance

https://www.nordicgodojo.eu/post/33/how ... -online-go

Put the games in the embedded eidogo player and forum members will more likely examine them.

I presume you mean using AI during the game, rather than to study before the game.

Some games of the same player where he is known to not be using AI (eg before strong bots if he was same strength years ago, in person game records where more easily believed to have not been cheating) for comparison allow stronger conclusions.

I think that just from a game by itself, you cannot say whether AI was consulted. If you know a player's recent games, you may be able to make a case that the play in the game is improbable, but I suspect this will be not convincing enough for anything approaching dan levels. It becomes hopeless if players use AI only in some moves.

So, proctors seem like the only solution to me.

Using a VR set as a “proctor”, as Antti proposed, is interesting, but also seems too easy to spoof to me. Besides, I am a bit concerned about the development of surveillance technologies.

Another option could be a two-webcam setup (one from the front, one from the back), but again, it seems easy to spoof.

In both VR set and two-webcam setup I would feel quite uncomfortable that any movement I make or do not make might be seen as suspicious. It's the classic surveillance conundrum: do you really want to put everyone under suspicion just to catch the few bad actors? I don't want to get philosophical here, but the answer should generally be “no”.

In conclusion, I think that actual human proctors are the only way forward. Proxy venues, as Marcel proposed, seem like a very useful thing to me.

There is one more option: play only bots, or care not whether your opponent is one, but this is of course not realistic for tournaments.

_________________
A good system naturally covers all corner cases without further effort.

The play is obviously stronger than any of the KGS ranks next to his name. Without checking vs AI, I'd say first game he hit all the opening points AI likes, which a mid-dan player like myself who has trained with AI can identify, though the chance of getting a dozen such decisions right in a row diminishes. I'd be tempted to atari at a10 before defending, but is that bad human atari love because a11 is also sente? If bot doesn't want to that's another point towards bot cheating, though of course strong humans can also resist ataris.

Game 3 loss ending looks like a misclick/too eager click of expecting W to answer at h16 (0 point gote) and then playing g6 as next planned move without realising white actually played h4 sente. This could happen with a non-cheating human not paying attention, but switching between a bot and the game board probably makes such mistakes more likely.

Game 4 move 32 is a move I learnt from bots, likewise 34 not being slow but good.

Right. We have a sandbagger, to be sure. But an AI aided sandbagger? Quien sabe?



The sagari to a11 is superior. White's reply is the ugly connection at b9. Ugh! As for the protection, I did not learn the keima connection until I was 4 kyu.

But what about ? That's a human looking move, isn't it? It induces White to make good shape at D-06, weakening the nearby Black stone at D-08. And what does Black gain? Isn't it better to leave D-06 as an option for Black?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

15-block Katago likes both: defend directly or atari first. Katago likes . In fact, most black moves are among Katago's suggestions, with a few exceptions like , , but there are no big mistakes.

Hmmmm. Recently didn't we find around 80% concordance between KataGo and LZ? A nearly 100% concordance is amazing.

Previously I have pointed out the weakness of confirmatory evidence, in this case, agreement between the human's play and the bot's choice. The evidence here is not that agreement, per se, but the difference between that rate of agreement and the rate of agreement between two top bots, or the rate of agreement between a top bot and a professional.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

It depends what you mean by "considered". The hane is sixth in terms of policy, and loses 3.5% compared to B3, so it was "considered" by Katago only in a weak sense.

Thanks for the clarification.

I brought up concordance (exact match between the game play and the bot's top choice) because it is something on which we have hard, if currently limited, evidence. But it should not be hard to gather, as far as professional play is concerned.

For instance, in the Elf commentaries all of Elf's top choices are tagged as tesuji, TE[1]. So you can just count how often that tag appeears in the game record, excluding variations, and compare with the number of plays. Take GoGoD 2018-05-28l, with Meng Tailing, 6 dan, (W) vs. Kang Tongyun, 9 dan. There were 269 plays, 116 of which were also Elf's top choice, for a concordance of 43.1%. It only took me a few minutes to get that number. OC, that is just one game, but IIRC the concordance rate between pros and top bots is around 50%.

Edit: Not that it is a good way to cheat, but apparently some people just do play the AI's top choice. If you are checking for that kind of cheating, you want to make it difficult to find a match, and then if you find a higher rate of matching than top humans have, you have some evidence.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

In game 1, Black chose Katago's top choice 37% of the time, so if Black is a high dan amateur player I don't see any evidence of cheating based solely on concordance with AI moves.

Just 4 games is really unlikely to be enough to determine if a player was using an AI or not.
But why the question?

_________________
North Lecale

These responses are really disappointing and make me sad as hell. I'm no sure if you people are blind, complicit or just plain dumb.

If you don't see any evidence, that's because you didn't bother to check.
Seriously, the moves are mostly (I'd say around 95%) LeelaZero suggestions. Do moves 103 to 109 in game 4 look normal? How about moves 244 to 252 in that same game? How about checking the moves against the LeelaZero that came with the previous version of Lizzie (you know, the one everyone has, the easiest to get)?

I don't care about punishing this person specifically, I honestly don't, we all make mistakes. What I do care about is how cheating is detected and handled, specially in tournaments where there's prizes. What really frustrates me is that this person is so blatant and obvious, and organizers (and apparently people in this forum) can't make up their minds. Check any pro game against LeelaZero and you see at least some divergence (I'd say at most 90%) and nobody plays the moves in exactly the same order.

I fear some people just checked parts of some of the games and once they saw a divergence in 1 or 2 moves, decided it was impossible to know for sure. This person was very blatant, and even after these games specifically were brought to the attention of organizers (and you guys), there's still doubt? How?

If it's this easy, I might as well start cheating in online tournaments. People are too easy to fool.

With the various AI programs available today, I don't think it's hard to cheat. Given the strength of widely available AIs, I think we've crossed that bridge.

So you can probably go ahead and cheat and win a tournament without getting caught. But why? I guess if it's for prize money, there's some incentive there. But there're probably easier ways to make money.

Tournaments are fun for me because you can test your strength, and get the real pressure of playing a serious game where the other person is trying to win, too. I don't think I'd get that fun if I used AI, because I'd know that the person playing wasn't me.

_________________
be immersed

@Sampi: you quoted me out of context. I just analyzed game 1 with KataGo and said that there is no evidence of cheating based solely on concordance with AI moves (I insist on the word "solely").

On the other hand, Antti Törmänen analyzed the game further and concluded that Black was probably a cheater. He did not just count the number of blue moves, but also noted that some surprising moves were considered by the AI, and that most of the game is high dan level while several moves are kyu mistakes.

I am ready to accept the verdict obtained after a thorough analysis by a skilled, experienced and impartial player like Antti.

And by the way, several months ago, I and other people showed Antti a game we found suspicious, and after the same kind of analysis he concluded there was probably no cheating, and we accepted his conclusion immediately.

I don't think it's necessary but if it will help you come to the obvious conclusion then here you go:





Both games are a few days after the games I already posted and it's unlikely the person cheated.

If the conclusion is obvious, why did you ask the question?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

For clarity, do you include my responses in this?



More information about the account also helps. If it's a new account for a KGS+ tournament that could plausibly be a strong Chinese amateur, who happened to have a ddk ? rank because he misunderstood automatch or timed out his first game then I'm less likely to think it's cheating than if it's an established stable ddk KGS user with hundreds of games.

Also, differential analysis with multiple bots is very useful, as I and Bill have said previously. If a player gets 80% of some matching metric with LZ v157, 83% with Elf, 73% with LZ 240, 84% with KataGo it's more plausible they are a really strong human player than if they get 95% with LZ v157, 60% with Elf, 64% with LZ 240, 72% with KataGo. A high match with the default network with Lizzie is even more useful information.

Has it crossed your mind that maybe we are smarter than you?



If you say that, it's because you didn't bother to read jlt's note. Of course he checked. Are you calling him a liar?



You don't care enough about it to avoid jumping to conclusions yourself. Is that how you want tournament organizers to act?

I am afraid, from your remarks, that you did not ask your question honestly. Perhaps you were looking for agreement to bolster your own conclusions, and willing to dismiss out of hand people who disagree with you. Frankly, your question was deceptive. It wasn't really a question. Some people might call that cheating. What do you think? That's a real question, BTW.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.