“Decision: case of using computer assistance in League A”

[Javaness2]

Yes

[Bojanic]

Actually there are other PGETC players with several games almost completely Leela-like (even some famous and distinguished player). They are all cheaters? Or maybe in hundreds of PGETC games it is possible to observe these coincidences from time to time?

This is the main question we need to answer in the future.

First, others did it is not an excuse.

I have found several more games in which deviations histogram is close to Leela. In some short games, one player dominated another. Since it was mainly fight, there was lot of similar moves to Leela, but also some of the different moves.
I have one game I am very suspicious of, but in it some tenuki moves are different.

Those two Carlo's games are closest to Leela of all games. And since it is two games of one player, it is even more suspicious.

If you eant to prove that others cheated, it would be goodthat you make prošer analysis.

[Bill Spight]

As for the 98% matching evidence, you must understand that matching one of a bot's top three choices was chosen in order to generate impressive matching numbers, not through any theory of how a player might have cheated. (This motive may have been unconscious.) And restricting the possible matches to the fifty moves between moves 51 - 100 is also suspicious. In addition, it is confirmatory evidence instead of disconfirmatory evidence. IOW, it is not just unsound, it is crap.

I fully understand how the 98% came about. The choice implies a theory of how the player cheated. (I.e. sometimes choose 2nd or 3rd move to make it not too obvious.)

OC, one can come up with such a theory. But the reasons for the choice of that methodology should have been given in the original verdict and ruling. Sorry, but when people here have talked about how they might actually cheat, has anybody said, well, I'll pick one of the bot's top three choices? No. Picking the top choice is, I understand, done in cheating in casual online chess. Not in online tournaments because it's a dead giveaway. It may be done in FTF tournaments if the player thinks they can get away with it, but the physical evidence can be uncovered in those cases. Anyway, if picking the top choice is a dead giveaway, picking one of the three top choices is almost a dead giveaway.

Not that someone might not cheat in that fashion, particularly the first time. But this is a case of fitting the theory to the evidence. That does next to nothing to bolster the theory itself, and even less to support the charge of cheating.

Yes, it is limited. Yes, it may be wrong.

It's crap.

What people don't seem to appreciate at all is that this isn't a scholarly discussion about the quality of the evidence, but a decision about how Go will be in the future.

The two are not at odds. Not at all.

Making it almost impossible to catch a cheat (all the doubts piled up in this thread do exactly that) will only lead to more cheating and all pervading hypocrisy.

My view is that this is about making it possible to catch cheats.
Edit: And not just sow suspicion and distrust.

----

On a slightly different tack, today's top go bots have tactical weaknesses but excel in whole board judgement and what in humans would be called intuition. Humans are quite good at learning such things, largely through imitation. That is why I think that go will experience a flowering in the coming years. I would not at all be surprised in the pros 20 years from now are two or three stones stronger than the pros of today. And they will get there in large part through imitating bots. (By contrast with chess, where engines play differently from humans.) Defining cheating at go as playing like a bot is not only mistaken, it is counterproductive, insofar as it discourages players from imitating bots.

[Bill Spight]

Actually there are other PGETC players with several games almost completely Leela-like (even some famous and distinguished player). They are all cheaters? Or maybe in hundreds of PGETC games it is possible to observe these coincidences from time to time?

This is the main question we need to answer in the future.

First, others did it is not an excuse.

It is not an excuse. It is an indication that playing like Leela is not the same as cheating.

If you meant to prove that others cheated, it would be goodthat you make proper analysis.

(Spelling corrected by me.)

Equating playing like Leela with cheating is not a proper analysis.

[Bojanic]

Regarding top moves in Leela, it is very useful to see how the analysis changes after more variations. Sometimes it changes from A to B, then to another move. In most of times, suggestions change places, meaning that moves B or C xould have been A for a while.
Since Leela analysis is live, you can atop it at any moment.

[Bojanic]

Equating playing like Leela with cheating is not a proper analysis.

Since you quoted me, you are implying that I did it?

Since you obviously have not bothered to read my paper, on one hand you have two online games similar to Leela, and on another two live games with much lower level of play. Comparing those two games shows large difference in play. Draw your conclusion...

[Gobang]

Who are they, and which are the games you mention?

Yes, where do they live and who are their parents? Honestly, does anyone really care?

And why this fixation with Leela and comparing everything with Leela? There are other go programs, just as good or even better then Leela.

[theoldway]

Since you obviously have not bothered to read my paper, on one hand you have two online games similar to Leela, and on another two live games with much lower level of play. Comparing those two games shows large difference in play. Draw your conclusion...

What if later one founds a live game similar to Leela and an online game of a lower level (this latter is already available, maybe even the former)? Will you draw the same conclusion?

As many told you before, human ability can fluctuate so much, you need to consider thousands games of hundreds players in different settings to establish a solid method. A single game can tell you everything you want if properly choosen.

[Joaz Banbeck]

...I think that go will experience a flowering in the coming years. I would not at all be surprised in the pros 20 years from now are two or three stones stronger than the pros of today. And they will get there in large part through imitating bots...

I too think that go will experience a flowering in the coming years, but I think that those pros will get there by having implants and essentially being cyborgs. ( I look forward to this, although by that time I will probably be too old for a surgeon to take the risk )

When cyborgs become the norm - and they will! - the whole question of cheating with bots will disappear. We will look back on this thread as a quaint reminder of the days when people were unaugmented.

Not only will go experience a renaissance, but other fields will too: art and music especially. ( The greatest benefit to humanity, however, will be that nobody will wander out into traffic while looking down at their phone )

[Bill Spight]

Equating playing like Leela with cheating is not a proper analysis.

Since you quoted me, you are implying that I did it?

Here is what I quoted:

If you meant to prove that others cheated, it would be goodthat you make proper analysis.

(Spelling corrected by me.)

And you were referring to this quote:

Actually there are other PGETC players with several games almost completely Leela-like (even some famous and distinguished player). They are all cheaters? Or maybe in hundreds of PGETC games it is possible to observe these coincidences from time to time?

This is the main question we need to answer in the future.

You responded:

First, others did it is not an excuse.

I interpret "did it" as "cheated". But theoldway did not say or even imply that others cheated. He did say that they played like Leela and questioned whether they cheated.

Your comment suggests that you are taking similarity to Leela's play as evidence, even as a statement, that these other players "did it", i.e., cheated.

You continued:

I have found several more games in which deviations histogram is close to Leela. In some short games, one player dominated another. Since it was mainly fight, there was lot of similar moves to Leela, but also some of the different moves.
I have one game I am very suspicious of, but in it some tenuki moves are different.

Those two Carlo's games are closest to Leela of all games. And since it is two games of one player, it is even more suspicious.

It certainly sounds like you are taking playing like Leela as evidence of cheating. And in that context, you are exhorting theoldway to make a "proper analysis". To which I responded that equating cheating with playing like Leela is not proper.

Since you obviously have not bothered to read my paper, on one hand you have two online games similar to Leela, and on another two live games with much lower level of play. Comparing those two games shows large difference in play. Draw your conclusion...

I did read your PDF file, if that is the paper you are referring to. I was disappointed in how much of the paper was devoted to similarity to Leela's play. I also pointed out that you had found an important piece of evidence, the mistake that both Leela and Metta made. And I praised you for focusing on tenuki.

In earlier discussions I had emphasized the importance of the differences between play in presumably non-cheating games and in possible cheating games, and you seemed to agree that that is important. In your recent posts, such as the one I quoted, you seem to be laying emphasis on similarity to Leela's play, instead.

[Tryss]

Show us the game and your analysis, it can be an interesting example to study. What else is the sgf tag for here?

The game :



Winrate evolution graph :

WhiteWinrate.png (13.86 KiB) Viewed 11498 times

I wanted to attach the GRP analysis, but it say "The extension rsgf is not allowed.". I did the analysis with 5k playout and LZ #147 (if I remember correctly, but it's at least #145 or newer)


Note that :

1) the game was no komi, LZ consider 7.5 komi, so it's not totally accurate (but I don't know how to correct this easily)

2) there was inaccuracies after move 44, but considering they didn't drop my winrate under 97%, I considered them irrelevant

3) Leela 11 would give a different evaluation

4) as the game was basically over early, I had few possibility to make real mistakes.



Note that 3 & 4 cannot be used as an argument to prove that I didn't cheat with LZ. After all, if I used LZ, Leela 11 evaluation are meaningless, I could just have verified with LZ that my planned move don't loose too much winrate. And even if the game is over early, I could have cheated too, after all, I didn't make serious mistakes later.

[EdLee]

Since you quoted me, you are implying that I did it?
Since you obviously have not bothered to...

Please calm down. Re-read carefully what was said.

If you eant to prove that others cheated, it would be goodthat you make prošer analysis.

The typos suggest haste, and possibly strong emotions.
( Or a bumpy bus ride. )

Breathe.

[Bill Spight]

Since you quoted me, you are implying that I did it?
Since you obviously have not bothered to...

Please calm down. Re-read carefully what was said.

If you eant to prove that others cheated, it would be goodthat you make prošer analysis.

The typos suggest haste, and possibly strong emotions.
( Or a bumpy bus ride. )

Breathe.

As I discovered back in the 1980s, the internet is a hot medium, in McLuhan's terms. Back then, one of the most frequent online sentences was "You didn't read what I wrote." We can cut each other some slack.

[maf]

The game :

First of all, beautifully played and excellent work for 3 kyu. I would have guessed that white is not kyu for sure.

Since the game was 'over' so quickly for the bot, but not over until much later for a human, I think the graph is useless in this case and we cannot draw any conclusion from it. In other words, we do not know if you made mistakes after move 40, because you may have done many but they were not relevant enough. Does that make sense?

What we can tell is that within 40 moves, white made 3 (probable) mistakes. If we were to scale it up to 100 or 150 moves (which is not really statistically allowed), that's around 8 to 12 mistakes - a lot more than we saw in the infamous other games. Also, when even the 5th-best move has a winrate of close to 100, a cheater would not need to play the best move. So just from that I would say your game is no indicator of the methods being used so far being right or wrong. But you wrote most of that yourself already

[Tryss]

First of all, beautifully played and excellent work for 3 kyu. I would have guessed that white is not kyu for sure.

It's hard to guess the strenght of a player on a game alone. I mean, there is some indicators, but it can often be tricky. Here I think it looks like I played stronger than 3k because I dominated in this game. It's easier to play well when you've got momentum (but it's not necessarily easy)

Since the game was 'over' so quickly for the bot, but not over until much later for a human, I think the graph is useless in this case and we cannot draw any conclusion from it. In other words, we do not know if you made mistakes after move 40, because you may have done many but they were not relevant enough. Does that make sense?

Correct. And that was an important part of the point I wanted to make : winrate change don't exactly measure move quality

What we can tell is that within 40 moves, white made 3 (probable) mistakes. If we were to scale it up to 100 or 150 moves (which is not really statistically allowed), that's around 8 to 12 mistakes - a lot more than we saw in the infamous other games. Also, when even the 5th-best move has a winrate of close to 100, a cheater would not need to play the best move. So just from that I would say your game is no indicator of the methods being used so far being right or wrong.

Note that, while the first one (the "approach to the lower 3-4) was not considered by LZ, the two other mistakes where second choices by LZ. So before move #48, I only played one move that was not in the LZ top 3 choices, and only 4 that were not top 1. And after that, as you say, it doesn't really matter.

This was just a warning against the "similarity to bot play" metric without doing a statistical analysis of a significant number of games and without considering the state of the game.

Another warning : if a game is not lost by a blunder, you can always find a way the winner could have cheated to play exactly like in the game (Or a way to argue he didn't cheat, unless it's "always play top 1 choice by the engine"). That's very problematic and hard to avoid in this case, especially if you consider that the cheater try actively to avoid detection.