There's a reasonable chance the actual probability is nearer the larger side of these estimates. Because if there is sometimes any correlation at all between a match on one move to a match on subsequent moves, then the moves within a game are not independent, and we should expect comparable to the larger side. (e.g. if there are kinds of games/fights/sequences where each move is 85% and kinds where each move is 75%, and each kind of game is equally likely, then the overall probability for 98% is much higher than if every move in every game was uniformly 80%).



This.

Even if that shows 98% is a significant enough outlier that's not enough to convict IMO (of course if it's not an outlier then that's easy, case closed, less work, unless everyone is using Leela!). To do this well you need to analyse a decent number of games of people who have studied a lot with Leela and see how much of an outlier 98% is to that. It may be hard to find enough such people/games around 4d level. Carlo's offline games are a good start (if recent enough), but I want to know how many were analysed, not just a plural and result is 70-80%.

Update: Rather than going for a walk in the park at lunch I finished analysing moves 50-149 of my PGETC game . I scored 80% similarity and my opp* scored 90%. So we have the report at least 2 of Carlo's offline games are in the 70-80% range, and my data of [80,90]. 98 is looking less conclusive. I attach the spreadsheet I used for interest/verification. I note the sequence from move 93 to 136 would have all counted as copies under this metric were it not for 2 timesujis I played in the middle (most people don't get into byo yomi as earlier as I do). Next I might look at the stricter "top 1" instead of "top 3" metric.

* for ez4u, now ranked 3d, but a former 5d.

You are right, a spreadsheet error on my part!

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

I wonder if we'll get a comment from Metta himself anytime soon...

It's quite a complicated issue...

I really wonder why was only this game from the PGETC checked.
It seems a bit suspicious, considering it was perhaps the least impressive of his wins (he lost only to Csaba Mero and won against several 6d's and 5d's). Perhaps the reason is that it was the only game that someone appealed against (surprisingly, as it didn't change the outcome of the match, unlike other cases).

From Carlo's side, assuming the accusation is correct, I can understand why someone might choose to cheat when playing in a team. He's not playing just for his own sake and he knows that winning can have a great effect on his friends.. Staying or going down a league affects the whole next year.
(Still, no doubt, it's the wrong thing to do)...

Somehow I feel no problem with him being a referee in the EGC.
He'll be doing service to the EGF, and he'll be more obliged to prove himself...

How do you arrive at this?

@bernds:

The probability to play like Leela on moves 1,...,49 is p⁴⁹.
The probability to play differently from Leela on move 50 is 1-p.

So the probability to play like Leela on moves 1,...,49 and differently on move 50 is p⁴⁹(1-p).

In the same way, for each k, the probability to play differently from Leela on move k and like Leela on other moves is p⁴⁹(1-p).

Now, there are 50 possible choices for k (k=1,2,...,50), so multiply p⁴⁹(1-p) by 50 to get the probability to play exactly 49 moves "right" and 1 move "wrong".

Hmm. I have mostly escaped statistics in my education, and your explanation sounds plausible, but I still don't quite understand why we don't treat it as follows: if p is the probability that a player does not make an original move (0.89) and we assume independence, then the probability of the player not making an original move 50 times in a row is p⁵⁰ (which would come out to only 0.3%.

You are calculating the probability of making any nonzero number of original moves (including, for example, 13 original moves and 37 unoriginal moves). jlt is calculating the probability of making exactly one original move.

@bernds: That's also true, and doesn't contradict my post.

The probability to make all 50 moves like Leela is pⁿ which is about 0.3 %.

The probability to make exactly 49 out of 50 moves like Leela is np^n-1(1-p) which is about 1.8 %.

So the probability to make at least 49 out of 50 moves like Leela is pⁿ+np^n-1(1-p) which is about 2.1 %.

In my calculation I only used the term np^n-1(1-p) because the other one, pⁿ, is much smaller and can be neglected, since our calculation is very rough anyway.

First, thanks to Bill for the extremely thoughtful post on Bayesianism. I enjoyed it very much.

Standard hypothesis testing (a la Neyman-Pearson) can lead to trouble when placing too much importance on p-values. A small p-value indicates the data you observe is unlikely to be generated when your null hypothesis it true. This leads people to reject the null hypothesis. However, even with a low p value, your null hypothesis may actually be the most likely of a bunch of unlikely hypotheses that could have generated that data.

Let's consider three hypotheses:

1) The alleged did indeed use the computer for assistance during his tournament game.
2) It is a complete random coincidence that most/all of the alleged's moves mirror the highest value moves from Leela.
3) A less nefarious middle ground between 1 and 2: the user trained himself on the software, and has acquired the ability to mimic successful play by the computer, even if he doesn't know why all of these moves are good.


1 seems possible, given that the game was played by people on a computer in remote locations.

2 doesn't seem likely, for all of the reasons people are describing with probability calculations.

3 could be possible. A nice way to test this would be to observed how the alleged plays over the board. If he can sustain the same proximity of play to Leela in an in-person tournament where he doesn't have access to a computer, then that would lend weight to this hypothesis.


Finally... to be falsely accused of something is a terrible thing. I would be in favor of voiding the game given the evidence, but not banishing the person from Go existence.

Another thing that occurred to me: while I'm not sure how much difference it makes, the probability of "this game contains a 100 move sequence with X similarity to Leela" is greater than "this particular 100 move sequence from the game have X similarity to Leela". A matching sequence could be found at 40-140, 50-150, 60-160, etc, and while those aren't independent, the chance that one of three will match is greater than the chance that a specific 100 move sequence will match.

I did not read anything that made me think the arbitrators went fishing, but it's something else that we need to be careful about, and a question that would be nice to have answered: how were moves 50-150 selected?

_________________
Occupy Babel!

While arbitrary, selecting this stretch of moves doesn't seem unreasonable.

The very beginnings and endings are likely to be replicatable by many players.

By move 50, you are into the middle game, where "book learning" stops, and you would not expect play to be the same between a human and a computer. (or between any two separate entities, really)

Approaching move 150 is likely macro-endgame. Here too you are likely to see differences between a human and a computer (which is likely to play sub-optimally if it increases its win
probability).

I am not sure, but with this sort of training, won't the player learn to answer with "Leela's type moves" only to Leela herself? And when faced with an opponent with a different style than Leela, won't the similarity of his moves with Leela's move going to drop drastically?

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

@pnprog,

This is very similar playing exclusivly tygem for 2 years and then change the server or going to a real life tournament.

Seeing as I got 80% similarity in my last league game and I've never studied with Leela (and my opponent got 90%), it suggests a mid-dan playing a sensible game can easily get a high similarity. I wonder if the strongest players in league A have a lower similarity (Lukas Podpera 7d said on FB he is usually 70-80%) because they are stronger than Leela; also it would be interesting to check styles, e.g. I expect calm territory Artem is more similar to Leela than crazy fighter Ilya is. The game in question was played on 28th November 2017, Carlo played in the 2017 EGC in July and 3 Italian tournaments since. In the EGC he entered as 3d and won 10/15 across the main and weekend, gaining 50 GoR, a pretty good result but not as spectacular as you might expect from a 6d beater. The Italian tournaments suffer from being against mostly weaker players: I expect his opponents' Leela similarity would be lower as they play worse moves, and would his responses also be less similar or could he keep playing Leela honte style to answer their mistakes? There were just 2 of 15 >= 4d, he beat 4d, lost to 5d. So if you want to analyse his Leela similarity in offline tournaments within a few months of the Israel game against similar strength opponents then there's only 2 games to use. Not enough for quality stats.

For the record, I had made a small python script to estimate those probabilities, and the result is consistent with jlt method of calculation.

Here it is:

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

I agree that it's a reasonable range. The question is: was it the only range they considered? Because if they considered more than one range, that somewhat raises the chances of a significant similarity being found.

_________________
Occupy Babel!

For a kyu player to play moves that match 98% (or whatever the exact no. is) of a high dan bot that has beaten a pro already? That's way too unbelievable now isn't it?! Even pros have trouble matching move for move or even playing the same top few moves for that high a percentage of their moves.

Also, you guys might be mistaking very obvious real life situations that the judges and players there were able to witness, e.g. the accused kept leaving the table way too often. Because there's no way the guy would arouse suspicion had he just sat there the whole game.

The player is 4d EGF.


The game was played online.

No that much similar in fact, because when playing exclusively on tygem for 2 years, you won't be playing exclusively the same opponent like he did. At least you will play against various styles and various levels.

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!