“Decision: case of using computer assistance in League A”

[Uberdude]

I independently analaysed the game to find the Leela similarity metric: I got 98% for Carlo (aka 49 out of 50 matched) and 80% for his opponent Reem (aka 40 out of 50 matched). Some people on facebook were reporting 93% for Reem, which would indicate this was the kind of game with many obvious moves so 98% is not damning, which I did not reproduce. Some caveats: Leela (I used version 0.11 which was released a month or two before the game in question) is not deterministic and can change its preferences as you let it analyse longer (I was usually around 30k - 50k nodes), so someone else could get different results. My feeling from watching the analysis is that 80% could go up a bit but not as high as 93%, also an odd number is impossible with a denominator of 50. Also does anyone know how strong Leela 0.11 is under these or other conditions?

In addition, Stanislaw Frejlak (Polish 6d) made a good analysis of the game:

I analysed the game against Reem Ben David. Tl;dr from what I saw I would not accuse Carlo and I wonder where the 98% claims come from.

I know that the methodology I used might be not sufficient. I hope the official methodology was much more thourough and sofisticated. And also that this game is not the only evidence for such a decision.

And first of all, I find the rumours about Reem getting 93% completely ridicolous (Reem didn't play like Leela at all). Ok, let's go.

Thorough description of my analysis of the game:
Firstly I watched the game on my own trying to guess next moves. Then I uploaded it to Leela looking at her suggestions. Many of the moves in this game are really obvious and I had the same first intuition as Leela and so Carlo did. All other moves I checked giving Leela time to calculate from 30 000 up to 100 000 nodes. After that time Leela seemed to be decided not changing her views anymore.

Of course the opening is not the best part to analyse. Anyone studying with a program can learn its josekis. Anyway I started from the very beginning. So, the first move I found strange was 11 and Leela is really suggesting it. 13 I would rather push through but I checked weiqi.tools and pros like both options. 16 is a mistake, Leela agrees. 17 is strange, Leela agrees, but still it's an opening, still doesn't really matter. 29 is not played by pros at all. Leela prefers it a bit at F14 but she says Carlo's move is also good.

Josekis finished, we're entering a middle-game. Move 33 I found really strange (even more than white's 16). I thought that maybe this is this Leela style, but nope, Leela doesn't even think of this one. Of course she peeps C14. Everyone peeps... 37 is interesting. Probably many people would descend without thinking as Carlo did. But Leela suggests peeping! 38 is slow... My Leela prefers F3 for black now. Of course Carlo jumped, every human jumps. For 43 I considered G13. Leela prefers Carlo's move though. To 45 Leela approves. I also do. 51 probably I would not play. I didn't know where to play. Maybe I would try K16 starting a fight which is bad for black. Typical for my style... Leela suggests the move Carlo played. 52 is crude. 53 I'd consider pulling back, Leela and Carlo prefer hane. 55 I would connect at M17. Leela would push at M15. Carlo bumped at K17 which I find too simple. With 59 fun starts. In a fast game probably most of us would jump at J11. Leela also would but after some time she found E13 and realized it is very good. Carlo also played E13. It's the same with 61, not really obvious to exchange it but they both like it.

Ok, then white cut off two stones which surprised me. It's very slow, Leela agrees. From this point on, the game is undoubtedly good for black. White tries to surround the smallest moyo in the world with his thickness while black is occupying four corners. It makes the analysis harder, because black just needs simple moves. 75 they both like. However, Leela would tiger after it. Carlo of course extended as every human would. 79 Leela doesn't push through. I wouldn't either. Carlo did. 84 is super slack. Darn, it wasn't any software what decided a result of this game but white's moves like 16, 70, 84. Black's 85 I find strange. Why so calm? I thought Leela will recommend it but she doesn't! She plays a checking extension like a normal go player. But Carlo is playing safe knowing he's ahead. 87 they both like. Yeah, it's big.

After 96 everything seems more or less settled and Black's ahaed. Now black tigers twice which looks too bad (aji-keshi). Knowing that computer softwares like to play stupid exchanges (u remember Lee vs. Alpha game 2, moves 117 and 150? ^^), I expected it to be a Leela's suggestion. It is not. Firstly Leela wanted to keima at N14. Then she changed her mind to K13. The same after the tiger exchanges. However, Carlo plays N14. Move 105 I think I would eat a stone. Carlo blocked. Leela would extend at R13. Move 111 Leela answers to white's move. Carlo decides to take two stones. Funfact: after atari on two stones Leela would love to go back to O13. Carlo doesn't. Move 117 - in the first instinct I'd triangle. Leela considers connecting. But then she saw a wedge. True, wedge is powerful. Carlo simply triangled though. 119 they both peep. Good move. Then however, Leela doesn't think of keima [Uberdude: in my analysis Leela did find keima]. Of course she jumps. Carlo plays keima though. 123 Carlo attaches, and so Leela does. I didn't see it. It's really powerful. 125 Leela pushes at P12. Carlo turns instead. 127, 133 they agree. 135 Leela would just connect. The sequence starting from 139 is powerful and they both play it. But it feels natural also. 151 Carlo calmly protects his shape. Leela prefers to peep at N9.

Conlusion: moves 11, 29, 51, 59, 61 and 123 seem more or less suspicious. On the other hand moves 33, 37, 55, 85, 97, 117, 121 doesn't seem to be Leela-style. In general the game was simple and good for black from early middle-game so it can hardly provide strong proofs.

If we have no official study published I encourage others to do such analysis on your own. When we compare our results, we'll probably learn more.

[Uberdude]

Here's a graph of the binomial distrubution jlt was calculating, with probability of success (aka playing top 3 move of Leela) set as 0.9, 50 trials (for one player in moves 50-149) and getting 49 of them right (2.8%). There's quite a few simplifying assumptions here, e.g. my suspicion is 0.9 is a too low estimate of the success probability for a player training with Leela, and in a simple game where he gets early lead in mid game and opponent doesn't make it crazy complicated (as lightvector points out this game type means probabilities are not independent). Anyway, visualisations help; x axis is number of matching moves, 49 is highlighted, y axis is probability of that outcome. We can see the standard deviation of this distrubution is about 2, so 49 is a bit under 2 sds from the mean.

binomial.PNG (106.77 KiB) Viewed 16684 times

And same again if we assume probability of success is 0.95, here although 47.5 is the expected value we can see 48 is actually the mode and getting 49 is totally unremarkable.

binomial95.PNG (111.2 KiB) Viewed 16684 times

From https://homepage.divms.uiowa.edu/~mbogn ... s/bin.html if you want to play

[quantumf]

It's unfortunate that the first instance of a sanction for bot-based cheating (that I know of) is based on a statistical analysis, that so far at least is somewhat unconvincing. I wish it had been more clear, e.g. caught with a phone or something like that. There will be no way of knowing for sure unless the player admits it.

However, it was just a question of time before this happened, and now that it has, it's time to figure out how to deal with it going forward. I'm not clear that Leela itself is equipped to provide data about misuse, partly because it was not coded as such, partly because it just isn't that strong. Following it's suggestion, at least against a 6d, is certainly not guaranteed to result in a win (this may depend to some extent on the hardware involved). In contrast, in Chess, following Stockfish suggestions will always beat the top player in the world.

In this regard, the approach that chess.com take may be instructive. Of course, they are a private organization, and can use their their terms and conditions to enforce banning based on a statistical likelihood. IGS (the platform used the Pandanet games) and other servers may want to consider approaching chess.com and asking for help on implementing a cheat detection system. I'm not clear whether chess.com will help, as presumably their cheat detection system is part of their USP, but there's no harm in trying. Presumably all go servers will now be incentivized to deal with the issue.

https://www.chess.com/article/view/ches ... -detection
https://www.chess.com/blog/DanielRensch ... n-chesscom

[Uberdude]

I computed a new stricter metric "similarity to Leela's top 1 move". Carlo / Reem went from 98% / 80% with top 3 comparison to 72% / 54%. Uberdude / Norway went from 88% / 80% to 62% / 54% (I had previously said Norway on the top 3 was 90%, this was a mistake from double counting a move on the boundary of the 2 batches of work). I don't know why they picked top 3, I imagine the reasoning was a cheater might not always pick the top 1 move to escape detection and could still gain an advantage from picking top 3 and 3 sounds quite a small number. But as we see top 3 gives high similarity for many games. 72% similarity of top 1 move against a context of [54,62,54] doesn't scream cheat at me (more data obviously better to learn what is typical distribution of this top 1 metric), nor is 72 such an emotive number. The usual "Leela is not deterministic so your results may vary a bit" caveat applies.

Spreadsheet attached for transparency.

[Javaness2]

Don't you need some additional metrics, a la Ken Regan - naughty link: https://www.cse.buffalo.edu/~regan/pers ... cleKWR.pdf
I don't see how to make a good test for ELO (GoR)
I don't see how to make a novelty detector - probably impossible
Is it worth adding a companion to Leela match such as GnuGo match? (i.e. this move was probably sooooo obvious, because even GnuGo would play it)

Besides PGETC I suppose that this event is worth looking at for reference data http://www.europeangodatabase.eu/EGD/To ... n=14713996 , April to August 2017

[Knotwilg]

I agree the game should be forfeited and replayed. The punishment is too harsh for what's merely confirmed suspicion.

[Uberdude]

Oh yes, you certainly need more than my little example, but I see the burden of proof being on the prosecution: they need to show that 98% is an outlier of sufficient magnitude, whereas if I come along with the first game I happened to test with fairly high numbers of the same metric then it's similar to how 1 counterexample can disprove something but you need a lot more evidence to prove (or at least convince of sufficient confidence, we aren't dealing in mathematical absolutes). JeanSebL made a good point on facebook that Leela's policy netwrok has been trained to predict human moves, and is better at it than humans. So playing lots of moves that Leela likes is not a good indication of cheating, but playing lots of moves that Leela's policy network gives low weight to, but then the MCTS ends up liking (a-la AlphaGo's 5th line shoulder hit in game 2 vs Lee Sedol) is much more suspicious. Someone also analysed the game with Zen 7, and Carlo also got 49 out of 50 in Zen's top 3. So obviously he used both Leela and Zen to cheat . In my spreadsheet I marked really obvious "only moves" (erring on the side of not classifying) like capture after atari so those could be excluded, and using GnuGo to find these would be a more objective way.

[mhlepore]

The binomial calculation requires that *each* of the N observations be generated with probability P, but we know that some of the moves are trivial, and others are quite complicated. So it seems we can't use the binomial distribution - at least in the strict sense - to calculate a stretch of 50 moves.

Plus, aren't we really only interested in a few moves and not the long sequence?

[jlt]

The binomial calculation requires that *each* of the N observations be generated with probability P, but we know that some of the moves are trivial, and others are quite complicated.

IMO this doesn't affect the conclusion much.

Assume that it would be normal for a 4d player to agree with Leela 90% of the time, i.e. on 45 moves out of 50. As I said earlier, the probability to agree with Leela on 49 moves out of 50 is about 2%.

Now, suppose that typically, 10 moves out of 50 are not obvious. This means the probability that a 4d player agrees with Leela on a given non-obvious move is p=0.5. The probability to agree with Leela on exactly 9 out of 10 of these non-obvious moves is 10p⁹(1-p), which is about 1%. This is not small enough to punish, otherwise you would punish an innocent at each tournament.

More interesting would be to analyse Carlo Metta's win against a 6d opponent, and check similarity with Leela, like the attached game.

[mhlepore]

...
Now, suppose that typically, 10 moves out of 50 are not obvious. This means the probability that a 4d player agrees with Leela on a given non-obvious move is p=0.5. The probability to agree with Leela on exactly 9 out of 10 of these non-obvious moves is 10p⁹(1-p), which is about 1%. This is not small enough to punish, otherwise you would punish an innocent at each tournament.
...

In my opinion this drastically overstates the likelihood of a human thinking like a strong AI at these critical junctures. My understanding is that these AI moves are not simply "non-obvious," it's that they are completely outside the box.

[Uberdude]

In my opinion this drastically overstates the likelihood of a human thinking like a strong AI at these critical junctures. My understanding is that these AI moves are not simply "non-obvious," it's that they are completely outside the box.

Nope, bear in mind we are talking about regular Leela here (I used version 0.11), not Leela Zero -- LeelaZero was much too weak back in November when this game took place. Did you read Stanislaw's description of the moves in the game? Or could you tell us which of Carlo's moves in that game are "completely outside the box" Leela style (if anything I think it's white who played more funky moves e.g. o6, black was mostly a solid territorial style, with an early AlphaGo-style 3-3 invasion which regular Leela doesn't even dream of (but Leela Zero probably loves it)). I'm not even clear this Leela is super strong, maybe 5-6d overall (some bits strong, but iirc messes up 25k nakade too)?

[jlt]

In my opinion this drastically overstates the likelihood of a human thinking like a strong AI at these critical junctures. My understanding is that these AI moves are not simply "non-obvious," it's that they are completely outside the box.

Stanislaw Freijlak mentioned moves 51, 59, 61 and 123. A human wouldn't find these moves at first thought, but do you think these moves are really outside the box? If they are, how would you measure objectively their unlikelihood?

(Edit: posted without having read Uberdude's message above)

[Uberdude]

Here's a diagram for 51. Wacky? Outside the box? I think not. One space checking extension against an unsettled group. Stanislaw's idea of k16 (which he admits is probably bad) is wacky to me. What else to do here? Well there's d2, but that's more like early endgame. Could add a move to top right, but it's a 3-3 so that's not so big. Middle left group is not totally safe so could add a move to it, but where? Pincering the top group to keep it weak means white will come out and you can probably defend that group naturally whilst developing the top side. I've kind of run out of ideas but if you pressed me for another it would be some reduction move in the L8 vicinity.

When I did the analysis the first time I put down L17 as Leela's #1 choice. In fact in this position when I let it run for 300k nodes it decides L17 is #2 and e13 is #1 (by 0.14%). So Leela seems to see e13 as an important good exchange in that position. I can understand that: one of the first things my eye was drawn to in this position was the f12 weakness comboing with the f14 push and e13 attempts to fix that in sente, but my worry was it might not be sente (Leela thinks it is). In the game black only played e13 for 59 once white had jumped out so that weakness becomes more obvious.

$$Bcm51
$$ ---------------------------------------
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | X X . . . X O . . . . . . . . . . . . |
$$ | . O X X X O . . O . 1 . . . . . X . . |
$$ | . O O O X O . O . , . . . . . , . . . |
$$ | . . . . O X X . . . . . . . . . . . . |
$$ | . . . . O . . . . . . . . . . . . . . |
$$ | . . O . . X . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . X . X . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . O . O . O . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . O . . . . . . O . . X . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . X X X O . . , . . . . . , . . . |
$$ | . . X O O X O . . . . . . O . . X . . |
$$ | . . . . X O . O . . . . . . . . . . . |
$$ | . . . . . . O . . . . . . . . . . . . |
$$ ---------------------------------------

Click Here To Show Diagram Code

[go]$$Bcm51
$$ ---------------------------------------
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | X X . . . X O . . . . . . . . . . . . |
$$ | . O X X X O . . O . 1 . . . . . X . . |
$$ | . O O O X O . O . , . . . . . , . . . |
$$ | . . . . O X X . . . . . . . . . . . . |
$$ | . . . . O . . . . . . . . . . . . . . |
$$ | . . O . . X . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . X . X . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . O . O . O . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . O . . . . . . O . . X . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . X X X O . . , . . . . . , . . . |
$$ | . . X O O X O . . . . . . O . . X . . |
$$ | . . . . X O . O . . . . . . . . . . . |
$$ | . . . . . . O . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

[mhlepore]

In my opinion this drastically overstates the likelihood of a human thinking like a strong AI at these critical junctures. My understanding is that these AI moves are not simply "non-obvious," it's that they are completely outside the box.

Nope, bear in mind we are talking about regular Leela here (I used version 0.11), not Leela Zero -- LeelaZero was much too weak back in November when this game took place. Did you read Stanislaw's description of the moves in the game? Or could you tell us which of Carlo's moves in that game are "completely outside the box" Leela style (if anything I think it's white who played more funky moves e.g. o6, black was mostly a solid territorial style, with an early AlphaGo-style 3-3 invasion which regular Leela doesn't even dream of (but Leela Zero probably loves it)). I'm not even clear this Leela is super strong, maybe 5-6d overall (some bits strong, but iirc messes up 25k nakade too)?

I'll defer to you, since you are stronger than me, when it comes to describing which moves are funky and which are not. Maybe outside the box was not the best way to describe my basic feeling that
1) it is hard (at least for me) to apply intuition to every single move made by a reasonably strong AI program,
2) there will be moments in the game where finding the next move is hard, and therefore
3) it seems unlikely that two players - even with similar tendencies - will play identically at all of these moments.

Just my opinion, and it is intended to apply to the general issue of humans mimicking other strong players (human or AI) and not to any one specific game.

[Uberdude]

3) it seems unlikely that two players - even with similar tendencies - will play identically at all of these moments

I agree, identically in all these moments is unlikely. But this 98% similarity headline is playing Leela's top 3 choices which is much broader. With only top 1 in this game black was 72% or 36 out of 50. So 14 moves different, or -4 correct out of the 10 hardest moves we were focusing on.