“Decision: case of using computer assistance in League A”

[AlesCieply]

Ales, servus! Actually the UK team is not my team. I am not a member of the BGA. I have not played a tournament in (guessing) over 20 years. I meet a tiny handful of UK players only a very few times a year in connection with T Mark Hall's legacy (though when he was alive I did get some news relayed from him). I know nothing about the EGF event, or the EGF personalities.

None of that strikes me as peculiar behaviour either individually or in combination, so I find the fact that you find it odd... well, odd. And odder yet that you seem to think it matters enough to post here. I'm not at all offended BTW - I'm amused, so thank you for this morning's smile!

Well taken, John. Thanks for the explanation, I did not know you were that detached from the UK go scene. It was also a kind of weird way to say you "hello" from me here, I admit maybe not so well thought out. The problem with discussions on internet is that mostly one knows nothing about people on the other side of the wire. You might be the only one on this topic (besides Lukan) who I met in person. If not for the CM case, I would probably not have ventured here as normally I prefer to stay out of the discussions on internet.

[Bill Spight]

Yes, the odds against all of those things happening are quite large. That is true of many coincidences, as well.

What do you feel are acceptable odds against which an innocent player must not be found guilty? 10:1? 100:1? We're so far above those numbers at this point.

May I refer you to Ken Regan's work on detecting cheating at chess. Links are in this post: viewtopic.php?p=228848#p228848

http://www.duhaime.org/LegalDictionary/ ... ities.aspx makes it clear that for civil cases, it only needs to be that the defendant is more likely to be guilty than innocent to rule against them.

I have been a certified contract bridge director for a long time. The laws of contract bridge are based upon a civil law model, and the aim is to restore equity after an infraction. One way to do that in go would be to replay a game. Cheating is a more serious matter, and requires investigation and serious sanctions at the level of organizations. A criminal law model is more appropriate.

[Simba]

Yes, the odds against all of those things happening are quite large. That is true of many coincidences, as well.

What do you feel are acceptable odds against which an innocent player must not be found guilty? 10:1? 100:1? We're so far above those numbers at this point.

May I refer you to Ken Regan's work on detecting cheating at chess. Links are in this post: viewtopic.php?p=228848#p228848

Thanks, Bill. I'm actually already fairly familiar with Regan's work having browsed chess forums in the past.

Forgive me - I'm not too eager to trawl through 29 pages of content again to check - do you recall by any chance whether or not someone has already applied Regan's methodology to any of Carlo's games?

[Fenring]

but its 200 moves to check versus one, not polynomial versus exponential add to that he have just to deeply check the moves moves who are not in in top3 leela's choice(which in a classic game is only 30% of the moves).
On a small amount the difference P vs NP is irrelevant.

Your argument is flawed in the following two fundamental and irretrievable ways:

1) You have completely misunderstood what is being discussed here. We're not talking about any sort of top 3, we're talking about Leela 0.11 being unable to find a sequence except when it's explicitly shown it, i.e. ANY sensible amount of analysis with Leela 0.11 won't find that this is a good sequence (Uberdude did 100k, and I'm sure more could be done). Leela 0.11 thinks that Carlo's move is 'game over, you just lost and threw away your victory' until shown the refutation. And the context is to do with the original poster's remark that Carlo was planning on using this knowledge to claim he wasn't cheating (when in fact he was, just with Leela Zero).

2) You've assumed that there is such a move in every game (your 200 vs one comment). Leela is far stronger than this. And even if it WAS the case, that you knew a game with 200 moves had one such move, you'd have to test every move, with multiple responses, and find the refutation. That is far, far more work per move than simply being told where it is and checking it.

Go back and read what is written more carefully before you reply again; trying to apply an old context (the 98% thing) to a new situation is not relevant and makes you look foolish.

[cut] someone who dont look the games,and dont know if carlo cheated,but think probably yes [cut]

When i see this is the evidence for his opponent, i begin to believe he is not guilty.

Then isn't it good that personal feelings have nothing to do with logic and analysis? I don't care if you like or dislike me; this isn't about that. At least do the rest of us the courtesy of looking at the games and reading things properly if you're going to try to voice an opinion.

i think i read thing properly, so to answer to your two remarks which for me are the same:
The very difficult problem here is to create a situation where the best move of Leela Zero is a game over for Leela 0.11.
But to find it in a existing game is a completely different task.
Because its just checking games.
So i open the game with the the 200 moves Carlo played,on this 200 moves i can already eliminate the 60-70% who are in top3 of Leela, and i check the other with Leela Zero to find on which moves leela zero and leela 0.11 disagree,and i have my candidates for a deeper analysis.
If a move like that doesn't exist,i go to another game from Carlo, until find such a move played by Carlo.
The scamer do not have to find the refutation, the move is already played.

So for me when you say "its easy to check what he say,but for him he would be very hard to create this statement", its really flawless, the scamer have a really limited number of moves to check.

[Simba]

i think i read thing properly, so to answer to your two remarks which for me are the same:
The very difficult problem here is to create a situation where the best move of Leela Zero is a game over for Leela 0.11.
But to find it in a existing game is a completely different task.
Because we are just checking games.
So i open the game with the the 200 moves Carlo played,on this 200 moves i can already eliminate the 60-70% who are in top3 of Leela, and i check the other with Leela Zero to find on which one both disagree,and i have my candidates for push through the analysis.
If a move like that doesn't exist,i go to another game from Carlo, until find such a move played by Carlo.


So for me when you say "its easy to check what he say,but for him he would be very hard to create this statement", its really flawless, the scamer have a really limited number of moves to check.

I'm sorry, but you're still very much misunderstanding what's going on here. I could try to rephrase things if you would particularly like me to? I'm not sure how invested in this you are given that you hadn't even looked at the games when you last posted.

As it happens, the move in question (156) was Leela 0.11's third choice, according to https://www.reddit.com/r/baduk/comments ... d/e0c309j/. The ranking Leela 0.11 gives to the move is irrelevant for this discussion.

[Fenring]

Look at the game is useless for me: i'm 2k,so i can't judge the difference betwen a 4d-6d-bot lvl.

What i dont understand is why it would be a very difficult task for a malicious person to produce such a statement?(Np-problem)
All he have to do is to check the Carlo's moves where Leela 0.11 and Leela Zero disagree? check the moves "Leela 11 says it is very bad move but Leela Zero says it is super so he plays it", i dont know how many of them exists,and on which metrics it is based(top 3 choice,Winrate variation) but it doesn't matter,we are clearly not in a situation "impossible to find,easy to check" like PvsNp problem.

[Javaness2]

This post from reddit strikes me as a crock of shite.
Yes, CM could have cheated, but then why tell people about it. Is the allegation that the entire Italian squad was complicit, if so then doesn't that strike you as slightly beyond belief.
Yes okay, Leela 11 and Leela Zero differences exist, but so what? Why would CM want to cheat with the obselete Leela 11 in the first place? Why not pick 1 of many strong LZ networks?
CM certainly played very strongly this game, some proper computer analysis would be welcome, taking into account timings.

[Bill Spight]

Thanks, Bill. I'm actually already fairly familiar with Regan's work having browsed chess forums in the past.

Forgive me - I'm not too eager to trawl through 29 pages of content again to check - do you recall by any chance whether or not someone has already applied Regan's methodology to any of Carlo's games?

As far as I know, no-one has come close to doing so.

First, Regan distinguishes three kinds of evidence, physical, behavioral, and statistical. Without physical or behavioral evidence he requires extremely good statistical evidence, equivalent to a difference of 5 standard deviations.

Second, go does not yet possess the tools that chess has to do the job, statistically. We are unable to reliably measure the difference in a player's level of play in a single game or group of even games and his current level of play in games with presumably no cheating. We are unable to reliably assign a difficulty rating to individual plays. We are unable to reliably order the choice of individual plays. (Yes, top bots do evaluate plays, but they do not require accurate evaluations to beat humans or the previous version of themselves.) Ten years from now, we may have good enough tools, especially if we try to develop them, but not yet.

Now we are able, to some extent, to rate the difficulty of individual plays. goproblems.com does rate the difficulty of individual problems, so in theory we could rate individual plays themselves, using only human judgement. Furthermore, players may be able to judge the difficulty of plays. Such judgements are of course imperfect, but we can gain reliability by combining the ratings of individual players. Of the 50 plays examined in the Metta-Ben David game, I judged that only 12 were difficult enough to be relevant to the question of cheating. Using similar but different criteria Bojanic found 6 important plays by Metta between move 50 and 105, a smaller range of 28 plays by him. He and I agreed on 4 of those 6 plays. My guess is that with even several judges we would have consensus on at least 3 plays. Uberdude has also tried his hand at evaluating the difficulty of individual plays.

Third, I think that the choice of plays itself offers behavioral evidence, not of a statistical nature, but relying upon go analysis. Stanslaw Frejlak made a go analysis of the Metta - Ben David game. (See viewtopic.php?p=228924#p228924 ). Also, in your game vs. Metta, if Black 157 descends, White can win the resulting semeai. The play is pretty much a one lane road, so is within the capabilities of a European 4 dan. OTOH, we know that in the heat of battle mistakes are made. Such go analysis is like expert testimony in court, not infallible, but not worthless, either.

[Bill Spight]

As it happens, the move in question (156) was Leela 0.11's third choice, according to https://www.reddit.com/r/baduk/comments ... d/e0c309j/. The ranking Leela 0.11 gives to the move is irrelevant for this discussion.

Precisely. As is its evaluation of the play. The anonymous accuser has created a diversion.

[Bill Spight]

The very difficult problem here is to create a situation where the best move of Leela Zero is a game over for Leela 0.11.

Why bother? Let's not get distracted by shiny baubles.

[Gobang]

Since people have been free with their opinions, to the extent of calling others liars, etc, then I will be free with mine.

1. The organizers of the online tournament made a mistake when they reversed the Metta decision. They were faced with 3 choices, uphold the decision, reverse it or cancel the tournament. Given the situation I feel that cancelling the tournament would have been the correct choice.

2. It is a mistake for someone facing such serious allegations of cheating to work as a referee in a major tournament. It is not likely to lead to anything good.

3. Any adjudication in a tournament should be done independently and impartially and free from any meddling and or pressure from the organizers of other tournaments.

4. No cash prizes or ratings should be awarded for performance in online tournaments.

5. This should not be about any individual person, regardless of what he or she may have done. This issue is much bigger than that.

[Jan.van.Rongen]

As it happens, the move in question (156) was Leela 0.11's third choice, ...

What does that show? Nothing IMO.

For the 5 alternatives White has for this move (h19, J-19-J16) 3 win the game if my analysis is correct. J18 played by Metta is a well known shape move. It is not even too difficult to see that it works when you notice the dame zumari of Blacks bigger dragon.

You are a 3 dan who lost to a 4 dan. What's wrong with that? Why put up a show how you let your team down etc. ? Your only "evidence" that white cheated is an unnamed person that made some remarks on reddit.

We are in phase 3 of a classical witch hunt. In phase 1 a very biased analysis seems to "prove" that the victim is guilty. Then in phase two that biased analysis is debunked by an expert, but the original hunters don't really study that but just go ahead and produce new "evidence". That is called a Gish Gallop, it is the "yes but" phase.

When that doesnt work anymore we get the "I have heard from a reliable source" phase - and that is phase 3 in which this witch hunt is now. The phase of disinformation, propaganda and fake news.

So I have a question for Ales Cieply who wrote the original "98%" analysis. That was debunked in much detail by Metta's PhD advisor Francesco Morandin, and I have never seen a detailed reply from Cieply to that report from 23 April 2018. So I have a simple qusetion: where is your reply?

[Bojanic]

There is one important point that we have not discussed in this topic.
That is how big difference is between 4d and 5d, and 5d and 6d.

From my experience, I started playing at age of 14, and after playing for 4 months some 5-6 quick games a day with my brother, we became some 5k. (that is 1k / week). Only with playing.
After that, since I did not have proper teacher, I had to work more on theory and read books, and it took me some 1,5 years to reach 1 dan (total of 2 years).
To reach following categories, it took me a year for each dan, and I reached 4d at some 19 (5 years of studying). It could be faster if I had coach, more books, etc.

Next, there was great difference for me between 4d and 5d. That was difference to reach top players in country.
After studying with them, it was necessary to read even more books. I had to move to stronger, Japanese literature, and since I don't know to read it, it was difficult to understand. Finally I realized simple recipe - player who plays last in diagram is not worse in situation, it is draw or he is better.

At peak, being strong 5d (2523) which some might say is pretty close to 6d, I realized just how far it was:
by looking at players with greater rating, I spotted that all of them had either spent some time studying go on Far east, or were from countries with several 6d+. (same as today, btw)
http://www.europeangodatabase.eu/EGD/Find_Player.php (sort by last column)
That meant I would have to spent lot of time studying, playing on strong tournaments, travelling to Japan - basically devoting my life to that for more than a year, which I could not do since I started to work, and had no support from Go Federation. And for that level, you have to be strong at all segments of game, you can not just count on your talent at one segment, it is necessary to study everything. And all that work for some mere 50 rating points.

That is why it is no surprise that so far there was only around 100 players from Europe who reached 6+d level. Of them, majority spent months in Far east studying go, or were trained in strong local centers.

Therefore, when we have a case that someone jumped from 5k to 1k quickly, sure, it is easily possible.
From 1d to 3d? Yes, even without too much reading.
And that someone who was 4d plays like 6+d?
Sorry, sorry, sorry. It is just insult to European best players who kicked their ass of studying to become that strong.
Please bear that in mind when we discuss this case. Lot of work has to be put into it, and it can be easily shown.

[Bojanic]

Of the 50 plays examined in the Metta-Ben David game, I judged that only 12 were difficult enough to be relevant to the question of cheating. Using similar but different criteria Bojanic found 6 important plays by Metta between move 50 and 105, a smaller range of 28 plays by him. He and I agreed on 4 of those 6 plays. My guess is that with even several judges we would have consensus on at least 3 plays.

Bill,
in Metta-Ben David game, I estimated that middlegame is from moves 45-105, which is 60 and not 50 moves. As you are used to hear on forums - why don't you read what was posted?

In middle game section, we agreed on all 4 important moves, but please not that I also analyzed sequences between them.
Regarding additional plays you mentioned, I agree that they were important, but they are IMO endgame+fighting important moves, and therefore have less strategic value then middle game moves. I have no objections on finding important moves in endgame too, you can do it easily if you are interested. BTW please note that 2 moves you added were also Leela's top choice.
And we have also discussed move 137 from the same game. It is even more important than moves you added, since heavy fighting erupts after that.

So, if you are willing, you can send me list of important endgame moves from 3 games I analyzed (since game with Kim was finished earlier), I will make my list, and we can work on them on same way as middle game moves.

[AlesCieply]

So I have a question for Ales Cieply who wrote the original "98%" analysis. That was debunked in much detail by Metta's PhD advisor Francesco Morandin, and I have never seen a detailed reply from Cieply to that report from 23 April 2018. So I have a simple qusetion: where is your reply?

Jan, it would be good if you checked your records. I have not written the original "98%" analysis. I even did not have Leela installed at my PC at that time to do any analysis. I did start my own analysis (a completely different one) at the time the case reached PGETC appeals committee.