“Decision: case of using computer assistance in League A”

[Gobang]

This is biggest scandal in European Go I can remember of (and I play for 30 years).
It has everything:
- cheating in important game,
- referee in most important tournament involved,
- political pressure to influence referees,
- and now, even internet bots, who attack people who think differently.
All things separately unheard of now, not to mention combined.

To add cherry on the cake, funniest thing is that bots are so bad, that most of their messages actually work against their goal.

You have a point. The situation is indeed stranger than fiction. If this was written as a soap opera story line it would be labeled poor because it is so implausible.

You are right about most of these things being unheard of, but "internet bots, who attack people who think differently." are common and have been for many years.

I agree that some messages that have probably been intended to be in favor of the person accused of cheating have indeed worked against that goal. It is hard to tell if that was the intention to begin with or not though.

[Bill Spight]

Just seems like it'd be easy to circumvent any cheat detection method we come up with. Maybe I'm not thinking creatively enough.

I think that you are being realistic about online cheating. Without physical or behavioral evidence cheating is very difficult to detect, as Regan points out. An intelligent one time cheat may be next to impossible to show, but if the player continues to cheat over time, the probability of sequential success diminishes exponentially, at best. Edit: At best, from the cheater's perspective, I mean.

[Bill Spight]

Bojanic has spoken of, and written statistics about, forced without explaining it. Bill, you do the same, speaking of sequence that is a unit, one lane road, sente sequence, forced, fairly predictable, important play, kikashi sequence - all without explaining these phrases.

There is no such thing as an easily identified unit outside a local endgame with a verified sente sequence. E.g., one might interrupt several of those "units" and play at Q17.

Statistics can never provide statistical evidence for anything if they rely on undefined terms. If statistics shall measure "forced", then completed research and general agreement of its definition is required to PRECEDE any statistics using it. Otherwise, you do not detect cheating but only detect your own preconceptions about cheating.

Same for other terms, such as "middle game".

Judgement may be subjective and imperfect, but that does not mean that it is totally wrong. There are in fact ways of combining the judgements of different people and assessing their combined reliability. In the sequence I showed, Bojanic and I agreed that three plays were significant. In itself that agreement does not mean much, and I would certainly not put myself forward as a worthy judge for this case; I was illustrating how the process might work. Ten years from now we may well have computer programs (not necessarily programs that play the game) that can reliably assess the difficulty of individual plays, but for now we have to rely upon humans, faute de mieux.

[Gobang]

In this thread there is talk of a "we" who should come up with an anti cheating system. Who exactly are these "we"? Random people in an internet thread? What expertise do these "we" have"? What resources? How much time will "we" devote to this task? Will "we" be reimbursed by somebody? If "we" come up with a system, will anybody use it? How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?

[maf]

In this thread there is talk of a "we" who should come up with an anti cheating system. Who exactly are these "we"? Random people in an internet thread? What expertise do these "we" have"? What resources? How much time will "we" devote to this task? Will "we" be reimbursed by somebody? If "we" come up with a system, will anybody use it? How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?

Can you clarify? I'm not sure I understand your intention. Is this a genuine question? Sorry if I'm misreading.

[Bill Spight]

If I am not mistaken, in PGETC sgf files there is a time stamp?

This could be number in brackets, in the end, indicating remaining time:

From Metta-Ben David game
http://pandanet-igs.com/system/sgfs/637 ... 1511906173

I checked thinking time of Black for moves 51-149. He spent:

• 2 seconds on move 51
• 3 seconds on moves 73, 95, 137
• 4 seconds on moves 63, 93
• 5 seconds on moves 91, 109

About "tenuki moves" 51, 59, 65, 87, 97, 101: times spent were 2, 12, 8, 11, 56, and 8 seconds.

Botvinnik pointed out years ago that time spent on a move is an indicator of subective difficulty. (He recommended studying positions where you took a lot of time.) On that basis stands out as a difficult move for Metta. To me that makes sense. Is it kikashi or aji keshi? Not so easy to judge.

Conversely, the times taken indicate that , , and are part of a subjective unit for Metta. As is Black 109 (a different unit, OC ).

This suggests that time taken could be taken as an indicator of subjective significance for Metta.

[Charlie]

Why do you think it can not be done quickly?
While waiting for opponent's move, you position your cursor in other program, ...

Of course, it can be done quickly. No doubt. As a StarCraft player, I'd say that the real-time latency induced by relaying the moves to Leela would be measured in milliseconds and so the only cost, as measured on the wall-clock, would be Leela's thinking time.

But what of the mental cost in concentration?

Hypothetically, if one was playing in this way -- blitz-clicking moves between two programmes -- and also being careful to sample within Leela's top-3 suggestions in order to disguise their cheating, would they really be sufficiently engrossed in the game to find their own, good moves? I think not.

I think someone could randomly deviate and chose to ignore Leela but not with any great level of skill. Not at that speed.

Of course, I am not a 4-6 dan player, myself. I also manage to misclick often enough even without cheating and needing to replicate the game in another program. Maybe rapid relaying is a skill that can be learned through practice.

[Bill Spight]

In this plan it seems to me that there are four significant Black choices: moves 85, 87, 97, and 101. It also seems to me that this plan, and these plays are well within the competence of a European 4 dan.

Estimating strength of the moves is very subjective, and furthermore, amateur players can be strong in one part of the game, and weak in another.
In this case, we have simpler job - there was only one AI program available and strong enough at the time, and, surprisingly, most of Metta's moves were same as program. Now, Metta could played some of the same moves - but all of them, in same sequence?

I think you underestimate the difficulty of the task. Not to belabor the issue, I have written a lot about that in this thread. Also, having identified the significant plays those are the plays to focus on, not the other plays in the sequence. Regarding all plays in the sequence equally was one failing of the statistical method used to reach the original verdict. I thought that we agreed on that. The only moves that we both consider significant are 87, 97, and 101. Those are the three plays to look at in the range being considered.

I don't know whether you agree with me about Black 105 and 111, but this demonstrates how human judges could reach agreement about which plays are significant.

I partially agree - those moves are significant endgame moves. Since there is no strategic aspect in them, they carry less weight than middle game moves.
I think endgame should be analyzed separately. Also problem with endgame in some cases is that due to large difference it is not so important what to play.

Those moves may be considered to be yose, but they are not endgame plays. Check out John Fairbairn's writing about the difference between the two. Yose can occur even in the opening.

[Bill Spight]

ut what of the mental cost in concentration?

Hypothetically, if one was playing in this way -- blitz-clicking moves between two programmes -- and also being careful to sample within Leela's top-3 suggestions in order to disguise their cheating, would they really be sufficiently engrossed in the game to find their own, good moves? I think not.

An important point. That is why the mistake that Metta made that was Leela's top choice was an important piece of evidence that Bojanic discovered. Left to his own devices, Metta might well have found the right play.

Edit: I don't mean that I think that Metta was not left to his own devices.

[Bill Spight]

In this thread there is talk of a "we" who should come up with an anti cheating system. Who exactly are these "we"? Random people in an internet thread? What expertise do these "we" have"? What resources? How much time will "we" devote to this task? Will "we" be reimbursed by somebody? If "we" come up with a system, will anybody use it? How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?

Broadly speaking "we" are the community of serious go players. I expect that the Chinese pros are working on this problem without paying any attention to Western amateurs.

When I was president of the New Mexico Go Association, "we" was me. I was able to enlist some help running tournaments and so on, but if I did not do the work, nobody did. I also helped with the writing of a tournament director's guide for the AGA. In that case, "we" was Terry Benson, myself, and whoever else he managed the recruit.

As a practical matter, "we" is whoever answers the call.

[Jan.van.Rongen]

Why do you think it can not be done quickly?

Because I tried a year ago in many games on KGS. Look at this rank graph on KGS:

http://www.gokgs.com/graphPage.jsp?user=mrooijer

Those are all games against bots with 5-15 seconds per move. 5 seconds was very tight. I lost some of those games on time.

Your reasoning that it is possible to play faster is invalid, not because one cannot play faster, but because that assumes that Black was playing fast because he cheated.

[Bill Spight]

move 97 – low suggestion, although sente move. For this move, move 101 was A suggestion from 10k - 56 seconds - it is strange in analysis and in duration, maybe some kind of disturbance, or getting up?

Was the one move of the 50 moves originally considered that did not match Leela's top three choices?

[Jan.van.Rongen]

On the computer Carlo Metta might have used in his PGETC games. In the Italian appeal they specify what computer they performed their counter-analysis on: Intel Core i7, 2.60 GHZ, RAM 16GB, GPU NVIDIA GeForce GTX 960M. Operating system - Windows 10. They also say there that it analyses about 100k nodes in about 30s. I asked the question what computer Carlo used in his PGETC games and the answer I got (from Mirco Fanti, the Italian team captain, as he insisted any questions should not be asked Carlo directly but should go through him) that it was the one used in the analysis. I conclude from this that most (in not all) of the Italian counter-analysis was done by Carlo himself.

That conclusion is not justified. Carlos is a PHD student of AI, and it is logical that he uses a laptop with a GPU. The best buy (about 1000 euro) would be the configuration mentioned above: an 8 thread Intel i7 CPU, 16 GB and this mobile version of the GTX 960. I chose the same configuration for the same purpose, but bought it a year later when the 960 was replaced by the 1050 / 1060.

This configuration allows us to build and train medium sized neural networks, f.i. with Google's Tensorflow library. For larger networks one could use cloud computing - I even think that I saw Google giving away training time on their tensorflow GPU's that trained AlphaZero.

Anayway, that Carlos and his PhD supervisor have the same config for their laptops is quite likely.

Another point to note is that the laptop screen is not big enough to view Leela 0.11 and a Go server client at the same time. You need to toggle between three windows: Leela 0.11 board, Leela 0.11 Analysis and the (Pandanet) client. That takes time.

Thus the fact that Carlos played a lot of his answering moves almost instantly is evidence in favour of not cheating. For two reasons: (a) they are probably logical moves and the fact that Leela also recommends them is not releveant; and (b) that there is no time to switch windows and look at the Leela recommendations.

[Bojanic]

Your reasoning that it is possible to play faster is invalid, not because one cannot play faster, but because that assumes that Black was playing fast because he cheated.

Jan, I did not bring question of time, it is not even in my work.
It was brought here is argument that some of the moves could not be found in such short time. As you can see from data I posted, it is possible, but I dont think it is accurate.

Regarding motorics, it varies greatly among men. In KPMC 2006, korean playerwould wait calmly until 29, and then pick a stone, played a moveand pusha clock in less than second.

PS not all his moves were fast.

[Jan.van.Rongen]

Was the one move of the 50 moves originally considered that did not match Leela's top three choices?

I do not have that original analysis, but in my runs with Leela 0.11 it was not always recommended in the top 3. Note that the original method is not stable: there is some randomness in the behaviour of Leela 0.11

In this case Leela 0.11 thinks that K13 and N14 are the best moves, better than M6 or L7