“Decision: case of using computer assistance in League A”

theoldway · **#481**

As one already observed, Chess World took years to develop a solid anti cheating method. Considering Go complexity I think that we'll need at the very least other 4 or 5 years to solve this problem.

Most important thing is that a future solid method needs to be accepted by all EGF association. If just one association does not accept a method, then it is not valid. It requires unanimity like the EGF constitution. That's why we cannot judge past disputes, some associations will be adverse to every method that will convict one of their players for a past dispute.

From this point of view, this case is a dead end and continuing to discuss it is pointless, we need to focus on the big picture.

There will be no possible conviction, under democratic rules and institutions, without material evidence and common established rules, exactly as stated by the decision of acquittal of the appeal commission.

Bojanic · **#482**

Actually, no. When proven of using means that are not allowed, in other sports sanctions could be brought against sportsman decades after event. So, this case can be examined for much longer, though I think that already after EGC and AGM lot of progress would be made.

jlt · **#483**

Bojanic wrote:

If I am not mistaken, in PGETC sgf files there is a time stamp?

This could be number in brackets, in the end, indicating remaining time:

From Metta-Ben David game
http://pandanet-igs.com/system/sgfs/637 ... 1511906173

I checked thinking time of Black for moves 51-149. He spent:

2 seconds on move 51
3 seconds on moves 73, 95, 137
4 seconds on moves 63, 93
5 seconds on moves 91, 109

About "tenuki moves" 51, 59, 65, 87, 97, 101: times spent were 2, 12, 8, 11, 56, and 8 seconds.

Bojanic · **#484**

Regarding tenuki, definitions, and why I consider it important:
in middlegame, and in endgame, we have a move that is not linked to a group of stones where previous moves were played, followed by sequence of moves that might follow it. Playing tenuki move in middle game is hard, it cannot be learnt from the books like fuseki, you have to be strong to get it right. It could be played on entire board, and finding correct move, or move similar to some AI program, is therefore much more difficult. That is why we have to give such moves more attention. It is a bit wide definition of tenuki, and any move which could fit into this description should be analyzed, even some close ones.

Sequence following tenuki could be few moves, or tens of moves long. Some of it's moves could be forced, other's easy to spot, while generally all the good moves are located in vicinity, therefore they are much easier to be found or stumbled accidentally upon. Sequence of moves could contain lot of moves similar to AI best choices, depending on position and player's strength. Analysis of such sequences should be done on entire sequence.

----

One example from current analysis: in one game that is very similar to Leela, almost every move, except few, are Leela's top choices. On it's own, very strong evidence. But, statistics could be deceiving - moves that are least similar, were middle game tenukis, which, given player's strength, shows that he could probably play most of tenuki moves in the game without help. And that means much more difficult further examination...

Bill Spight · **#485**

Jan.van.Rongen wrote:

After move 139 the black advantage Leela 0.11 estimates is 70%, but AQ finds it only 55%. It is white's last chance to overturn the game, and AQ indeed can win this position with white against Leela 0.11. That means AQ's evaluation of this position is probably more accurate.

Isn't AQ stronger than Leela 0.11?

Quote:

AQ vs AQ is not interesting at all in this discussion.

How do you test winrates? If AQ vs. AQ means that White wins only 30% of the time, doesn't that support Leela's estimate?

Quote:

AQ also thinks that white 70 was a big mistake (cutting off two stones is too small).

Me, too. In fact, I thought it was so big a mistake that it made most of the remaining plays that were assessed (up to move 150) too easy to draw much of a conclusion from. Even if Black cheated, White helped a lot.

Quote:

Then some here (correctly) pointed to the time record in the SGF. I already did the analysis. This record shows that black in this game played a lot faster than white, whcih might make one suspicicious, BUT a lot of moves were made so fast that there was simply not enough time to transfer the white move to Leela 0.11 and wait for a reasonable analysis from that engine. 80% of the black moves were fast moves.

Interesting point.

Bojanic · **#486**

jlt wrote:

From Metta-Ben David game
http://pandanet-igs.com/system/sgfs/637 ... 1511906173
About "tenuki moves" 51, 59, 65, 87, 97, 101: times spent were 2, 12, 8, 11, 56, and 8 seconds.

Carlo's machine's speed was given as 100k analysis in 30 seconds.
While I was examining Leela's analysis, I watched how suggestions progressed, and wrote after what time moves appeared as top choice, which is written in paper: (those values could be shorter, esp, if position was analyzed before).
move 51 – A , from 2k variations (immediately) - 2 seconds, match
move 59 – A , from 40k - 12 seconds, close match
move 65 – A , from 2k - 8 seconds, match
move 87 – A , from 5k - 11 seconds, match
move 97 – low suggestion, although sente move. For this move, move 101 was A suggestion from 10k - 56 seconds - it is strange in analysis and in duration, maybe some kind of disturbance, or getting up?
move 101 – A from 2k to 20k, and after that B. - also match.

Please note that some of the moves could appear in previous moves analysis, ie move 101 appeared first in move 97, therefore it could be shown earlier or found faster. This method is IMO not conclusive. (though I have to notice that in this case it fits quite nicely).

Bojanic · **#487**

Jan.van.Rongen wrote:

Then some here (correctly) pointed to the time record in the SGF. I already did the analysis. This record shows that black in this game played a lot faster than white, whcih might make one suspicicious, BUT a lot of moves were made so fast that there was simply not enough time to transfer the white move to Leela 0.11 and wait for a reasonable analysis from that engine. 80% of the black moves were fast moves.

Why do you think it can not be done quickly?
While waiting for opponent's move, you position your cursor in other program, and observe KGS for move. I think that easily after 1s opponent's move can be played in program, and while it calculates move, cursor transferred to KGS window and prepared to play. Basically it means only 1s delay, which is negligible. In such situation, it is not possible to focus on game, you have to just follow suggestions.
Needless to say, you would have to have lot of practice for such feat.

theoldway · **#488**

Bojanic wrote:

Please note that some of the moves could appear in previous moves analysis, ie move 101 appeared first in move 97, therefore it could be shown earlier or found faster. This method is IMO not conclusive. (though I have to notice that in this case it fits quite nicely).

As long as your method tries to fit a single case, it is worthless.

Go world needs to find a general method. That's why we need to forget this case. Any method related to it is tainted by preconceptions and will be successfully appealed like the first one used by the league manager.
Furthermore, this trial has already reached a final verdict after the acquittal.

Bill Spight · **#489**

Bojanic wrote:

Regarding tenuki, definitions, and why I consider it important:
in middlegame, and in endgame, we have a move that is not linked to a group of stones where previous moves were played, followed by sequence of moves that might follow it. Playing tenuki move in middle game is hard, it cannot be learnt from the books like fuseki, you have to be strong to get it right. It could be played on entire board, and finding correct move, or move similar to some AI program, is therefore much more difficult. That is why we have to give such moves more attention. It is a bit wide definition of tenuki, and any move which could fit into this description should be analyzed, even some close ones.

Sequence following tenuki could be few moves, or tens of moves long. Some of it's moves could be forced, other's easy to spot, while generally all the good moves are located in vicinity, therefore they are much easier to be found or stumbled accidentally upon. Sequence of moves could contain lot of moves similar to AI best choices, depending on position and player's strength. Analysis of such sequences should be done on entire sequence.

There are short units of play, such as hane-tsugi, longer units, like one lane roads, and longer sequences that are not so tightly linked, such a plans. In the Metta-Ben David game it seemed to me that the sequence Black 85 - Black 101, was a plan to solidify Black's advantage after White's big mistake, :w70:

, and wedge on the previous move, :wc:

. (I am not sure that a kyu player would come up with such a plan, but it would not be unusual for a mid-dan amateur, IMO.) It starts with the miai exchange, :b85:

-

.

Click Here To Show Diagram Code: [go]$$Bcm85 Metta (W) - Ben David $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . . . . . 1 . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . W . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . X . X . . . . 2 . . | $$ | . . . . . . O X X O . . . . . . . . . | $$ | . . . . . . O X O O . . . O . . X . . | $$ | . . X . . . . O . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . O . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Next we have the sente sequence, :b87:

-

.

Click Here To Show Diagram Code: [go]$$Bcm87 Metta - Ben David $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . . . . . X . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . X . X . . . . O . . | $$ | . . . . . . O X X O . . . . . . . . . | $$ | . . . . . . O X O O . . . O . . X . . | $$ | . . X . . . . O . . . . . . 6 . . . . | $$ | . . . X X X O . . , . . . . 4 5 . . . | $$ | . . X O O X O . . . . . . O 3 . X . . | $$ | . . . . X O . O . . . . 0 2 1 . . . . | $$ | . . . . . . O . . . . . 8 7 9 . . . . | $$ ---------------------------------------[/go]

This is almost a one lane road. :w88:

is not forced, but is fairly predictable. The rest of Black's moves stem from the desire to settle the position. I am not sure that I would call :b87:

a tenuki, since after :w86:

White threatens to attack the bottom right corner. Still, I think that it is an important play.

Next, we have a kikashi sequence to bolster the center.

Click Here To Show Diagram Code: [go]$$Bcm97 Metta - Ben David $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . . . . . X . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . X . X . . . . O . . | $$ | . . . . . . O X X O 1 . . . . . . . . | $$ | . . . . . . O X O O 2 3 . O . . X . . | $$ | . . X . . . . O . . . 4 . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Again, I think that this sequence forms a unit.

Click Here To Show Diagram Code: [go]$$Bc Metta - Ben David. $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . 1 . . . X . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . X . X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Finally, Black plays the double keima at 101.

In this plan it seems to me that there are four significant Black choices: moves 85, 87, 97, and 101. It also seems to me that this plan, and these plays are well within the competence of a European 4 dan.

This is something I considered writing up about this game early on, but I decided that focusing on the statistics was better at that time.

(Edit: Actually, I did say something along these lines here: viewtopic.php?p=229069#p229069 )

I see that you agree with me that moves 87, 97, and 101 are significant.

Click Here To Show Diagram Code: [go]$$Wcm4 Metta - Ben David $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . 2 . | $$ | . . . . O O . . . . . . X . . . X 1 6 | $$ | . O O O X X . . O . . . . . . . 4 3 . | $$ | . X . O O X . O . . . . . . . . 5 . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Next, Black 105 - 109 form a unit. B 105 is a significant choice. At least Black has to decide which side to hane on. Perhaps 109 is also significant, but it combines attack and defense, guarding against the clamp.

Click Here To Show Diagram Code: [go]$$Wcm10 Metta - Ben David $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . X . | $$ | . . . . O O . . . . . . X 1 . . X O X | $$ | . O O O X X . . O . . . . . . . X O 4 | $$ | . X . O O X . O . . . . . . . . O 2 . | $$ | . . X X X X X X X . . . . . . . O 3 . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

also strikes me as a significant choice, as it allows White to seek complications.

I don't know whether you agree with me about Black 105 and 111, but this demonstrates how human judges could reach agreement about which plays are significant.

Kirby · **#490**

theoldway wrote:

Go world needs to find a general method. That's why we need to forget this case.

I'd suspect if we identified a general method, cheaters could simply target playing good moves that wouldn't be detected by this method to avoid getting caught.

¯\_(ツ)_/¯

Bill Spight · **#491**

Kirby wrote:

theoldway wrote:

Go world needs to find a general method. That's why we need to forget this case.

I'd suspect if we identified a general method, cheaters could simply target playing good moves that wouldn't be detected by this method to avoid getting caught.

¯\_(ツ)_/¯

It may be an arms race, but that doesn't mean you throw in the towel.

Quote:

Trust in Allah, but tie your camel.

Kirby · **#492**

Just seems like it'd be easy to circumvent any cheat detection method we come up with. Maybe I'm not thinking creatively enough.

RobertJasiek · **#493**

Bojanic has spoken of, and written statistics about, forced without explaining it. Bill, you do the same, speaking of sequence that is a unit, one lane road, sente sequence, forced, fairly predictable, important play, kikashi sequence - all without explaining these phrases.

There is no such thing as an easily identified unit outside a local endgame with a verified sente sequence. E.g., one might interrupt several of those "units" and play at Q17.

Statistics can never provide statistical evidence for anything if they rely on undefined terms. If statistics shall measure "forced", then completed research and general agreement of its definition is required to PRECEDE any statistics using it. Otherwise, you do not detect cheating but only detect your own preconceptions about cheating.

Same for other terms, such as "middle game".

Bojanic · **#494**

Bill Spight wrote:

In this plan it seems to me that there are four significant Black choices: moves 85, 87, 97, and 101. It also seems to me that this plan, and these plays are well within the competence of a European 4 dan.

Estimating strength of the moves is very subjective, and furthermore, amateur players can be strong in one part of the game, and weak in another.
In this case, we have simpler job - there was only one AI program available and strong enough at the time, and, surprisingly, most of Metta's moves were same as program. Now, Metta could played some of the same moves - but all of them, in same sequence?

Bill Spight wrote:

I don't know whether you agree with me about Black 105 and 111, but this demonstrates how human judges could reach agreement about which plays are significant.

I partially agree - those moves are significant endgame moves. Since there is no strategic aspect in them, they carry less weight than middle game moves.
I think endgame should be analyzed separately. Also problem with endgame in some cases is that due to large difference it is not so important what to play.

Bojanic · **#495**

theoldway wrote:

As long as your method tries to fit a single case, it is worthless.

Go world needs to find a general method. That's why we need to forget this case. Any method related to it is tainted by preconceptions and will be successfully appealed like the first one used by the league manager.
Furthermore, this trial has already reached a final verdict after the acquittal.

This gem has to be preserved.

This is biggest scandal in European Go I can remember of (and I play for 30 years).
It has everything:
- cheating in important game,
- referee in most important tournament involved,
- political pressure to influence referees,
- and now, even internet bots, who attack people who think differently.
All things separately unheard of now, not to mention combined.

To add cherry on the cake, funniest thing is that bots are so bad, that most of their messages actually work against their goal.

Gobang · **#496**

Bojanic wrote:

This is biggest scandal in European Go I can remember of (and I play for 30 years).
It has everything:
- cheating in important game,
- referee in most important tournament involved,
- political pressure to influence referees,
- and now, even internet bots, who attack people who think differently.
All things separately unheard of now, not to mention combined.

To add cherry on the cake, funniest thing is that bots are so bad, that most of their messages actually work against their goal.

You have a point. The situation is indeed stranger than fiction. If this was written as a soap opera story line it would be labeled poor because it is so implausible.

You are right about most of these things being unheard of, but "internet bots, who attack people who think differently." are common and have been for many years.

I agree that some messages that have probably been intended to be in favor of the person accused of cheating have indeed worked against that goal. It is hard to tell if that was the intention to begin with or not though.

Bill Spight · **#497**

Kirby wrote:

Just seems like it'd be easy to circumvent any cheat detection method we come up with. Maybe I'm not thinking creatively enough.

I think that you are being realistic about online cheating. Without physical or behavioral evidence cheating is very difficult to detect, as Regan points out. An intelligent one time cheat may be next to impossible to show, but if the player continues to cheat over time, the probability of sequential success diminishes exponentially, at best. Edit: At best, from the cheater's perspective, I mean.

Bill Spight · **#498**

RobertJasiek wrote:

Bojanic has spoken of, and written statistics about, forced without explaining it. Bill, you do the same, speaking of sequence that is a unit, one lane road, sente sequence, forced, fairly predictable, important play, kikashi sequence - all without explaining these phrases.

There is no such thing as an easily identified unit outside a local endgame with a verified sente sequence. E.g., one might interrupt several of those "units" and play at Q17.

Statistics can never provide statistical evidence for anything if they rely on undefined terms. If statistics shall measure "forced", then completed research and general agreement of its definition is required to PRECEDE any statistics using it. Otherwise, you do not detect cheating but only detect your own preconceptions about cheating.

Same for other terms, such as "middle game".

Judgement may be subjective and imperfect, but that does not mean that it is totally wrong. There are in fact ways of combining the judgements of different people and assessing their combined reliability. In the sequence I showed, Bojanic and I agreed that three plays were significant. In itself that agreement does not mean much, and I would certainly not put myself forward as a worthy judge for this case; I was illustrating how the process might work.

Ten years from now we may well have computer programs (not necessarily programs that play the game) that can reliably assess the difficulty of individual plays, but for now we have to rely upon humans, faute de mieux.

Gobang · **#499**

In this thread there is talk of a "we" who should come up with an anti cheating system. Who exactly are these "we"? Random people in an internet thread? What expertise do these "we" have"? What resources? How much time will "we" devote to this task? Will "we" be reimbursed by somebody? If "we" come up with a system, will anybody use it? How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?

maf · **#500**

Gobang wrote:

In this thread there is talk of a "we" who should come up with an anti cheating system. Who exactly are these "we"? Random people in an internet thread? What expertise do these "we" have"? What resources? How much time will "we" devote to this task? Will "we" be reimbursed by somebody? If "we" come up with a system, will anybody use it? How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?

Can you clarify? I'm not sure I understand your intention. Is this a genuine question? Sorry if I'm misreading.

“Decision: case of using computer assistance in League A”

Who is online