LZ v Golaxy

Vargo · #1

LZ13(c0cb60) has begun today its 7 games match against Golaxy (a chinese strong bot)
Both run on powerful computers (10x1080Ti v. 4xTitanV) 40 min.
LZ lost its first game and played some really strange moves...
According to some people, 11% of its moves weren't on the "best moves list", even with 50k playouts or more.
LZ was white.

the game

Move 150 was particularly weird... (but LZ was already losing by that time)
On my computer, even with 2'000'000 visits, this move isn't even considered (cf this !)

Roy777 (involved in this game) said :

Quote:

It was a technical error with the connection software that reads one screen's input and then figure out what move to play on the other side. (There's no direct GTP interface with this server.)
Apparently LZ might have a bug with pondering and byoyomi time periods, at least some people are claiming that, and so they had to manually change the time_settings command and that is what broke the connection software.

Each day, there will be another game that you can follow live if you want.
Information : here

jokkebk · #2

Now 0-2 for Golaxy. Jonathan Roy had made a nice stream with Lizzie analyzing the game real-time (we're so spoiled nowadays!):

https://www.youtube.com/watch?v=Dj3dC1C13P0

Based on the first two, first hunch would be that Golaxy has a slight edge -- not a huge one, otherwise the games would've ended sooner I guess.

dfan · #3

3-0 now. I was following along at home with the latest Leela Zero net (#123), which thought that LZ's win rate was as high as 85% before plummeting down to 50% in one move and then much lower soon after, which indicates that Golaxy just read better (there was a pretty complicated fight).

Vargo · #4

Golaxy was black, as it won the first two games, it had to play with a kind of handicap :
its first move must be at least on the 6th line !
Here is Game 3

kyotosato · #5

4th game LZ (black) won by resigned. B+resigned

Vargo · #6

Yes, 4th game was played with regular network #124(8a045bce)

You can see the game here

Uberdude · #7

Win rate and # playouts of LZ in game 4, the big jump up is around when it lost the ko but cleaned up lower right corner and snuck into right side. Did white make a mistake here, or was it just clarifying a previously confusing position?

Vargo · #8

LZ won again ! Up to now, it's 3-2 in favor of Golaxy, in this "best of seven" match.

It seems the network used for the last two games was Network 32179366 (356x20) and not #124 or #125

You can see the game HERE

Golaxy's beginning was a bit strange, and it never recovered !

Golaxy is a strong bot :

Quote:

Golaxy ( 星阵(B) ) is now playing on Tygem, beating pros with one-stone handicap and 0 komi, demonstrating that it's indeed capable of playing with variable komi.

(alreadydone00)

Vargo · #9

6th game, LZ black, and still the 6th line handicap for Golaxy's first move.
LZ lost rapidly by resignation...
The game is HERE

Two interesting comments (on reddit, cbaduk)

Quote:

Looks like LZ has a new 20x256 network 57f59. It is probably the v10 network trained on the new 15b games. Golaxy is using a new network too.Unfortunately, LZ can only use 3x1080ti today because the two persons owning stronger machines are busy today.

(diador)

Quote:

Leela's top right corner is totally dead. It was a tricky situation involving a ladder. Leela thought that when white pushed through P18 and O18, it could hane to kill those three white stones. But white's 18th move at D6 was a ladder breaker.If Leela had played its 25th move at N18, white would could with N17 and the ladder (starting with M17) fails. Leela lost this game because of its ladder reading.

(Platean)

Vargo · **#10**

Golaxy wins the last game.

The 7th game is HERE

Final score : Golaxy wins 5-2

Let's wait for another 5-10 networks, and have a rematch ;-)

hydrogenpi7 · **#11**

Vargo wrote:

Golaxy wins the last game.

The 7th game is HERE

Final score : Golaxy wins 5-2

Let's wait for another 5-10 networks, and have a rematch ;-)

fyi at least two of the games had operator "misclicks" or "error of the connector" on the LZ volunteer team side.

As well as for two of the games, LZ was only using half the hardware as Golaxy, ie 5x1080 vs galaxy's 10x gtx 1080ti

and the second to last game that it lost, LZ only used 3/10th the hardware specs of golaxy.

These matches were a farce for Golaxy PR. Its not serious and nothing more than a joke

The volunteer LZ team side was horrible unprepared, almost gross negligence if this was a job they'd be prosecuted for malpractice.

One day a game had to be delayed before a guy got stuck in traffic, another day a guy got sick and couldn't use his computer resources...

They were changing hardware specs EVERY SINGLE DAY, and during the first game also changing time settings within the one very same exact game itself.

This was probably the most blotched Go match in the 5000 history of the game.

///

and the final game was likely rigged as well, in that there was human interference that harmed LZ's play...

Code:

Within the context of analysis of (potentially) bad (and/or buggy) moves of LZ, I would like to offer the following opinion, @kuba97531 I concur that I too when attempting to duplicate or replicate the results, I could not get repeat nor duplicate LZ's first move no matter which network I tried....
So
Here is my theory, since we know the following facts :

   * The LZ moves are not automatically inputted into the game/match against its opponent Golaxy
   * It is a manual process that requires a human hand to 'click' and input the moves
   * There were on several occasions "operator misclicks" from on the volunteer LZ team side
   * There was announced via translation that game 1 had an 'error of the connector' which indicated to mean possibly another human input error.
   * Apparently in the games that were misclicked, the game was still able to proceed and continue, and resulted in further moves from both sides immediately after the misclick(s) took place, thus indicating that Leela Zero integrated those misclicks into its official line of play.

One can permissibly infer and/or reasonably assume from the above, given the fact that it seems multiple people cannot duplicate the first move from LZ's side in the 7th game, that it seems likely that either LZ volunteer team again 'misclicked' and/or they deliberately may have chosen to take a sort of creative liberties or creative license, if you will, by playing their own human whim / choice and overriding the LZ's engines decision, in order to then start the game off an unusual path.... (for whatever reasons that's unknown at this point since afaik they have not published the game 7 logs)
And to qualify my remarks this later /final paragraph is purely my own speculation as to one conceivable subjective justification if indeed the conjecture above is correct, due in part to the unique handicap rules in place for this match, it appears that first move in game 7 was decided by its human operators, esp since it played tengen from the start. Given the constraints that no first move could be beneath the sixth line, I find it hard to entertain the idea that Golaxy in and of itself would have chosen Tengen without human intervention. Now, since the handicap rules only specify it has to play above the fifth line for the very first move, there was no indication that this first move had to come from the engine itself and nothing that would have barred (afaik) the human operators of the Golaxy engine from picking the first move for it. Now, and this is were my guess is that perhaps it was likely the volunteer operators of the LZ team ALSO took creative liberties and took that to mean that its response to Golaxy's first move that LZ's first move could also be "hand picked" and that it did not need to be strictly engine based throughout the entirety of the game/matches. In lieu of there being an official logs published that provides evidence to the contrary, from the LZ volunteer side for the seventh match, it’s still an open question as to what exactly occurred. But given the totality of the circumstances and the aforementioned known facts proffered above, it is abundantly apparent to me that LZ's first move in game 7 was interjected improperly by human intervention.

https://github.com/gcp/leela-zero/issues/1226
http://archive.is/PRoik

Uberdude · **#12**

I agree there are several fishy and unsatisfactory aspects of this match, but you don't really help your case with exaggerated claims like

hydrogenpi7 wrote:

This was probably the most blotched Go match in the 5000 history of the game.

I am not familiar with every single Go match over the last 5000 years, but the start of the 2011 EGC in Bordeaux was pretty bad: they almost postponed the first round (about 400 games) by a day; in the end it was about 2 hours late so many players had already disappeared off sightseeing, and there were several conflicting versions of the draw.

hydrogenpi7 · **#13**

Uberdude wrote:

I agree there are several fishy and unsatisfactory aspects of this match, but you don't really help your case with exaggerated claims like

hydrogenpi7 wrote:

This was probably the most blotched Go match in the 5000 history of the game.

I am not familiar with every single Go match over the last 5000 years, but the start of the 2011 EGC in Bordeaux was pretty bad: they almost postponed the first round (about 400 games) by a day; in the end it was about 2 hours late so many players had already disappeared off sightseeing, and there were several conflicting versions of the draw.

It is not exaggerated when taken into modern context. Has there ever been a match in which both hardware specs and the network weights of the engine not to mention the binary code of the engine itself is changing every single day? A match in which two out of the seven games are resulting in mis-clicks, and the very first game had its time settings changed midgame? I cant think of anything that tops this, be it going back a month, a year, or 5000

In fact... So asymmetrically unfair and lopsided as to grotesquely allow the side ostensibly purported to be receiving the handicap to in effect actually turn around and be the side giving the handicap as indeed it was constrained to use only 30% of the hardware performance allowed to its allegedly stronger opponent? Maybe this is like the elephant in the room or emperor has no clothes situation but to any reasonable mind it would be self-evident that one has to wonder what exactly the heck is going on?

Or what about its last game, the seventh's game that Golaxy claims to have taken victory for? Well the volunteer LZ team purported to have used the official 15 block 192 filter Leela Zero network 125 on 4xTitanV, and I followed along, using the exact same network, exact same hardware specs, exact same LeelaZ binary executable engine file, and throughout a continous pondering to more than 1 million playouts it never choose nor considered choosing the first move that the LZ volunteer team had hand played in the seventh game, basically this was an improper interjection and a human intervention against the rules and against the spirit and the entire purpose of the game/match. Frankly, I don't think anything like this egregious has ever happened before in the history of computer Go in a public match.

Uberdude · **#14**

hydrogenpi7 wrote:

It is not exaggerated when taken into modern context.

So not the last 5000 years then.

hydrogenpi7 wrote:

Frankly, I don't think anything like this egregious has ever happened before in the history of computer Go in a public match.

This is getting better with more qualifiers, seeing as computers haven't been around for 5000 years so you can exclude all of Jowa's shenanigans and other such exciting tales of intrigue and scheming from Go history. But what about that North Korean bot that did well in some competition that apparently plagiarized the source code of another bot. I'm sure there are plenty of tales of bad organisation and mess-ups neither of us are aware of. As you say "I cant think of anything that tops this", but maybe someone else can.

My message is you have a strong point by just focusing on the facts of the case, by adding such unnecessary grand claims you don't make your point stronger, you actually make it weaker.

hydrogenpi7 · **#15**

Uberdude wrote:

hydrogenpi7 wrote:

It is not exaggerated when taken into modern context.

So not the last 5000 years then.

hydrogenpi7 wrote:

Frankly, I don't think anything like this egregious has ever happened before in the history of computer Go in a public match.

This is getting better with more qualifiers, seeing as computers haven't been around for 5000 years so you can exclude all of Jowa's shenanigans and other such exciting tales of intrigue and scheming from Go history. But what about that North Korean bot that did well in some competition that apparently plagiarized the source code of another bot. I'm sure there are plenty of tales of bad organisation and mess-ups neither of us are aware of. As you say "I cant think of anything that tops this", but maybe someone else can.

My message is you have a strong point by just focusing on the facts of the case, by adding such unnecessary grand claims you don't make your point stronger, you actually make it weaker.

Your point is taken with regards to overexaggeration causing negative impact on the message itself.

Outside of the realms of Go, the whole PR stunt pulled by Deepmind with the Stockfish thing was pretty outrageous as well, esp considering the weight they had in the community and the public influence and broad exposure... many in the Chess community echo'd the same sentiment that Deepmind didn't really play a fair match when it claimed victory. It seems shenanigans happens on every level... So okay perhaps the Golaxy thing wasn't the worst thing to happen to Go match in 5000 years, but to say that they could have done much better is probably also an understantment. .

https://groups.google.com/forum/#!topic ... xSnY8xy7sY

Hopefully it provides some benefit for others doing matches in the future in terms of general guidance as to what not to do.

John Fairbairn · **#16**

Quote:

Your point is taken with regards to overexaggeration causing negative impact on the message itself.

Outside of the realms of Go, the whole PR stunt pulled by Deepmind with the Stockfish thing was pretty outrageous as well, esp considering the weight they had in the community and the public influence and broad exposure... many in the Chess community echo'd the same sentiment that Deepmind didn't really play a fair match when it claimed victory. It seems shenanigans happens on every level... So okay perhaps the Golaxy thing wasn't the worst thing to happen to Go match in 5000 years, but to say that they could have done much better is probably also an understantment. .

I can feel T Mark looking down on this from the galaxy and itching to be able to respond, so I'll do it for him with one of his favourite sayings: "I've told you a million times not to exaggerate."

macelee · **#17**

hydrogenpi7 wrote:

These matches were a farce for Golaxy PR. Its not serious and nothing more than a joke

I totally agree with some other members on this forum - you exaggerated here as well.

I am not sure how good PR is defined. But this event definitely reached Go fans very well. Simple statistics: the most popular game in this series had about 490,000 views on the Yikeweiqi platform. In this game alone there were 934 comments posted by the spectators. To put this in context, this week's biggest pro tournament game - the deciding game for the Chinese Tianyuan title - had 996,000 views; yesterday's Changqi Cup game with Ke Jie attracted 638,000 viewers.

As far as I can see, the PR worked very well. Golaxy got plenty of media attention, before its planned big game against Ke Jie. Mr You, the author of Abacus and also behind Golaxy, is almost a celebrity in the Chinese Go circle.

Uberdude · **#18**

macelee, were there any more details on the design of Golaxy? I heard it is neural network with something like 19 blocks, so it doesn't sound as novel as suggested.

macelee · **#19**

Uberdude wrote:

macelee, were there any more details on the design of Golaxy? I heard it is neural network with something like 19 blocks, so it doesn't sound as novel as suggested.

I've not read anything concrete. But people are talking about how Golaxy plays more like a human. Also the developers seem to suggest that Golaxy doesn't play for safety when it leads a lot. Golaxy is also said to support arbitrary board size and arbitrary komi, which is special.

macelee · **#20**

It turns out that Golaxy has another aspect that is more similar to human than to other bots. Instead of giving a winning percentage, Golaxy gives its positional judgement in points (e.g. black leads by 6.2 points) as if it can count.

LZ v Golaxy

Who is online