LeelaZero adventures on Fox

Uberdude · **#41**

sporeyang wrote:

I'd like to help with the Chinese information issue.

Uberdude wrote:

Attachment:

chinese name.PNG

The message is interesting. It's the second half of a Chinese ancient poetry.
You may know "绝艺(Fine Art)". The name "Fine Art" is derived from the first half of the same poetry.

About the username "石庚" seems to be a normal name. After a little search, I found a website of someone says that his nickname and Fox id is 石庚. He said he is an amateur 5 dan and a private tutor of go.

Thanks for the detective work. I know Chinese 5d can be really strong, but I'm doubtful that white play was without AI assistance. Compared to when I review games of top pros with LZ, he had more matching moves of the stand-out "wow, cool bot move" type. I could well believe a top Chinese 5d could find any individual one of these, but so many together seems unlikely.
- s2 my LZ didn't find, can believe strong human finds it too.
- rest of that fight was almost one-way-street, though r11 was a decision point (s11 q10 other choices) and my LZ picked r11 too.
- d18 was LZ's choice with other choices b16 or f16 but all fairly normal so no biggie.
- e19 was a human move, my LZ wanted sharp q17
- p13 honte was a move my LZ kept flipping between other choices and reanalysing now doesn't like; if I was playing centaur style I would probably omit it and play out the q9 etc sente squeeze to stop LZ wasting playouts on wanting to play sente moves so it can devote full resources to thinking about important things on upper side (hmmm, is the observed bot habit of playing early sente moves which waste ko threats the bot itself learning this way of making it play stronger overall! :scratch:

).
- l17 wedge was LZ's exact spot, it's a 3rd line wedge around the middle so not that suspicious if just 1.
- p17 armpit hit LZ #1, classic bot move, that is gradually becoming more of human go vocabulary but fairly suspicious for a Chinese 5d to find it in 20 seconds IMO
- next few moves same as LZ but fairly natural flow for a strong player
- s17 very nice move LZ found, yeah a strong ama could on a good day, but this is a very good day! suspicious.
- g10 reduction exact same spot as LZ (and Elfv1), very suspicious.
- f12 same as LZ, other choices were going in below my left side stone, but plausible to pick this one

Uberdude · **#42**

topazg wrote:

I do somewhat wonder if the comments in the 9d game imply that at the moment there is some real possibility for professional strength players and engines to form an interesting hybrid team. I would be very interested Andrew, if you did a best of 21 or something against that guy, whether the match result would be close or not.

I think I would lose badly. I'm 6 losses in a row now at 9d. The last but one opponent did a few early LZ matching tell-tale moves in a row so I switched to Elfv1 to see if it could do any better but it couldn't. Are their better moves coming from the strong human operator, or their more powerful computer (or better network, things like LZ-Master or PhoenixGo)? Maybe a bit of both, but given my LZ has found some of these good moves that it didn't see in the game when I've given it more time afterwards my guess is it's mostly the latter. Let's see if LZ 0.16 speed boost can help...

@abcd_z My LZ has thought it was losing all these 6 games it lost and was black in 5 (as white losing bigger than half point after an unexpected good move from opp leading to dramatic trade so I resigned). Maybe I'll get lucky with a bot opponent playing white who thinks it's winning by half but then I actually win with the Fox 6.5 komi, but not happened yet*...

* But I just saw it happen to Elf on WBaduk vs "takabayama 7p", presumably Takabayashi Masahiro (http://www.nihonkiin.or.jp/player/htm/ki000179.htm), Elf as white played an unnecessary defensive move inside its territory as the final move of the game (under Elf's assumed 7.5 komi chinese rules would still win playing it) and then lost by half a point according to the server. That's the kind of artificial stupidity a centaur could avoid :lol:

Uberdude · **#43**

Finally played a human 9d (1061 win, 872 loss) and LZ won in a much more expected manner: winrate steadily climbing in opening with nearly every move of his being slight mistake to 74% at move 46 which was a ladder atari which currently worked, he plays ladder breaker (shoulder hit LZ expected: it's a good move globally even if there weren't a ladder), LZ doesn't realise and answers locally (net ladder stone answer at f11 was slightly considered at -7% so still winning big so if I was playing centaur I'd play that 7th choice move), subsequent fighting you can see LZ trying to create a ladder-making move in sente but with the ~2k playouts in 20 secs I get there is quite some time wasted each move re-reading the ladder (after every exchange it's "Oh, maybe this atari captures these stone, read-read, oh no it doesn't) so with a slightly less powerful GPU / faster game it'd do the dumb ladder ataris. Also because its instinct is to think black can capture with atari, as white it thinks it needs to extend there to prevent that, so initially thinks moves like k15 cut are better than they really are as it expects white to d12 nobi and allow black to capture 2 stones with m16, but of course human white who understands ladders won't allow that. Once the ladder is clearly gone by move 67 LZ lost its advantage and is back to even at 47%, but then quickly re-establishes a lead by ruthlessly taking advantage of a Fox 9d's mistakes:
- q12 was too thin: o12 cut was easy for me to see but r11 was a very nice move making miai (if white answers r12 q11 r14 then black loosely surrounds at o8 and semeai seems to favour black). Should m10 honte first and black needs to defend right, then e12 honte, black defends g14, d11.
- pulling out at h15 was some bad aji I also wondered about, LZ demonstrates great tesuji at f13. It must have seen this line earlier which made h16 a valid net.
- h14 should capture once, then the trade ends with a white gote at h11 then j9 is more efficient than at h10.
- wants s15 sente yose before h10
- black is 90% when blocks s16 reverse sente. I was intrigued it thought this better than a move like L4 on lower side, would a top pro have the same thinking?
- o4 at p4 was sharper, it's the kind of move I think a real human 9d can find and can miss, but the bot 9ds I play never miss the sharp local move like this.
- move 114 was another mistake where LZ quickly saw shortage of liberties tesuji.

Attachment:

9d human ladder recover.PNG [ 160 KiB | Viewed 20357 times ]

And another similar win vs Japanese 400W 262L 9d, same ladder mistake in that double approach joseki, but then mistake recovered in fighting soon after, and 90% by move 100.

Uberdude · **#44**

2 undeserved wins today against maybe bots: first was fairly close according to my LZ, then losing a bit from unexpected good move in middlegame and shortly after opponent ran out of time; second my LZ thought it gained a few percent to start, in a direction decision middle-game position most playouts move was something like 54% at 3k, promising 2nd choice was 56% but I only had time to get it to 1.5k (and % stayed higher) and played that move, but then some moves later opp played a good unexpected move (so maybe my fault as if I'd gone LZ's #1 move the fighting would have gone different direction and that blindspot not mattered) and I was losing, but then opp critical misclicked (in place he was probably expecting me to peep but I didn't, maybe getting his Fox and bot boards mixed up?) and let his time run out to resign.

Uberdude · **#45**

Had a nice game and a chat about bots with a Chinese 9d with Clint Eastwood avatar called 猪猪学棋1 today, lost by 0.5. Game was 40s 5 times so usually about 5k+ playouts per move (but less when he did unexpected moves). Pretty close throughout, LZ had some weak groups to attack but he managed to settle them skillfully. Going into endgame LZ thought it was winning a bit but then it flipped, one of the Chinese kibitzers mentioned k10 (in Fox coordinates with i, L10 below) which was a point where he'd just played an unexpected move and I played L10 as move with most playouts (about 3k), whilst the 2nd choice at about 1.8k and around 1% higher winrate was j10 so maybe that was indeed a better move and preserved what I guess was a half point black lead at that point (I shoulda spent an overtime on it).

Attachment:

clint winrate.PNG [ 230.97 KiB | Viewed 20100 times ]

Chat:

Uberdude · **#46**

One of those Chinese kibitzers was talking about new LZ 0.16 giving 20% speedup. I just tested on my 1060 GPU with a 40 block network and I go from 3.3k playouts in 30 seconds to 5.9k, a whopping 79% speed increase. Nice!

Used that for a 15s game, and it was the first instance of LZ as black thinking it was losing under the 7.5 komi area rules it assumes, but actually won on Fox 6.5 komi. This was a rematch against this 208W 130L Chinese 9d, the first game LZ won after a pretty even first 120 moves but then did better in a tactitcal centre fight. This game LZ mishandled the invasion of the right side (with more time after found moves it thought better) as although it got in the centre was damaged and the whole group lacked eyeshape. But then he made a big endgame mistake, starting at r1 block (t2 fallback safer) and then with t4 atari he didn't connect (LZ thinks connect is fine and white lives but black gets outside libs in sente) but lost a few points in gote and LZ thought it was winning. But then over the rest of the endgame LZ's winrate dropped again below 50, but actually won by 0.5. Also at the end he ahd a chance to win with a tesuji: LZ hadn't seen h16 atari was good, but when played winrate plummeted because white g18 sets up a shortage of liberties tesuji which captures the n19 tail.

Attachment:

lion winrate.PNG [ 188.37 KiB | Viewed 19990 times ]

sporeyang · **#47**

Quote:

猪猪学棋1 [9D]: 你59手如果长，我很难办 = If you are 59 hands [seconds? was 40 s not 60s game] long, I am very difficult to do.

He means if you play nobi (J14 ?) at move 59 , it will be difficult to deal with.

Quote:

猪猪学棋1 [9D]: 我要坏菜 = I want bad food [this was said at time of top side fighting, I think he means he wanted to capture stones that weren't good idea to capture, causing indigestion]

He means his situation is not going well. Not specifically related with capturing.

BTW, it writes "#1898" in leelazero7's introduction, maybe you want to correct it. I think the introduction in English is OK since LeelaZero should have been well known to Chinese 9Ds.

Uberdude · **#48**

sporeyang wrote:

Quote:

猪猪学棋1 [9D]: 你59手如果长，我很难办 = If you are 59 hands [seconds? was 40 s not 60s game] long, I am very difficult to do.

He means if you play nobi (J14 ?) at move 59 , it will be difficult to deal with.

Interesting, that was a key moment where LZ kept changing its mind. To start with it wanted to j15 atari and black doesn't collapse on the outside but the shape is quite icky. It was also looking at fancy moves like h16 but then solid connect was #1 just before my overtime period was up so that's the one I played.

Uberdude · **#49**

Coolio, I'm playing FineArtB currently in a 2 handi game! As we know FineArt is a super-strong bot I'll try to help LZ with exploring the promising blue circle moves and avoiding ladder blunders. One thing I've noticed already is FA plays moves which my LZ, in the time available, sometimes sees as higher winrate but hasn't yet risen to top playouts.

Lost. LZ thought it kept the 2 stone advantage for 130 moves, but then winrate quickly dropped off with left side fighting; I regret only spending one 1 minute period on c13, that's a critical time and just after playing d9 fancy knight move rose to top, who knows if FineArt had a good counter to that too (LZ aims at d7 cut and then f7 g7 making good use of dead 2 stones aji, by attacking white on left black gets out to centre). J16 was also a surprising good move. s8 push was me taking a rising blue circle over j6 jump which was probably a mistake.

Attachment:

fineart winrate.PNG [ 231.68 KiB | Viewed 19905 times ]

Uberdude · **#50**

First game with new #191 network, game featured a nice squeeze at lower side (although g2 capture seemed very natural and good aji, LZ after a while said white should have h3 connect, worse for points but taking a liberty means the squeeze doesn't exist; also m5 atari should l7 and black squeeze at m8 not so wrap-around). Winrate stayed around 58% for a while, but then dropped without white playing unexpected moves with top right group running. White then extended lead to 60% going into yose but whilst a12 didn't lose l11 was a bit better and would have avoided the game-changing aji, after k10 atari white needed to capture, m14 wedge was a great tesuji to get l12 in sente and then e10 breaks into the white territory and black's won.

Uberdude · **#51**

Just had an exciting 15s game with a 107W 26L human 9d in which the lead changed hands several times. Winrate grew slightly in opening, then dropped quickly with cutting on top left (probably thought f11 was a working ladder initially? actually mainline back then was black c9 at b12 which could be based on idea that black needed to play b12 to avoid ladder). At move 112 winrate was up to 83%, but then he played a sente move for a semeai, and at only 15s LZ tenukid (at 20s would have answered at h16, can lose semeai but then capture 3 stones in sente, I suppose with low playouts it didn't see his hane gained a liberty in sente to win the semeai) for -40% mistake. LZ then got back to slight lead, but then into endgame he took lead with me playing moves that soon after the overtime period ended LZ decided wasn't quite the best. But then LZ took the lead with superior play with the centre yose play around m8, making a nice little bunch of centre points which he could have more aggressively reduced e.g m11 at j7.

Attachment:

LZ 22nov.PNG [ 250.67 KiB | Viewed 19727 times ]

topazg · **#52**

Really enjoying these Andrew, keep them coming

I must admit, the idea of someone running a fairly serious #playout LZ bot on a correspondence server would be very interesting to me. Somewhere at the level where reading event horizon issues are very rare would I suspect make for some really super high level games to study or learn from, particularly if the operator put in the bot suggestions for the opponent after the game in a review (something like OGS' review system I guess)

Uberdude · **#53**

Did another low playouts game: #188 on a 2015 MacBook Pro so that's only about 50 playouts in 30 sec overtime, but did get up to a few hundred with reuse in expected sequences. I'd expect this can beat Fox 9ds so long as high-stakes complicated fighting and ladders can be avoided. LZ 2nd line crawled to kill a group leaving a corner ko, he spent several moves to avoid that ko and patch aji and LZ thought it was leading. LZ made weak groups it just managed to save (such confidence to live in the shadow of that ponnuki!), though f6 was a good move it didn't expect. Opp ran out of time at move 152 when LZ though it was at 64% and expected black to start the semeai on right side which is some ridiculous approach ko.

Uberdude · **#54**

Small world! Just played a Hong Kong Fox 9d who turned out to be a student in England who went to the London open last year (came 2nd, 1st was Chinese 5d) so I saw him but didn't speak, told him about the Cambridge tournament next March so hopefully will meet there. He kept game even for 65 moves, but 20% mistake in life and death and then LZ ran away with the game.

Uberdude · **#55**

Played again against 小咕咚77, who I'd lost to before (I think that game I started with LZ, then switched Elf when I though he was LZ too). This game I used Elf v1 from the start and it was close but somewhat surprised to win in the end!

Attachment:

winrate vs mastor77.PNG [ 182.95 KiB | Viewed 19591 times ]

Uberdude · **#56**

Played 2 more games as Elfv1 at 20s a move which meant I was usually 5-10k playouts, against a 82W 15L Korean who said he wasn't a pro. First game was particularly exciting with big dragons fighting for life via a ko. Elf with its strong opinions about small opening mistakes though it was leading early because he made the c18 b18 bad exchange (the bot joseki here is same shape as game but without that exchange, which is a slight minus because it means if the white groups needs to live it can make more eyeshape with a gote move by capturing a stone at d18 than had black not made the exchange). Elf took a while to find d11 which had e14 clamp as a sharp continuation. At move 72 instead of top right corner Elf also thought about defending left at e9 and then we would probably have had a more peaceful yose, but the s14 hane connect is a classic large yose protecting the corner and also creates some bad aji in black's right side which Elf used to spectacular effect in a big trade starting from 96 (96 first instinct was defend at m16 but then went for the clamp and fighting in right side). Quite a lot of times he played moves low down in Elf's playouts with similarly good or occasionally better winrates once Elf saw them, the Elf's blindspot and not very exploratory foibles. He killed the top left (so c18 minus was relevant but now resolved) in exchange for me killing the right side. Elf thought l8 was a slight mistake but j10 was the key point for reducing white's right and patching up bad aji of black's left (if tenuki then white will g10 or h10 which thickens centre territory whilst threatening to pull out the d11 stones) and a high stakes fight with neither side backing down ensued in which resurrected the top left dead group but the big lower side group wasn't alive, but neither was black's now cut middle group. Throughout this fighting Elf thought it was around 80% but it appears this was a horizon effect of not realising it couldn't win the ko, or rather if it wins ko as in game the lower side group dies because white can't kill the black middle, L11 was a good move it didn't see until rather late. So white lower group ended up dying and the game was over.

Edit: reviewing with LZ 157 and 191 they both criticise Elf's c12 damezumari which did surprise me during the game. They think capture and try to get black to spend a move at a10 because then white doesn't mind losing the ko so much and also with k3 n1 p5 combo there's aji to make a spare eye if black spends a move threatening the o10 cut. Given the other LZ 157 discussion linked on reddit I wonder if 157 with its extra playouts in the same time is better at winning positions like this which require reading a deep ko fight.

Attachment:

Elfv1 vs sorryz game 1 winrate.PNG [ 204.57 KiB | Viewed 19496 times ]

2nd game same opening to start and not so high-stakes fighting. Black h15 was a move Elf didn't consider (expected h16 kosumi) and when he played it winrate dropped a lot: at the time it thought it couldn't hane (not sure why) and needed to have exchanged k11 for l10 before playing h14 (is there a ladder, I can't find?), but reviewing now it says natural g15 hane would have been better than the f17 dodge recommended at the time. Elf got in trouble when it jumped into lower left in endgame (didn't expect natural f2!) but managed to make a ko and magically survive. There was a big ko trade in the endgame but it didn't change the result.

Attachment:

Elfv1 vs sorryz game 2 winrate.PNG [ 202.36 KiB | Viewed 19496 times ]

Uberdude · **#57**

Played a game with Elfv1 against an all-wins-to-9d then 1W 1L guy, I found it a wonderful example of Elf's strength in the early game and how it can dodge attacks flexibly for a good result: by move 30 it was at 90%. P14 was a good move from the opponent though that Elf barely considered but at +2% when already down to 11% it wasn't a blindspot that mattered. Thanks to p14 black captured some cutting stones for 4 points but white got the corner in exchange and cornes are big, and then black made another thick ponnuki but in exchange white thickly captured 2 on the outside (r7 was better at j14 to save those cutting stones), and thanks to white s13 the r10 stones weren't even dead yet so black's actual profit was not so much. Ignoring peep at 117 to live and then ignore another peep was another highlight.

Uberdude · **#58**

Just had the win a game by half a point (under Fox 6.5 komi) that the bot thinks it's losing (under 7.5 komi) situation I'd talked about before. I suspect opp was using a bot too, d1 hane losing move was Elf's first choice (not LZ 157 or 191) and a typical bot doesn't matter if I lose points and only win by half move as a human surely would have played a ko threat as he still had plenty. 2 moves prior to that I had switched from Elf v1 to LZ #157 because I suspected it was such a close game and thought 157 might do better at not losing points in a close yose because it won't think it's losing with such a big percentage as Elf. So I don't know if I'd have won had I kept using Elf, but a bit of centaur to pick my guess at the best bot for the stage of the game might have helped here (Lizzie 0.6 has a handy switch network without reloading game feature I just discovered).

Uberdude · **#59**

Another cheeky half point win as black, against a guy running Elfv1 on far faster Tesla v100 GPU. I switched from #191 to #157 again in endgame (move 165). He would have won if I didn't fill in his own territory as the last move of the game -- I don't know if these other people playing with bots on Fox relay the moves manually like me (so could insert a "don't do artificial stupidity" filter to not play such moves, or if it's automated). Still, I was impressed that #191 could hold its own against Elfv1 on superior hardware (though a winning bot can just let you lose by half so not sure how close it really was).

Attachment:

vs v100 elf.PNG [ 246.29 KiB | Viewed 19314 times ]

Chat

Uberdude · **#60**

Played 2 games today, both against other people using latest LZ 193. First I used elf v1 and lost, 2nd I used 191 and was surprised to win seeing as he had the faster 1080 GPU and slightly stronger network (he resigned at move 123). I couldn't get 193 to load but it would be interesting to see where they differed in judgements. Seeing as I was playing another bot I tried to help mine with taking promising winrate moves over the highest playout (e.g. q7 over
q3, n13 over m13 which made a big difference in the fight*), could this really be enough to make a slightly inferior network beat another on much better hardware?. Or maybe just this one game I was lucky to be on the side of the LeelaZero which didn't misjudge the weakness of a group (m15 was cool too).

* This game was 30 second overtimes, so after about 20 seconds m13 was #1 playouts but n13 higher winrate, I spent a bit under 10 seconds to check n13 really was better and then played it. As a test I gave 191 2 minutes for that move and m13 remained #1 (n13 didn't even get higher winrate, the promising alternative this run was f13) but again when played it liked n13. So if we assume that with enough time LZ would eventually give more playouts to n13 (perhaps not true as the exploration parameter seems too low) it's like in this instance I made my GPU more than 4 times faster (and a 1080 is about 2-3 times as fast I think). Another test over 5 minutes (during which e15 was another interesting promising move) m13 was #1 with 41k playouts and 57.4%, n13 800 playouts and 56.4%. Play n13 and for first ~2k it says lower winrate, but once at 4k it's up to 59.4%, and keeps on rising: after 10k at +3% to 60.2%, after 30k at 3.4% to 60.5% (during which it thinks white p8 peep may be slightly better before defending at p13), after 40k +3.5% to 60.6% and white's #1 playouts answer is now L14 but p8 highest winrate (horizon effect of wanting to play a sente move to postpone the inevitable badness? perhaps not, to start with it think black would answer at q8 but then finds black q14 as better resistance).

Attachment:

vs 193 winrate.PNG [ 254.05 KiB | Viewed 19067 times ]

LeelaZero adventures on Fox

Who is online