Game 4: Katago Gets White And It's a Brawl.
In the prior game, Golaxy and its 60x distributed engine played white and crushed Katago. Is taking white enough to let Katago catch up? This game is the most interesting so far, with neither side willing to back down and take a close loss. I've even added a few variations to the sgf.
White: Katago 1.61 with 8 2080ti gpu's - averaged 8k playouts per second on 40 block engine
Black: Golaxy Pro AI 60x speed - averaged 14k playouts per second on 50 block engine. Distributed architecture
Time: Two hour base time, 60 second byo-yomi
Comments: * My low-power Katago did a better job of predicting Golaxy's moves than KataGo's moves. Katago struggled to agree with some of its moves, even after being shown them. * Move 4, Oooh, diagonal opening. This could be fun. Do bots often choose diagonal openings? * Move 19, It's really intriguing how Katago shifts from upper right to upper left and back again over the next few moves. * Move 28, Low-power katago prefers P18, by 3% and 0.4 points at 200k playouts. I guess more playouts yields quite a shift. * Move 42, It felt like Golaxy pushed white around, but Katago is fine with this shape, thinking it leads (62.4% winning chance, 0.8 point lead) * White 48 surprises low-visit Katago, with B17 and E8 getting the overwhelming majority of the visits. Through 100k playouts, white 48 had only 45 visits and was 2% and 0.2 points behind the other candidates. Katago instantly like it when shown. Finding unlikely moves is the power of many playouts. * white 68: low playout Katago does not prefer this move, far preferring N6, then M6. Even at 700k playouts, Q8 is 1.6% and 0.3 points behind. But this moves sets the table for the coming fight, clearly splitting black in two. * White 78 is a cool move, either gaining a partial base as in the game, or gaining sente before running to the center, as in the variation. Katago thinks white leads (64.9%, 1.4 points) * White 84 What the hell, a quick forcing move, who cares about the fight. * White 88 I suspect that white sees the fighting overflowing to this side of the board, and it decides to get a quick forcing move in. Interestingly, at 300k playouts, katago doesn't seem certain that it's truly forcing, and black could tenuki at p9 (39% winning rate), in many variations dominating the lower right, and losing out on the bottom or the left. I played a ton of variations starting at p9, all varying between 30-45% winning rates for black, with drastically different looking outcomes. Perhaps this chaos would have been a better choice for Golaxy. * White 98 is hard to find at low playouts. At 100k playouts, Katago thinks it is 8% and 2 points worse than O5, and thinks it's a losing move.Even when shown P5, it takes katago 8000 playouts to realize it's good. * White 106 is somewhat hard for katago to find, katago preferred L9 for at least 200k visits. It took 50k playouts before it agreed with the move, after it was played. * 108 Despite the fact that both black and white have two weak groups, Katago's estimate of score hasn't changed much (60%, 2.8 points). Given that each has one group sealed in, and one group running, it's hard to see it ending peacefully. * Black 111 seems necessary to live with the bottom group, but white 112 pretty much kills black's lower right group. See the variation for black's best attempt to save the group in ko. * Black 113 and 115 are cool. When katago avoids the ko in the lower right, these moves enclose white, and add a new group to the fight. I think white 88 saw this coming. * black 143 - Are this, 145 and 147 horizon extending moves? They seem a waste of ko threats. * white 158 creates two cuts, and turns black's attack into a fight. * black 171 creates a huge ko, but black needs larger threats than white. * if black 173 captures the ko at F1, white captures the middle black group with P11 and leads by 72%, 1.4 points. Golaxy chooses to fight on, instead. * from 174, both sides make outside groups safer, fill outside liberties, and exchange monster game winning ko threats. * Golaxy resigns after white 244. If black answers this threat, it does not have a threat big enough. See the variation, in which white throws away the upper left, but still wins handily. 99.5%, 9.5 points
I am not going to say Katago is stronger than Golaxy. But I am delighted to see that it can play well even against a super-strong distributed version of Golaxy.
Time for a wee bully-pulpit:
There are at least three ways to measure an algorithm's (bot) strength.
1) Which is most efficient to develop? Google's Deepmind spent a fortune building AlphaGo. They dedicated resources, both hardware and engineering that few others if any could match. Now we've reached the point where a mostly solo developer, David Wu, renting cloud GPU's is able to challenge multinational corporations. Pretty cool. It would be fun to host an AI Go Challenge that included development in the process.
2) Which bot is strongest without hardware restrictions? This current World Go AI Championship is following this format. I guess its fine, but not really meaningful. The reality is that a worse engine on better hardware can win. If one of the finalists in this year's contest comes with a bank of A100's to compete against 2080's, it's going to do very well. So what does this format prove? Well deep pockets can make the strongest bot. I think both Golaxy and FineArt will continue to do very well, so long as they retain funding, even if their algorithms fall a little behind. (Note, I have no reason to believe they've fallen behind, and I'm not saying they have.)
3) Which bot is strongest on equivalent or identical hardware? For me this is a real contest. Let each developer equip identical machines, and see who comes out on top. I also think individual games are somewhat meaningless. Just as I wouldn't claim katago is stronger than Golaxy despite winning three out of four of these recent games, I find the existing championship format of very few games somewhat meaningless. I'd like to see each bot play other bots many times. They aren't human, there is no reason to restrict bots to one game a day.
Ah well, that's enough for today. I liked this game!
Next - What happens if you let Katago play with ten cards?
_________________ - Brady Want to see videos of low-dan mistakes and what to learn from them? Brady's Blunders
|