Some more info: this "large board" KataGo network has been trained in selfplay since late 2023. Initial training used an 18b network (based on the latest kata1 run available 18b at that time). In September 2024, I switched to 28b and immediately noticed a significant strength boost on all board sizes, including large boards —prompting a full transition to 28b for both selfplay and training. Again, a huge thanks to lightvector for KataGo, and for his guidance in setting up the selfplay loop for large boards!
Training Process / selfplay mix: - Around 85% of the data from upscaled kata1 selfplay (adjusted to 50x50 tensor sizes) - Around 15% of the data from native large-board selfplay (generated directly on 50x50 tensors)
Hardware: 3 GPUs (RTX 4090, 4080, and 3090) running continuously for over a year.
Selfplay Evolution: - Games generated up to 29x29 in 2023. - From early 2024: expanded to 50x50, after resolving some stability issues. Proportion of large board games was progressively increased to 32x32 → ~40x40 → 50x50, also including rectangular boards (e.g., 7x50) like kata1.
Performance Insights: Kata1 (« the main distributed run ») Baseline: - ≤25x25: Strong native play. - ≤30x30: Occasional errors (e.g., premature passes/first-line moves). - 30x30 to 40x40: Frequent errors; above 40x40 requires quasi-systemic manual supervision to avoid frequent major mistakes or refusal to play (pass). - Note : surprisingly, kata1 natively plays quite well on “very rectangular” boards such as 7x50. So the LB-trained network is better, but not insanely better.
LB18b (Large Board 18b, from 2023 to August 2024): - Rapid (but limited) improvement up to 29x29, slower (but much bigger) progress beyond 35x35.
LB28b (Large Board 28b): - Trained initially using LB18b selfplay data: it outperformed LB18b on all board sizes - Then trained natively from LB28b selfplay data.
Quantitative Results (LB28b vs k1-s7503, 400 games, 1000 visits): - 19x19: ~25 Elo gain over k1-s7503 (boost probably due to a lower learning rate). - 25x25: around +200 Elo. - 29x29: around +600 Elo (395 wins, 1 draw, only 9 losses out of 405 games). - ≥30x30: LB28b largely dominates k1-s7503.
Progress Metrics on large boards (vs earlier LB-trained networks): LB18b (2024): - 37x37: +400 Elo (25 visits, April 2024 → Sept 2024). - 50x50: +700 Elo (25 visits, June 2024 → Sept 2024). LB28b (Sept. 2024 - March 2025): - 37x37: +150 Elo (1K visits, Sept 2024→March 2025). - 50x50: +350 Elo with 10 visits and +900 Elo with 100 visits (398 wins, 2 losses !).
Current Limitations (LB28b): - Scoring errors: some minor discrepancies (≤few points) on boards >37x37, some rare strong discrepancies - Dame issues: Residual neutral points on boards mainly above 40x40.
Next Steps: - Uploading LB28b vs kata1 29x29 games, and some 50x50 games in the coming days! - More large-board games to follow, with a few high visits large board games. - Releasing the network after (probably) a last training by mid-April
And attached, you'll find 8 games from the 29x29 match against kata1 at 1000 visits (with about +600 Elo, cf quantitative results above). It's a txt file that you can split in 8 different sgf files if needed.
|