Interesting Analysis of Ancient Top Professional Players

[illluck]

https://mp.weixin.qq.com/s/03YjCodvYpw8yRJywrXl1g

By Cui Can 5P

I just read a paper using Kata to analyze ancient games which was quite interesting
It compared the "big 3" Ancient Chinese players (Huang Longshi, Fan Xiping, and Shi Xiangxia), Dosaku, Jowa, Shusaku against modern top professionals and top amateurs

General conclusion: no significant difference in the first 180 moves between Huang, Fang, Shi, Jowa, and Shusaku vs top professionals, while Dosaku is a bit lower.

All were significantly "more accurate" compared to top amateurs.

[illluck]

With regards to opening, all had significant differences compared to top pros, but Huang/Fang/Shi are not significantly different from Jowa/Shusaku.

As a side note, even top amateurs had "more accurate" opening than the ancient players, indicating that there have been advancements in opening theory.

Some other interesting findings:

1) Even though Dosaku era openings look a lot more reasonable by modern standards, they actually were significantly worse than the Huang/Fang/Shi openings which look a lot more weird, probably because modern opening theory is derived from Dosaku's lineage.
2: Shusaku was incredibly consistent.
3: AI did not think very highly of Lee Changho compared to several other top pros, probably partly because the analysis took only up to 180 moves so his endgame was not fully considered. Another possible reason is that AI might not think too highly of playing "safer than usual" moves to ensure win.

[kvasir]

I have noticed that style plays a huge role in the accuracy of the first 50 moves.

So I am not surprised that Lee Changho does not score high, big part of his style was literally to walk a thin line of almost underplaying in the opening in order to direct the game into situations that played to his strengths.

I have been running some analysis on games from Go Seigen's books "black opening" and "white opening", that I think is based on serialized articles in Go World from the 60s and 70s. It is actually striking how low the accuracy is for the first 50 moves. I think it comes down to style, a clash of styles and possibly recording errors in a couple of games. Interestingly there are many games that seem to be "corrected" versions of actual title matches, these corrections are definitely not "computer approved" since they are sometimes worse than the actual games.

Then I have also checked practice games by pros that I have observed online and the accuracy in the opening is actually really astonishing, even though the style is not very AI driven. These seem to be practice games (I haven't tried to ask what they are up to), so they may be more focused on being good training partners than anything else.

I have also started checking my games, and really concluded that accuracy alone is not really a good measurement of quality. Usually the first 50 moves are pretty good in my online games, judging by accuracy, percentage AI move and top-5 AI moves, but my experience or feeling is that it is not closely related to how good the effort was. Sometimes it seems like the flow of AI moves is with one or both of the players, especially if the game is quiet, but other times it is like every move is terribly wrong every time according to the computer even though the game seems completely reasonable given the level of players, but when this is the case it is usually not until the middle game

I'd be interested if time limits were taken into account, it is obviously a huge factor. I wouldn't be surprised if longer time meant much higher accuracy, but then I'd also think that very fast games could have better openings than some longer games simply because the players fall back to something less provocative that just works. So that would actually be very interesting.

[John Fairbairn]

Thanks for thr tip-off. The results don't surprise me, in the sense that they seem consistent with several other recent opinions. But the methodology here is way more impressive, of course.

A couple of remarks:

With regards to opening, all had significant differences compared to top pros, but Huang/Fang/Shi are not significantly different from Jowa/Shusaku.

In one sense, Huang, Fan and Shi may be more impressive in that they appear to have played faster than their Japanese counterparts.

1) Even though Dosaku era openings look a lot more reasonable by modern standards, they actually were significantly worse than the Huang/Fang/Shi openings which look a lot more weird, probably because modern opening theory is derived from Dosaku's lineage.

I suspect Dosaku himself may have come out worse because almost all his best games were giving handicaps. As to Chinese openings looking weird, that is a consequence of group tax, but Katago has already given the thumbs up to the old Chinese style with that rule applied. I am munching contentedly from a pack of dried figs on which the advertising blurb says "Naturally ugly but delicious". I think something similar can apply to old Chinese openings.

As a side note, even top amateurs had "more accurate" opening than the ancient players, indicating that there have been advancements in opening theory.

Is this a fair comparison, given that old no-komi go was a kind of asymmetric warfare, with Black able to adopt a different style of play from White? Shusaku's consistency ("I had Black") seems relevant to that.

[jlt]

Is this a fair comparison, given that old no-komi go was a kind of asymmetric warfare, with Black able to adopt a different style of play from White?

The komi can also be changed in Katago's settings.

[John Fairbairn]

The komi can also be changed in Katago's settings.

I know, and have said so. But if it's set at 0 Black can play safe and White has to gamble - asymmetric.

(And it might be worth noting Shusaku thought komi should be about 5.)

[Knotwilg]

Thanks for the good post Illuck.

There are a few statements that in my mind don't make logical sense, so please clarify:

- Dosaku's opening is comparatively weaker than the ancient Chinese
- today's top amateurs have stronger opening than the ancient Chinese, indicating an advancement in opening theory
- opening theory has been derived from Dosaku more so than from ancient Chinese

How does that fit together?

Thanks!

[jlt]

Does Katago play the opening in an asymmetric way when the komi is set at 0 ?

[John Fairbairn]

Does Katago play the opening in an asymmetric way when the komi is set at 0 ?

I don't actually know but it's not really relevant when you're a human just trying to win the game and not produce perfect play. Remember the days when Monte Carlo bots won every game against you by half a point? You would probably never say they were just half a point better than you.

One thing that comes up in AI analysis of Shusaku's play is sheer consistency. I assume that's reflecting his play-safe attitude. At least it makes him stand put from other players, I gather.

[kvasir]

Does Katago play the opening in an asymmetric way when the komi is set at 0 ?

It does, it is actually very practical when you use katago to analyze no komi and reverse komi games. For example the variations look a lot more "normal" in the first 50 moves for black in reverse komi games than in komi games. There is also a lot more slack in the point estimates, comparing the top few moves. This is for reasonable amount of playouts, basically with komi it will search lot of fantastic variations early but without komi or reverse komi more normal ideas seem to have priority.

Of course this is subjective but I'd say "definitely!", and I am not talking about in the limit with excessive playouts only the practical experience.

[illluck]

I have also started checking my games, and really concluded that accuracy alone is not really a good measurement of quality. Usually the first 50 moves are pretty good in my online games, judging by accuracy, percentage AI move and top-5 AI moves, but my experience or feeling is that it is not closely related to how good the effort was. Sometimes it seems like the flow of AI moves is with one or both of the players, especially if the game is quiet, but other times it is like every move is terribly wrong every time according to the computer even though the game seems completely reasonable given the level of players, but when this is the case it is usually not until the middle game

I'd be interested if time limits were taken into account, it is obviously a huge factor. I wouldn't be surprised if longer time meant much higher accuracy, but then I'd also think that very fast games could have better openings than some longer games simply because the players fall back to something less provocative that just works. So that would actually be very interesting.

A really good point about accuracy and quality - it does seem like over enough games samples there is a clear correlation, but definitely valid (for example, someone at the Baidu Tieba noted that the "accuracy" difference between Lee Sedol and Lee Changho was greater than between Lee Changho and top amateurs).

[illluck]

Is this a fair comparison, given that old no-komi go was a kind of asymmetric warfare, with Black able to adopt a different style of play from White? Shusaku's consistency ("I had Black") seems relevant to that.

Definitely a good point as well. My personal suspicion is that Dosaku's "accuracy" was lower partly due to how much stronger than his opponents he was (so he probably didn't have to be 100% focused) and partly due to him taking white in a lot of games where a certain degree of overplay was probably needed.

For a bit of an additional context, below is a table somewhat demonstrating "consistency"

I think Shusaku's games also included games where he took white (though probably smaller proportion compared to Dosaku?) Would be interesting to see if there is a big difference in "accuracy" between games as black vs games as white.

Edit: Forgot to mention that the table indicates average estimated winrate decreases from the "worst" N moves in each game.

[illluck]

Thanks for the good post Illuck.

There are a few statements that in my mind don't make logical sense, so please clarify:

- Dosaku's opening is comparatively weaker than the ancient Chinese
- today's top amateurs have stronger opening than the ancient Chinese, indicating an advancement in opening theory
- opening theory has been derived from Dosaku more so than from ancient Chinese

How does that fit together?

Thanks!

Just my two cents:

Partly this is due to Group Tax which changes opening slightly (for example, Kata under Group Tax do not favor early 3-3 invasions), so there is some inherent difference there (Kata with Group Tax was used to analyze the ancient Chinese games).

Another part is due to the fact that a lot of opening choices are more or less driven by fashion - there are many moves that are close to optimal in the opening, and the ancient Chinese players had a preference for one set (e.g. 9-3 loose pincer to respond to keima approach) which, while looking quite strange in today's eyes, doesn't actually lose much compared to the more "meta" responses (and under Group Tax settings has been played by Kata).

A reason the author proposed is that might be due to the four "seated stones" at the star points at the beginning of the game. That may also just give fewer chances for mistakes compared to empty corners, but how much that impact is I have no clue.

Edit: Just re-read what you wrote and realized that the above doesn't directly answer your question. If we take those three premises as true (which it might not be), one interpretation is as follows:

Dosaku (earlier era than Jowa/Shusaku and Fan/Shi, around same time as Huang), the oldest "ancient Japan" style opening in the context of the players analyzed - call it J1.

Jowa/Shusaku, slightly more recent "ancient Japanese" opening - call it J2.

Huang/Fan/Shi - I think study didn't really differentiate too much (though it might have and I just forgot), assume they are similar, "ancient Chinese" style opening (has Group Tax) - call it C1.

Modern Amateurs/Professionals - sourced mostly from Japanese teachings (though obviously with later developments by all three countries) as they were the strongest players in the late 1800's/early 1900's. Call it J3.

The paper's interpretation of the analysis results is that J1 < C1 = J2 < J3

Does that answer your question?

[John Fairbairn]

it is actually very practical when you use katago to analyze no komi and reverse komi games.

kvasir (or anyone): when looking at old Chinese go, does Katago make any allowance for the facts that points count in a seki and there may be an extra point for last dame. I have in mind Ohashi Hirofumi's assertion that in a half-point game with 90% win rate, a one-point loss (or presumably point counted in a seki) can reduce the win rate to 10%.

[kvasir]

I think you are in luck, John Fairbairn, because I saw a post somewhere that someone did this experiment of training weights for KataGo to use group tax rules. There were some examples of how the large enclosure for a hoshi was now preferred. I'll need to look for this post.


Edit:
viewtopic.php?f=18&t=18031&p=263305#p263305
You actually replied to this thread, so maybe this is not new to you. Anyway, if you use KaTrain you can actually set the rules to "Ancient Chinese".

Edit 2:
You can actually check the exact rules available:
https://lightvector.github.io/KataGo/rules.html