Intro
I will probably again annoy a couple of people by doing this but I enjoy taking another look at professional commentaries with KataGo looking over my shoulder. In this case I'm rereading Lee Sedol's commented games Volume 1.

The first game featured is one early in his career, the 4th game in the finals of the Baedal Wang tournament, against Yoo Chang Hyuk. Lee desribes how impressed he is with his opponent, while determined to show his own prowess. The opening is described in full detail, including the variations he thought of during the game and why he or his opponent made the choice they made. All in all the commentary takes 120 pages, with full page game diagrams of only a couple of moves.

Discussion
Just like previous exercises, it is intriguing to see Lee Sedol claim certain variations to be "clearly better" or "intolerable" while KataGo makes no big deal of those. At one point he says "taking territory with 26 and 28 is certainly too passive and not my usual style. These moves demonstrate how immaturely I was reacting to the prestige of Yoo Chan Hyuk 9P." Both 26 and 28 are "blue moves".

Next, he says "29 is the wrong direction" (blue move), "30 is a common move with this shape but it was not good under these circumstances" and again 30 was the blue move. On the other hand 32 was "gote but thick" in his opinion, lost points according to KataGo. Throughout the opening there's hardly any correlation between Lee's claims and KataGo's evaluations. The score varies between 2,5 for White and 1,9 for Black but Lee seems to think he's constantly fighting an uphill battle because of the "mistakes at 26 and 28".

Lee highly values fighting spirit. "Moves like 67 and 69 reflect Yoo Chang Hyuk 9P's style well as he often uses neat haengma. However, a little more pressure is needed in this situation." Again, the good shapes of 67 and 69 are blue moves.

The next sequences allow White (Lee) to connect in sente and turn to a lingering ko. Lee describes this as "an excellent sequence" and "skilful connection". He seems to be happy with the flow of the game there, but KataGo thinks nothing really changes and Black is slightly ahead.

But then Yoo makes a dubious move and Lee says: "Black's hane 103 was an unexpected response to 102 (...) I felt this development was fairly comfortable for White".

Soon after, Yoo makes what Lee considers "the losing move". Lee: "Wedging at 107 is a tesuji that Black had been aiming for since (...). Instead of 108 I could also have drawn back (...) but I wanted to keep up the fighting spirit. Since it was quite difficult to reverse my disadvantageous position, I decided to continue actively. The losing move at 109 began with Yoo's optimism after judging himself ahead. If he had felt the situation to be close and had engrossed his mind in seeking the best move he would have found the hane at 113 easily."

After 107 KataGo has W+1,5. Lee's fighting spirit inspired 108 is a mistake. If Black plays the right move (the hane at 113) it's B+2. But 109 is indeed game losing: W+3,5. From that point onwards it's endgame. In that final stage Lee indeed maintains a stable lead which however fluctuates between 5,5 at 175 and only 1,5 at 196. Still Lee seems pretty comfortable there will be no upset.

Conclusion
I don't want to be dismissive of Lee's judgment. Rather it makes me think how our choices and evaluations are often influenced by personal motivations, much rather than by an objective, rational inspection of reality. The extreme intuition professional players have developed, is still laden with aspects of style, pride, fighting spirit, prestige ... Even at the time of writing the book, Lee seems to be biased by the feelings he remembers from when playing the game.

In Kahneman's book "thinking fast and slow" it is described how pathologies that affect the emotional center, make people unable to make choices, even if their rational, calculating abilities are unaffected. As if even if they knew the best move, there would no longer be a reason to play it.

Is this the game?
http://gokifu.com/s/etv-gokifu-20001204-Yoo_Changhyuk(9p)-Lee_Sedol(3p).html

You are annoying me by not including diagrams or the game record itself. Other than that it is interesting.

I have seen similar patterns in that commentaries are often not very objective. I think that is something that was clear to me before we had programs like KataGo but it can be hard to proof. Commentaries come in different flavors and they can have various degrees of objectivity. I don't have the Lee Sedol book but I think I know which book (books?) you are talking about and I think I browsed it.

I still appreciate commentaries because they can give you insight into how strong players analyze and think about positions. The way KataGo computes is often not a good way to think about the game. This again can be hard to proof for specific cases.

That's the one.

I didn't include any commented diagrams because at that point I might be infringing copyright, if not already.

I wanted to give enough information to get the gist of my meta-review across.

We may discuss the game here, reviewing with KataGo or other AI but I won't post any of Lee's diagrams.

Me too. Perhaps even more now that I have something to measure it up to, which is unbiased by Lee's judgment or persona.



KataGo's computations give me an objective measure. If it marks a move as positive and the professional commentary says why it is good, then there's a good chance we can learn something from it. Likewise if the professional explains why a move that KataGo marks as losing points, is bad. But if the two are contradictory, I'll take it that the pro evaluation, if not mistaken, is at least affected by personal preferences and should not be taken forward as truth, and it also makes KataGo's preference harder to understand, hence learn from.

Rather, doesn't this show how inapplicable some of Katago's suggestions are to human pros? Especially pros that haven't studied AI.

Aside, I'm still looking for a copy of Vol. 1. There was one in eBay recently but no luck. I did have other luck on eBay so no more complaints.

What I find fascinating from your post (and you allude to it in your conclusion) is the number of times that Lee and Yoo made the right move (or a fine move, anyway) during the game and evaluated it as poor later. It suggests that there is a difference in how Lee thinks while playing and how he thinks while writing a commentary, and the additional time for reflection is not purely beneficial. It’s possible to talk oneself out of a good move just as easily as it is to use logic to find the right move.

I think this suggests several important points for those of us who desire to evaluate our own play. First, it’s essential to continue to hone intuition, and the environment in which we do so is important. Learning to play the game and learning to evaluate the game are not identical skills. Second, in the age of instant AI analysis being available, a self review before looking at the computer’s suggestions is probably instructive. I think comparing our supposedly rational evaluation to a computer’s objective analysis can help us find holes in the heuristics we apply to move selection.

Thank for this; I found it very interesting!

AI analysis is not, repeat: NOT, objective. It may be helpful and provide a new view but this does not make it objective. AI can err and has erred, e.g., judged two very similar positions very differently or life and death status wrongly.

My thinking is that even if AI was 100% correct and never erred, that does not mean that an AI selected move is the correct move for a human player who cannot play at the same level as the AI. Just one example from this post:

26 and 28 might be blue moves for an AI because an AI knows how to take territory now and reduce the opponent's later. Whereas Lee Sedol, knowing his own play style, realizes that he cannot just take territory because he will not be able to reduce sufficiently later.

The AI suggested moves are the correct moves for the AI. Sometimes they are also the correct moves for a human. Sometimes not.

They are not absolute. But they are more objective because they are based on deep search. They calculate odds more precisely than pros.

With this argumentation there would be nothing to learn for amateurs by looking at professional games.

If a stronger player can play a better move because of their bigger skillset, then the weaker player should acquire that skillset - if they want to become better. There's nothing magical about AI. It's the same stones, the same rules, ... just better moves.

Fortunately professionals themselves take AI seriously and try learning from it. T

One "pro" (katago here) suggesting alternative and supposedly better moves for a pro game is nothing new. If you think about it, almost every comment made in the thousands upon thousands of commentaries by human pros on pro games is a case of the commentator suggesting a "blue" move for a "brown" move that the original pro thought was the best.

We tend to think of the latest comment as the latest state of the art, and so think of the latest commentator as the "god". This applies even to pros, apparently. You may recall in a different thread that I reported Hoshiai Shiho as saying that one of the big effects of AI for her was dispelling the notion that senior pros were gods.

But the situation is rather more complex than that. Nearly all my recent books have been pro-based commentaries on pro games, but these are not commentaries by a single pro. They are a composite of many pros' opinions (the most extreme case was one final commentary based, if I recall correctly, on over 50 different commentaries). What was remarkable about that, when I first tried it, was how much pros disagreed with each other. There are cases of the same move being regarded as brilliant by one group of pros, or as a bad mistake by others, with yet another group deciding that that move was not even worthy of a comment. There are few cases so extreme, but they are not isolated, and there are very many cases of a lower level of disagreement. I even incorporated AI comments in one book (First Teenage Meijin). This was because Shibano had become an aficionado of AI play, but apart from the obvious absence of "explanation" by the bot, the commenting process was not substantially different from the traditional method.

Indeed, there are cases of a pro disagreeing with himself. Go Seigen, for example, made a move he thought was best in the game, regretted it in the post mortem, then years later wrote a commentary saying his original move was the right one after all. What changed? Was it his increased understanding of go in the interim?

But is understanding even the right word? Much is made of the fact that AI bots can't explain their moves. Conversely, much is made of the fact that human pros can (and do) explain their moves, and when we read them we think we understand them. But if we really did understand, why is it that those of us (many) who have read hundreds, or even thousands, of commentaries are not 13-dan? In fact, I'd be reasonably confident in asserting that at least 90% of commentary readers end up no stronger than they were before. They may sometimes "understand" more, but then think that that understanding exempts them from the hard work of reading. As a result, they may even end up weaker.

Yet this fostering of a bad habit is not the only danger with seeking the enlightenment of so-called "understanding". A more subtle problem is that you may not have actually understood - a point being made by this thread, I think.

Another subtle problem is that, in the search for understanding, you try to devise all-encompassing theories or, worse, pseudo-algorithms that allow you to use numbers to count elements in an evaluation. Countless books on this idea in chess have so far simply prove this idea doesn't work.

My own experience with commentaries (rather vast, even if I say so myself) is that they do not have much value in promoting understanding. To make up a silly example, if a comment is talking about the best shape for a territory and says a move is good because it "represents the square on the hypotenuse", you would understand every word and you might even see a glimmer of a useful idea, especially if you are not a numbers guy. But it's sheer bunkum. You haven't actually understood anything. There's nothing there to understand.

What commentaries do provide, though, can nevertheless be very useful. I think that broadly they offer two things. One is a collection of "oh, I didn't know you could do that" moves. This seems to be the major contribution of AI, in fact. Even modern pros thought "oh, I didn't know you could do that" when they saw what bots were doing with Direct 3-3s, early shoulder hits and outrageous contact plays. We amateurs get the same experience even when we just look at life & death problems and joseki books. I think this is all truly, truly valuable, even when we don't "understand" the new move we've just seen.

The other facet of commentaries that I believe is useful is, rather than promoting understanding, in reducing uncertainty.

In life, not just on the go board, we are constantly faced with a welter of things we can never understand in the time available. Our brains have evolved ways of working that enable us to cope with this uncertainty that demands an instant response. This instant coping is based largely in reducing uncertainty, not on understanding. It is certainly different from how AI operates. The difference is important, because if you assume reducing uncertainty is the goal, that implies the situation is still uncertain even after you have reduced the uncertainty. From this it follows that a bot, another pro or you yourself can change your mind about the evaluation of a move in go. And after you have changed your mind, you can change it again later.

That leads on naturally to how you reduce uncertainty. That's a big question for another time - not least because the answers themselves are uncertain! But my own conclusions so far, in a nutshell, are that the answers lie in emphasising learning by experience (rather than understanding) in order to build up your intuition. You can never have too much of that. But once you've got something to work with, skill is demonstrated in the way you make your conscious brain work with your subconscious. Training or knowledge of related things can then help. But even here, reducing uncertainty is the key.

An easy but typical expression of this in go is that a player - even a top one like Yi Se-tol - can choose to play in a certain style, so that he ends up playing moves that may not be objectively the best but are moves that he is most comfortable with. Another way a similar idea is expressed is to fi find a plan and stick to it. Consistency is a way of reducing uncertainty.

My own gut feeling is that AI has supplied many new moves, but it is of no or little value in reducing the sort of uncertainty we humans have to cope with. Even if it learns to explain its moves, I'm not convinced it will help us much in playing better, except in a superficial way. Because it doesn't think the way our brains work, it can't genuinely reduce any uncertainty. In fact, by offering so many alternatives moves in a rainbow of colours, it may even increase the uncertainty! This may not matter much for pros, as they already have a high level of certainty, and they probably also get more out of the compensating rainbow of new ideas then we amateurs do. But I think it may matter for amateurs. For them, hard work (aka learning by experience) surely still has to be the ultimate answer. That, or just stick to being a go fan, in which case (as the similar but more evolved AI chess scene seems to show), human commentaries are the gateway to the best entertainment.

Contrary to John's experience to train intuition, replacing subconscious thinking by reasoning has enabled me to become stronger. Numbers in positional judgement, endgame or semeais reduce (and sometimes eliminate) uncertainty.

The second game Lee discusses in his book is his 3rd game in the 5th LG cup finals against Lee Chang Ho in 2001. Lee Sedol had won the first two games and was on the brink of victory.

With hindsight, having lost this game and next the finals, Lee is particularly critical of his own play.

At 12 he plays an unusual close extension on the 4th line which he will keep criticizing throughout the commentary. KataGo thinks it loses 0,7.

At 20 Lee plays an inside hane to a 3-3 invasion which he thinks is a fair choice. KataGo finds this a much worse mistake, losing 2,2. Lee goes on to criticize the 22-23 exchange which according to KataGo makes no difference

With 37 and 47 Black squanders his advantage. Lee says: "If Black plays [the blue move] instead of 37 (...) the result for Black is worse than in the actual game. After 47 Lee says: "When looking back calmly this result seems to be very balanced for both players. However, White played the slack move of [12] before, so the whole board position is favorable for Black." KataGo has B+0,6 at this point.

After 53 the score goes down to 0,2. Lee says "Even though there weren't any special mistakes made by Black or White, except for the extension at [12], White's flow is unfavorable. It demonstrates how bad [12] was".

Here Lee's anger with his move at 12 is completely taking over. Since that move, Black has gone up and down by 3 points, while that move hardly made a difference.

Then Lee says: "Creating another weak group at 54 was definitely an overplay. This was a splendid chance for Black to grap the victory. However Black attached with 55 and missed his chance by making a critical mistake in direction".

KataGo thinks 54 is suboptimal but for very different reasons than Lee. KataGo wants to play elsewhere. As for 55, the "critical mistake in direction", KataGo thinks that's the best move on the board.

The crucial moment is the 99-100 exchange. At this point Lee is already well ahead due to consecutive mistakes by Black. Lee ChangHo manages to lure Lee Sedol into a big ko, while Lee Sedol could have simplified the game. Here KataGo fully agrees with the assessment.

And here you go: http://gokifu.com/s/ei8-gokifu-20010515-Lee_Changho(9p)-Lee_Sedol(3p).html

Indeed.

A mountaineer can learn from an experienced colleague. If he follows the path taken by a mountain goat, he will fail, fall and die.

One question I have is if Lee Sedol actually wrote any of this. Ghostwriting takes many forms and it is sadly rather common.

Regarding the first game:

Giving ponuki at move 56 results in 4-5 point loss assuming white is playing thickly at move 96 to prepare for the ko. This seems to me like the kind of active but bad plan that loses games but black doesn't fight the ko and therefore gives some ground back. Is it bad technique or a special plan? If I did the same it would be bad technique but I can't speak for Lee Sedol and he is well know for playing aggressively and make moves that are hard to counter in practice.

When looking at the KataGo analysis it is hard to understand why 107 is called a losing move, I can only assume there are variations in the book. This is one difficulty with using the computer, it sometimes makes it look easy. In this case 107 is one of a few moves KataGo thinks reduces black enough but this is a very hard call in practice. However, there is a human tangible here and that is black didn't fight back when white ate the stone. The mistake is to play 107 and give up the stone in a way that is really bad, maybe it is not useless to put the "losing move" at the point when black made the mistake in his mind.

After move 126 my KataGo says W+0.4. I think that raises the question if the *final* reason black lost was endgame because black loses ground for the next 30 or so moves until it is W+5.6. It must be hard to impossible for anyone to beat Lee Sedol in endgame when behind by so much but less than a half point is not a lead. Somehow I take KataGo's estimates for granted in the previous sentence without trying to estimate myself, it is another pitfall of this kind of analysis.

I argued that Lee Sedol knew he needed to play a different move and you argued that he should study how his actual move was potentially the better move all along. Of course it makes sense to study moves and see how they work better or not. But like I said, Lee Sedol knows himself best and after review of his own game he thought that he did not play the best move for himself,

He's not wrong.

I might be completely mistaken, but I thought the main thing that Go game commentaries provided (like all game commentaries) was entertainment. I'm not saying that some people don't study game commentaries, but I believe that the teachings are intended to deepen appreciation and enjoyment of the game. Not to improve skill.

Right, 90% of people are enjoying entertainment. Not studying.

By the way, I find your books to be entertaining. Maybe I've gotten weaker but I had fun doing it.