Life In 19x19

Thanks, Bill. I can't claim to be any closer to understanding MoE in a meaningful way, but at least I now have some insight into why you belaboured the point, and I feel (intuitively) even more strongly that you were right to do so.

When I was a journalist one of my jobs was to go and listen a couple of times every month to civil servant statisticians supposedly explaining the latest inflation or unemployment figures to us. I was already familiar with Disraeli's adage that there are lies, damned lies and statistics. I soon came to understand that government statistics should be the next step in that list. From what you say, political pollsters belong in the same group. And I suppose nowadays we have to add fake statistics - but in place 4 or place 5 ?

John Fairbairn wrote:When I was a journalist one of my jobs was to go and listen a couple of times every month to civil servant statisticians supposedly explaining the latest inflation or unemployment figures to us. I was already familiar with Disraeli's adage that there are lies, damned lies and statistics. I soon came to understand that government statistics should be the next step in that list.

In Disraeli's day statistics meant the numbers compiled by the state.

From what you say, political pollsters belong in the same group. And I suppose nowadays we have to add fake statistics - but in place 4 or place 5 ?

Four out of five doctors recommend. . . .

Pollsters do not have it easy, if they want to do good work. Too much demand, too little time. Usually not time enough to test the polls. And wording matters. You can change the wording to mean what you think is the same thing and get a different result. There are other problems, as well. E.g.,"Dewey beats Truman." — "Clinton has a 98% chance of beating Trump."

Back in the 90s I happened to get the text of a political opinion poll. Do you prefer policy A or policy B? Each policy was described in a short paragraph. Right away I saw a problem. To check out my hunch, I counted the words in each paragraph and figured that the paragraph with more words in it was the popular choice. I scored 100%.

Bill Spight wrote:
Suppose that we have a winrate estimate for Black in a given position with White to play of 87%. What does that mean? Well, it is said that it means that Black has an estimated 87% probability of winning the game. That's ambiguous. First, there are two main schools of probability, frequentist and Bayesian. The Bayesian school is the oldest and newest. The frequentist school replaced it in the late 19th to early 20th century, but Bayesian probability made a comeback in the late 20th century. Both are in use today. The frequentist school says that a single event has no fractional probability; it either occurs or it doesn't. The probability is 0 or 1. By that thinking the margin of error of the 87% estimate is 87%. That's no help.

But suppose that we were to play out the game several times from that position with White to play, where the AI plays both sides with the same randomized, non-deterministic strategy. Then sometimes Black will win and sometimes White will win. These results will give us a second and different, statistical estimate of Black's winrate under those conditions of play. Furthermore, we can calculate the standard error of that winrate estimate and from that derive a margin of error within which we expect that the actual winrate lies. Typically we find the margin of error by multiplying the standard error by 2. We expect that the actual winrate will lie within that margin of error approximately 95% of the time. Obviously, this is not how the program makes its original winrate estimate. If the difference between the program's original winrate estimate and the statistical estimate is greater than the latter's margin of error, we get suspicious of the original estimate.

But note that we have only calculated the margin of error of the statistical estimate, not the margin of error of the original winrate estimate, which is what we want. What I want, anyway. We have found 1 error estimate, that's all.

There's a subtle but important point that might not be obvious to people.

Suppose I perform two self-play training runs, run A and run B, where the bot from run A plays with some amount of randomness and the bot from run B, except for perhaps the early opening, plays entirely deterministically. And yes, you can do this with AlphaZero-style training! As long as there is sufficient randomization in the variety of positions and trajectories you feed the bot as input, the whole self-play training process works even if at every particular moment in time the bot itself then behaves deterministically in those positions. Basically, this is because the neural net can't tell the difference, either way it sees the same thing: a never-ending stream of positions, never repeating (except for the very-early opening), each one labeled as "win" or "loss", without a way to tell counterfactually whether another alternative would have been possible.

So say I produce a final bot A and bot B that are both about equally strong. For any particular choice of search depth and other settings, Bot B is deterministic, but if you were to play a game against either one, you likely would not be able to tell the difference between A and B. Perhaps maybe the programmer hardcodedly adds some "artificial" randomization to the first 20 moves, so that even with several games, you normally wouldn't be able to tell. And also if you used them for game analysis, both of the would still report the same fuzzy winrates, like 30%, or 75%, or whatever (yes, bot B would still report such winrates even though it learned from data produced via "deterministic" selfplay).

Now we consider applying this procedure:

But suppose that we were to play out the game several times from that position with White to play, where the AI plays both sides with the same randomized, non-deterministic strategy. Then sometimes Black will win and sometimes White will win. These results will give us a second and different, statistical estimate of Black's winrate under those conditions of play.

The two bots vary drastically on the metric of this "second and different statistical estimate". For bot A, you'll get some fuzzy percentage. For bot B, you'll get either 0 or 1 - because it doesn't have a randomized nondeterministic strategy. Despite that, the winrate is still meaningful and about the same from the user's perspective - maximizing it still gives you superhuman play, and it looks about the same in many cases as bot A's winrates. And among "90%" positions for bot B, most of them do indeed play out to be wins. Nonetheless you took any sigle "90%" position for bot B, playing that position out repeatedly will either win 100% of the time or lose 100% of the time, so the "margin of error" on that position by this metric would be... well... either 10% or 90%.

The takeaway of this thought experiment is to make it more stark how, this metric - "how often does the bot actually win if repeatedly played from this position" - cannot, alone, be the story of what you want (very much agreeing with what Bill said). Despite varying drastically on this metric, in many of the ways that matter from a practical user perspective, the two bots are very similar. For both of them, the user is faced with the question "how much can I trust the bot's evaluations" and for both bots those winrates are often about the same. What this thought experiment shows is that in some cases, what you're measuring like this has less to do with "how much should I trust those winrates I see on my screen" and more to do with irrelevant details of the nature of the bot's implementation that don't significantly affect those winrates.

Okay, maybe this metric still tells you something useful in practice with actual bots, even if not the whole story. That's why I brought it up myself too as a starting point. But still, the question arises what is the right metric, and how do you measure it?

This is partly why I dislike the phrasing "the margin of error". Because repeatedly campaigning for research on "the margin of error", linguistically, loosely gives the impression to people listening in that "the margin of error" is a concrete thing that is already agreed-upon, unique, and well-defined. It's known how to compute it, and it's just up to those darned programmers to finally start doing so and reporting it. When, as Robert Jasiek pointed out, actually all we have that would be tractable are different proxies, no bright-line or canonical choice of which one to pick or how good they are.

Instead of "we should do more research into ways of the margin of error of bot winrates"...
I like the phrasing "we should do more research into ways of quantifying or measuring the uncertainty of bot winrates".
And instead of "in this situation, I estimate the bot's margin of error is X"...
I like "in {situations like this, openings, endgames, this exact case I tested}, on average the bot {fluctuates, changes-its-mind, disagrees, ...} with {itself, other bots, the empirical winrate from rolling out the game repeatedly,...} by X" (in each case, pick words that make it less ambiguous how you got X and what it represents).

Right now, a nontrivial part of the task isn't ready for the software engineers. Rather it's up to the people who happen to be both Go players and skilled mathematicians, to come up with proposals for the mathematical quantity to be measured that will most likely correspond to what their Go-intuitions want to see. Or at least, if the latter don't have clear proposals yet, then it at least shouldn't be a surprise if the former haven't coded it up yet.

Another way to demonstrate the noise: symmetric positions. Examples:

- Set up a parallel nirensei game for both: Q16, D16, Q4, D4. Which keima is better, R17 or R3?
- Set up a diagonal four-hoshi game: Q16, D16, D4, Q4. Now there are four equivalent keima. What is KataGo's score for each?

For Harleqin:
For a position that is: very familiar, with no complex tactics, plenty of visits, and making sure to compare apples to apples (approximately equal visits invested) - not much noise really. Whether "right" or "wrong", the bot has a pretty self-consistent opinion. If you alter those conditions, you might get different results.

Thanks, lightvector. You addressed some questions I was going to ask you later.

I agree that there is no such thing as the margin of error. It depends upon what number you multiply the standard error by. And that is a matter of judgement and intent. I tried to clarify that point in my post.

Sure, but also it's a bit more than just picking what number to multiply by once you've gathered stats and computed a standard error, there's also what stats one is supposed to gather in the first place. It's the issue of what stats to gather ("error relative to what?") not having a unique obvious answer that is why I'm a bit dissatisfied with phrasing that suggests there is a unique clear thing already.

If you have a concrete proposal that is better than what has been suggested so far ("error relative to what? how about error relative to empirical self-play win chances?"... for all its obvious flaws), let me know and I may implement it. Otherwise, we will continue to proceed as is and for lack of a clear better thing to try, perhaps less will be tried in the short term. Which is also fine, certainly the progress of AI in Go in just a several years has already been quicker than almost anyone expected before that.

lightvector wrote:Sure, but also it's a bit more than just picking what number to multiply by once you've gathered stats and computed a standard error, there's also what stats one is supposed to gather in the first place. It's the issue of what stats to gather ("error relative to what?") not having a unique obvious answer that is why I'm a bit dissatisfied with phrasing that suggests there is a unique clear thing already.

If you have a concrete proposal that is better than what has been suggested so far ("error relative to what? how about error relative to empirical self-play win chances?"... for all its obvious flaws), let me know and I may implement it. Otherwise, we will continue to proceed as is and for lack of a clear better thing to try, perhaps less will be tried in the short term. Which is also fine, certainly the progress of AI in Go in just a several years has already been quicker than almost anyone expected before that.

Last year, before the virus hit, I considered developing an analyst program, and guessed that, given my resources, that it might take me 5 years or so. Meanwhile, people are developing stronger and stronger programs, and the sky's the limit. But at least I would have an analyst.

The virus, however, has forced me to reorder my priorities. Before the vaccines I doubted if I would last 5 years. Now I think maybe 10, but the vaccines have not altered my new priorities.

In terms of classical reinforcement learning, Tesauro's temporal difference learning approach has a lot going for it. You are not reinforcing every choice made in a game all at once, but giving immediate feedback on each choice. I imagine that the AlphaGo team tried that approach out, but probably found that it was not efficient for developing a strong player. It still might be good for developing an analyst, however. I would want an analyst to assess specific plays. I may give it a shot, some time down the road.

As for comparing options, I still think that playing difference games is a good idea. Perfect play is not required to get correct results, and the results can be readily interpreted in terms of traditional human go evaluation. Aside from the fact that you double the playing area, how to handle ko fights is an open question.

Thanks, lightvector!

I guess that it is expected that the scores are very similar with »plenty of visits«, but while the score differences seem to be below 0.1 here, you can still see winrate differences of up to 0.2%-points.

With lower visits, which many of the convenient web tools using KataGo (e. g. katagui, ai-sensei, ogs) use, I see higher variance.

ez4u wrote:

Bill wrote:AFAICT, there is little, if any, research into whether a play that a bot rates as inferior is actually a mistake. Especially in the opening.

What would you consider a valid research method for checking bot recommendations? We already know that bot ratings change as we allocate more visits/playouts to a position and that different bots produce different recommendations. Do you think that some form of human analysis would be more valid?

I don't want to hijack this thread, and besides, I have touched on a number of matters in discussion with John Fairbairn and lightvector. I do plan to revisit these questions some time this year. And I owe some people replies on other matters, and I appreciate their patience with me.

Current top bots report winrates as evaluations. Ostensibly they represent the probabilities of winning the game, but it seems that they have never been validated as such.

Some years ago Ales Cieply provided some game reviews by Leela11 with winrates at both 100k rollouts and 200k rollouts. OC, we may take the top choices with 200k rollouts as at least as good or better than those with 100k rollouts, as a rule. But that does not mean that the top choice with 100k rollouts in such cases is actually a mistake. The difference in evaluations could just be random. I realized that if the difference was not random, it should persist over time. I found that if the difference was around 3% I could be confident that it would persist. So as a rule of thumb I took 3% as the margin of error for Leela11 with 100k rollouts, figuring that any play with a winrate at least 3% worse than that of Leela's top choice was very likely to be a mistake. OC, Leela11 with 200k rollouts was no guru, but go is not solved, so we can do no better than rules of thumb, unless we can actually prove that a move is a mistake.

Go does have absolute margins of error with regard to optimal minimax play in terms of scores. This is a natural way for us humans to evaluate positions and plays. It seems pretty clear that a loss of 1 point of territory is an error. Losing 1 pt. per move on average by comparison with your opponent's play means that you should be taking 9 stones or more. An estimated loss of ½ pt. probably indicates a mistake, don't you think? Whether a difference is noise or not can be reckoned by how it persists over time. Humans, I think, can get a good feel for that with some practice. In my 20s I had the good fortune to spend an afternoon talking statistics with the great Bayesian, L. J. Savage. OC, I was in over my head, but I still remember his kindness and generosity with his time. He gave me a pamphlet he had written about the best statistical test, the interocular traumatic test. It hits you right between the eyes.

I played white in this game and stopped a bit early; my mistakes by move 50 totaled about 27 points.

Game 24, position 1

Click Here To Show Diagram Code

[go]$$Bc White [b]a[/b] or [b]b[/b]?
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . 3 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 6 2 a . . . . , . . . . . 1 . . . |
$$ | . . 5 7 . b . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

My hypothesis from last game was that white should play a to emphasize the left side or b for flexibility to choose either the left or the bottom. Based on that theory, I should probably have played a here, but I wasn't sure if was decisive enough to choose the left side. I ended up playing b, but KataGo advises a.

Game 24, position 2

Click Here To Show Diagram Code

[go]$$Bc White [b]a[/b], [b]b[/b], [b]c[/b], or [b]d[/b]?
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . c 1 b . . . . . . . . . . . . . |
$$ | . . . X O . . . . . . . . . . . . . . |
$$ | . O . X O a d . . . . . . . . . . . . |
$$ | . . O O X O . . . , . . . . . X . . . |
$$ | . . X X X O . . . . . . . . . . . . . |
$$ | . . . . . X O . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Here's the famous modern joseki. How does white answer ? Note c would be a mistake. I ended up playing b, which is one of the joseki continuations, but a or d would be about 1 point better here.

Variation

Click Here To Show Diagram Code

[go]$$Wc :w1: at [b]d[/b] from the previous diagram
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 9 , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . 7 5 3 X . . . . . . . . . . . . . . |
$$ | 8 6 4 X O . . . . . . . . . . . . . . |
$$ | . O . X O . 1 . . . . . . . . . . . . |
$$ | . . O O X O . . . , . . . . . X . . . |
$$ | . 2 X X X O . . . . . . . . . . . . . |
$$ | . . . . a X O . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

White can strengthen the right-facing wall with . Now a black move at a would no longer be sente, so black must play to live. Now white can sacrifice three stones to enclose black with .

Game 24, position 3

Click Here To Show Diagram Code

[go]$$Bc White to answer :b1:. [b]a[/b], [b]b[/b], [b]c[/b], [b]d[/b], or [b]e[/b]?
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . c . . . . . . . . . . . . . . . . . |
$$ | . X b . a 1 . . . . . . . . . . . . . |
$$ | . O X . X O d . . . . . . . . . . . . |
$$ | . O . X O . . . . . . . . . . . . . . |
$$ | . O . X O . . . . . . . . . . . . . . |
$$ | . . O O X O . . . , . . . . . X . . . |
$$ | . O X X X O . O . . . . . . . . . . . |
$$ | O X X . X X O . . . . . . . . . . . . |
$$ | . . . e . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

I unfortunately played d. This is the worst move of the bunch, as it does nothing to help white win the capturing race.

Variation

Click Here To Show Diagram Code

[go]$$Wc :w1: at [b]a[/b] from the previous diagram
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . 3 . . . . . . . . . . . . . . . . . |
$$ | 4 X a . 1 X . . . . . . . . . . . . . |
$$ | . O X 2 X O . . . . . . . . . . . . . |
$$ | . O . X O . . . . . . . . . . . . . . |
$$ | . O . X O . . . . . . . . . . . . . . |
$$ | . . O O X O . . . , . . . . . X . . . |
$$ | . O X X X O . O . . . . . . . . . . . |
$$ | O X X . X X O . . . . . . . . . . . . |
$$ | . . . 5 . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

This tesuji seems to gain white one liberty, as black would need to defend the cutting point at a.

I played black in this game and lost about 35 points by move 80.

Game 25, position 1

Click Here To Show Diagram Code

[go]$$Bc Pincer joseki
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . . . . . . . . . . . |
$$ | . . . , a . . . . , . . . . . , 1 . . |
$$ | . . 5 . . c . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . b . . . . . . . . . . . . . . . |
$$ | . . 6 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 2 . . . . . , . . . . . 3 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

White played this two-space low pincer. I'm not very familiar with this pincer. I played at a, which it seems is indeed the most efficient response here, but I can also consider c as an option that avoids local complexity.

Game 25, position 2

Click Here To Show Diagram Code

[go]$$Bc Crosscut fight
$$ --------------------
$$ | . . . . . . . . . .
$$ | . . . . . . . . . .
$$ | . . . O b . . . . .
$$ | . . . 2 1 a . . . ,
$$ | . . X 3 4 . . . . .
$$ | . . . . . . . . . .
$$ | . . . c . . . . . .
$$ | . . O . . . . . . .
$$ | . . . . . . . . . .
$$ | . . . , . . . . . ,[/go]

After the crosscut, black should continue the fight with either a or b. Instead I backed down with c, which is losing about 2.5 points.

Click Here To Show Diagram Code

[go]$$Bc Variation
$$ --------------------
$$ | . . . . . . . . . .
$$ | . 8 . . . 6 . . . .
$$ | . . . O . . 7 . . .
$$ | . 3 . O X 1 . . . ,
$$ | . . X X O . . . . .
$$ | . 5 . 4 2 . . . . .
$$ | . . 9 . . . . . . .
$$ | . . O . . . . . . .
$$ | . . . . . . . . . .
$$ | . . . , . . . . . ,[/go]

There are many variations from the crosscut. Here's one example.

I played white in this game and lost about 35 points by move 80.

Game 26, position 1

Click Here To Show Diagram Code

[go]$$Bc No mistakes so far (for white)
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 1 . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 5 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . b . . . . . . . . . . . . . |
$$ | . . 3 c 6 d . . . , . . . . . 4 . . . |
$$ | . . . . 7 8 . . . . . . . . . . . . . |
$$ | . . . . 9 a . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

How do we decide which approach to play against a 3-4 stone? Recently, if there are other unresolved 3-4 (or 5-3 or 5-4) corners, then I play a low approach. If the other corners are all e.g. 4-4 points, then I play a high approach as in this case.

is new to me. I decided to play the solid connection at d, which does seem to be correct here. However, I think it's interesting that a, b, and c all got higher policy scores.

Game 26, position 2 + variation

Click Here To Show Diagram Code

[go]$$Bc Joseki continuation
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 7 . . . . . . . . . . . . . . . . |
$$ | . . . 6 . . . . . . . . . . . . . . . |
$$ | . . 5 4 . . . . . . . . . . . . . . . |
$$ | . . 3 2 . . . . . . . . . . . . . . . |
$$ | . . X 1 O O . . . a . . . . . O . . . |
$$ | . . . . X O . . . . . . . 8 . . . . . |
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

The joseki continued with , after which I played a. But white should have continued with . One possible continuation is shown.

Game 26, position 3

Click Here To Show Diagram Code

[go]$$Bc How to answer :b1:?
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . d . . . . . c . 1 . . . . . |
$$ | . . . X . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . b a . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X X O O . . . O . . . . . O . . . |
$$ | . . . . X O . . . . . . . . . . . . . |
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Black is developing the upper side. White should reduce this potential somehow. In the game I simply played a because it is very typical locally. However, b, c, or d would be slightly better in this case. I like d as it feels quite active.

Game 26, position 4

Click Here To Show Diagram Code

[go]$$Bc Choosing the direction
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . X . . . . . |
$$ | . . . X . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . O . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X X O O . . . O . . . . a O 2 . . |
$$ | . . . . X O . . . . . . . 4 . 3 1 . . |
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

is the wrong direction. However, is correct (i.e. it is better than a). This seems like a point of data supporting my recent hypothesis that a prioritizes the right side while is more flexible regarding direction of play.

hakuseki wrote:Game 26, position 1

$$Bc No mistakes so far (for white)
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 1 . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 5 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . b . . . . . . . . . . . . . |
$$ | . . 3 c 6 d . . . , . . . . . 4 . . . |
$$ | . . . . 7 8 . . . . . . . . . . . . . |
$$ | . . . . 9 a . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------

Click Here To Show Diagram Code

[go]$$Bc No mistakes so far (for white)
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 1 . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 5 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . b . . . . . . . . . . . . . |
$$ | . . 3 c 6 d . . . , . . . . . 4 . . . |
$$ | . . . . 7 8 . . . . . . . . . . . . . |
$$ | . . . . 9 a . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

How do we decide which approach to play against a 3-4 stone? Recently, if there are other unresolved 3-4 (or 5-3 or 5-4) corners, then I play a low approach. If the other corners are all e.g. 4-4 points, then I play a high approach as in this case.

is new to me. I decided to play the solid connection at d, which does seem to be correct here. However, I think it's interesting that a, b, and c all got higher policy scores.

Here is a similar position from GoGoD 1950-08-10a, 4½ komi.

Click Here To Show Diagram Code

[go]$$Bcm9 Kitani (W) - Takagawa
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . O 1 . . . . . . . . . . . . . |
$$ | . . . . O X . . . . . . . . . . . . . |
$$ | . . O 2 X . . . . , . . . . . X . . . |
$$ | . . . . 3 . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

And there's this position from GoGoD 2005-05-20a, 6½ komi.

Click Here To Show Diagram Code

[go]$$Bcm11 Seo Pong-su (W) - Pak Seung-hyeon
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . 1 X O 4 . . . . . . . . . . . . . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . 3 X . . |
$$ | . . . . . . . . . . . . . . . X O . . |
$$ | . . . O . . . . . , . . . . . , 2 . . |
$$ | . . . . . . . . . . . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

I played white in this online game. My mistakes totaled 12.6 points by move 80 and 43.3 points overall. This is a good result for me, but I still made a lot of mistakes that are clearer in retrospect, so I decided to review.

Game 27, position 1

Click Here To Show Diagram Code

[go]$$Bc White to tenuki
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 2 . . . . . , . . . . . 1 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 5 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . a b . . |
$$ | . . . 4 . . . . . , . . . . . , . . . |
$$ | . . . . . c . . . . . . . . . 3 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Since approaching a 3-4 point is more impactful than approaching a 4-4 point, I figured I should tenuki here and played b. However, since white doesn't fear a double approach, it would be better to play a, which takes gote for a better local result.

Game 27, position 2

Click Here To Show Diagram Code

[go]$$W
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . a . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . b . . 3 1 . . |
$$ | . . . O . . . . . , . . . . . , 2 . . |
$$ | . . . . . c . . . . . . . 4 . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

The joseki would be to settle my stones at a. But instead I played c, planning to jump to b if black pincers the two stones on the right. However, as already stated, it's really not urgent to avoid a double approach, so the extension at a would have been better (by about 0.7 points).

Game 27, position 3

Click Here To Show Diagram Code

[go]$$Wc Developing the upper-right
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . 4 . . . . . . |
$$ | . . . O . . . . . , . . b . d X . 6 . |
$$ | . . . . . . . . . . . . . a . . . . . |
$$ | . . . . . . . . . . . . . . e . 5 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . c . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 2 . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . 3 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O O . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . 1 . . . . . . . X . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

By the way, is a move I wouldn't have known to play before beginning my studies with KataGo. I used to think an extension like makes an approach unviable, but that is not at all the case.
Both sides seem to be developing the upper-right, so I think white should play another move in this area.
Unfortunately, I settled on the rather dull move of e, which is losing about 0.6 points. It neither secures the right side, nor truly threatens black's corner. Other possibilities are a-d.

Click Here To Show Diagram Code

[go]$$Wc Variation [b]c[/b]
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . X . . . . . . |
$$ | . . . O . . . . . , . . . . . X . X . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . O . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . 1 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O O . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . O . . . . . . . X . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

This move is the second-best option, and perhaps the easiest to understand. It's not very threatening, but it calmly secures white's side.

Click Here To Show Diagram Code

[go]$$Wc Variation [b]a[/b]
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . X a b . . . . |
$$ | . . . O . . . . . , . d . . c X . X . |
$$ | . . . . . . . . . . . . . 1 . . . . . |
$$ | . . . . . . . . . . . . . . . . O . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . e . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O O . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . O . . . . . . . X . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

steals the vital point of black's ideal double-knight's-move shape.
White is threatening to split black apart at a. Black should play a defensive move, such as b, c, or d. Then white will finish at e.

Game 27, position 4

Click Here To Show Diagram Code

[go]$$Wc Too many options
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . 3 . . . . . . X . . . . . . |
$$ | . . . O . . . . . , . . b . . X . X . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 2 . . . . . . . . . . . 1 . O . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . i . . . . . . . . . . . . . . . . |
$$ | . . . f . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . h . . . . . . . . . . . O . . . |
$$ | . . g . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . c d . . . . . . . . . . . . O O . . |
$$ | . e . O . . . . . 4 . . . . a , X . . |
$$ | . . . . . O . . . . . . . X . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

White's strategic goals here may be to expand the moyo on the right, to help the lower-left corner (which is blocked in by black extensions on both sides), or to split up black's left side.
I didn't really think about splitting up the left side because I thought this might just induce an attack on my corner. However, it seems to be quite viable.
I chose to play at e but since this is gote, c or d is likely a better option locally.
However, b, which helps to expand the right-side moyo, might be the strongest play.

Click Here To Show Diagram Code

[go]$$Wc Variation 1
$$ --------------------
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ . . . X 2 . . . . . |
$$ , 3 . 1 . . X . X . |
$$ . . . . . . . . . . |
$$ . . . . . O . O . . |
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ , . . . . . , . . . |[/go]

If black simply defends at , then white jumps to , which flexibly develops the top, center, and right side while keeping black's corner small.

Click Here To Show Diagram Code

[go]$$Wc Variation 2
$$ --------------------
$$ . . . . . . . . . . |
$$ . . . 5 . . . . . . |
$$ . . 3 X 6 . . . . . |
$$ , . . 1 2 . X . X . |
$$ . . . . 4 . . . . . |
$$ . . . . . O . O . . |
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ . . . . . . 7 . . . |
$$ , . . . . . , . . . |[/go]

If black plays the inside hane at , then we might see this kind of development -- with some sente sequences interspersed elsewhere on the board.

Click Here To Show Diagram Code

[go]$$Wc Variation 3
$$ --------------------
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ . . . X b a . . . . |
$$ , . 2 1 . 3 X . X . |
$$ . . . . . . c . . . |
$$ . . . . . O . O . . |
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ . . . . . . . . . . |
$$ , . . . . . , . . . |[/go]

If black plays the outside hane at , then splits black apart. A black move at a would be answered at b, so c is black's best option.

I think all these variations may bore readers, but at several times during the opening I found myself searching for a useful move to play in the upper-right corner, and the move I eventually played there was a bit lackluster. So these variations point to a blind spot in my reading.

Game 27, position 5

Click Here To Show Diagram Code

[go]$$Bc Misplayed reduction on the left
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . 1 . . X . . . . . . |
$$ | . . . O . . . . . , . . . . . X . X . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . O . O . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 9 X . . . . . , . . . . . , . . . |
$$ | . . 4 5 . . . . . . . . . . . O . . . |
$$ | . . 2 3 . . . . . . . . . . . . . . . |
$$ | . . 6 7 . . . . . . . . . . . . . . . |
$$ | . 8 X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O O . . |
$$ | . O . O . . . . . X . . . . . , X . . |
$$ | . . . . . O . . . . . . . X . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

After black's tenuki to I decided to split up the left side. Actually, both players made mistakes in this sequence, but the resulting position is what I expected. However, my positional judgment was wrong as this result is good for black. Although some potential territory has been erased, black's shape is nearly perfect while white has weaknesses.
The first mistaken move was :

Click Here To Show Diagram Code

[go]$$Bc Variation
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . 1 . . X . . . . . . |
$$ | . . . O . . . . . , . . . . . X . X . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . O . O . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . a b . . . . . . . . . . . . . . . . |
$$ | . . c 9 . . . . . . . . . . . . . . . |
$$ | . . 8 X . . . . . , . . . . . , . . . |
$$ | . . 6 4 5 . . . . . . . . . . O . . . |
$$ | . . 2 3 . . . . . . . . . . . . . . . |
$$ | . . . 7 . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O O . . |
$$ | . O . O . . . . . X . . . . . , X . . |
$$ | . . . . . O . . . . . . . X . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Instead, white should use to wedge and create cutting points in black's shape. White can continue with a, b, or c.