Counting liberties on large groups turns out to be a thing that is pretty difficult for a neural net with current known best architectures.

Leela Zero sticks to wanting to a spirit of wanting to minimize use of high-level human features and heuristics (that's the "zero"), and so does not provide liberty counts to the neural net as input. If you do provide them as input, although a bot may still have blind spots for tesuji or status of groups, I can confirm that blind spots for liberty shortages go away entirely, at least as far as I can tell.

I am reminded of the animals that can instantly distinguish between 5 and 6 objects, but cannot "count" any higher.



Since dame count is important for playing ladders, does providing them to the net improve the ability to play ladders?

Also, if you can provide dame count as input, what about komi?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Having successfully downloaded the Elf commented GoGod files and looked at a couple of games, I suspect that we should not give any winrate generated by Elf with fewer than 4k visits or playouts to have any credence at all. For instance, suppose that a play seems to be an error that costs 4 percentage points in the winrate, but is based on a visit count of 900. In the main line sequence, where Elf is playing both sides, a few moves later, where we anticipate that the actual winrate has not changed by more than a fraction of a point, the winrate is 2.5% better, with a visit count of 32k. I would go with the loss of 1.5% instead of 4%. (And 1.5% may well be within Elf's margin of error, anyway. )

When I get time, I'll generate a few statistics on this question.

Edit: Actual example from Segoe 7 dan vs. Go Seigen 4 dan, GoGod 1932-01-01a. Black 113 has a winrate estimate of 41% with 76k playouts. Segoe's next move was surely a mistake. Elf recommends a play with a Black winrate estimate of 41 with 79k playouts. Segoe's move has a winrate estimate for Black of 46% with only 610 playouts. It would show up in the winrate graph as a 5% error. However, Elf recommended play for Black 115 has a winrate estimate of 54% with 18k playouts, and Go Seigen's actual play has a winrate estimate of 55% with 12k playouts. So I would take take the winrate as 54-55%, and Segoe's error as costing 13-14%, a rather larger error than 5%. (You see why I only take the winrate estimates to two digits. )

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

For ladders, no, not significantly. Neural nets don't have a problem with counting liberties on small groups, just the ones that are large and often sea-urchin-like. And the ability to evaluate ladders is still critically poor on the positions where they haven't been played out yet, or haven't been played out much, where all groups involved are small.

The hard part of a ladder for the neural net is not anything liberty-related, but rather "understanding" that the stones diagonally a long distance away across empty space could possibly be relevant. Also, unlike humans, the bot is perfectly happy to read the ladder in one variation, solve it (because given enough playouts, the search does still solve it), and then fail to understand the same ladder in another variation, and another, and another. For every one of the dozens or hundreds of variations in any position, the search has to solve the ladder yet again from scratch. Humans would simply read it once and understand when a move could change the ladder's result and require a reread, but there's no currently known good way to make a bot do that in current architectures that "fits in" to the current neural-net driven search. That's for future research.

I've also experimented with adding ladderability of stones as an input feature too. The result is that the bot that never messes up common ladders (as far as I can tell) and has good evaluations for tactics that depend on them. The drawback is that it makes the bot weaker at solving rare positions like the Lee Sedol ladder game or that other Fine Art game where actually it's correct to chase a broken ladder across the board because the forcing moves gained will kill another group on that other side of the board, and where it requires actually chasing the ladder rather than simply playing a ladder breaker on that other side of the board directly. Because the bot, knowing that the ladder doesn't work, is less willing to spend reading effort on chasing the ladder out compared to a bot that has no idea if it works or not.

Since normal ladder situations are easily 100x or more common than situations where chasing broken ladders across the board is good (driving tesuji don't count, it's only the cases where you need to chase across a long distance that are tricky), for now I'm happy with this tradeoff, although I have some intuitions for future research on how one might try to get the best of both worlds.

Neural nets can easily handle a wide range of komi if komi is provided as an input and the training data contains a few percent of games with a wide range of komi. I do that in KataGo, and it works very well.

Here is another example of a major problem with winrate estimates, even of Elf. It also comes from GoGoD 1932-01-01a, with comments by Elf.

Go Seigen's move, Black 121, has a winrate estimate of 51% with 25k playouts. Then Segoe's move, White 122, has a Black winrate estimate of 50% with 54k playouts. Neither play has any variations, so presumably those plays were also Elf's first choices. So far, so good. Black 123 has four variations by Elf, each with a winrate estimate of 50% and playouts varying from 2k to 22k. The variation with 22k playouts does not have the comment, "Good", so it may not be Elf's first choice. I guess that Elf's first choice is Go Seigen's play, which has a winrate estimate of 58% with 30k playouts, and Elf's recommendation for White 124 has a winrate estimate of 58% with 49k playouts. Hello! That's a jump of 8% for no apparent reason. (Something like the horizon effect, I suppose.) But what does that tell us about Elf's winrate estimates? Even with several thousand playouts?

If humans are going to make use of bots' winrate estimates, we need to know something about their margins of error.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

THAT is the real problem. The issue isn't just the outcome of a ladder, but that the ladder (it's potential) makes a large number of remote moves sente. Or as in the sort of example given, playing out the ladder IS sente against some remote group (so either the ladder or the group)

A question about the diagram on the left side of Lizzie's interface. There is the x-axis which basically can be interpreted as total equal game, 50/50. But what about the one line below/above the x-axis? Why is it there? Does it mean that within that boundary the game is still open, outside basically lost? Because I realize that in lost games the graph usually exceeds this boundary by a lot.

I don't think they have any particular significance; they're just points of reference.

I think it's the 75/25% line. It's arbitrary, but indeed, it's close to the "serious advantage" limit

Since LZ starts Black with only 46% chance I wonder at what value it thinks the game is still on equal terms. E.g. if you have a value of 35% do you still have chances in the eye of LZ or is it a lost game?

Recent research by Viktor Lin ( https://explorebaduk.com/2019/04/08/how ... -research/ ) suggests that a winrate difference of around 10% translates to a difference of about 2 points of area/territory. OC, as we approach 100% or 0% that does not hold up. You would only have to be 10 points behind for Leela Zero to think you had a lost game. OC, at amateur levels that is not so. But if Black started with a 46% winrate estimate and dropped to a 35% estimate, she has probably lost around 2 points of area/territory to par.

Here is a quote from the site:



Edit: OC, a 2 point territory/area difference becomes more significant towards the end of the game.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.