Bill Spight wrote:BTW, what if you give Elf this position with Black to play?
$$Bc Easier corner
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . X . O O . O . . . . . . . . . . . . |
$$ | . . O , X X O . . , . . . . . X . . . |
$$ | . X X X . X X O . . . . . . . . . . . |
$$ | . . . O O . X X . . . . . . . . X . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . O . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . O , . . . . . , . O . . . X . . . |
$$ | . . . . . . . X O O O . O . . X . . . |
$$ | . . W . . . X . X . . O . . O X . . . |
$$ | . . O X . . . X O O O . O . O X X X . |
$$ | . O X O O . . O O X . X . . . O X O X |
$$ | . O X X . O O O X X X X O O O O X O . |
$$ | . X B . X O X X O X . X O X O X O O . |
$$ | . X X X O O X O O O X X O X X X O . . |
$$ | . . . . . . X X O . O X X X X O O O . |
$$ | . . . . . . . X X O . . . O . X . . . |
$$ ---------------------------------------
[go]$$Bc Easier corner
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . X . O O . O . . . . . . . . . . . . |
$$ | . . O , X X O . . , . . . . . X . . . |
$$ | . X X X . X X O . . . . . . . . . . . |
$$ | . . . O O . X X . . . . . . . . X . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . O . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . O , . . . . . , . O . . . X . . . |
$$ | . . . . . . . X O O O . O . . X . . . |
$$ | . . W . . . X . X . . O . . O X . . . |
$$ | . . O X . . . X O O O . O . O X X X . |
$$ | . O X O O . . O O X . X . . . O X O X |
$$ | . O X X . O O O X X X X O O O O X O . |
$$ | . X B . X O X X O X . X O X O X O O . |
$$ | . X X X O O X O O O X X O X X X O . . |
$$ | . . . . . . X X O . O X X X X O O O . |
$$ | . . . . . . . X X O . . . O . X . . . |
$$ ---------------------------------------[/go]
It still wants to f17 push and still doesn't consider e5 life, but things have changed a bit: it only gives f17 45% win (and falling at 1k playouts) because now it does expect in the principal variation white to kill the corner instead of answering the push, but like before expects white to spend 2 gotes to kill it when only 1 is needed so black gets 3 moves (instead of the 2 he deserves) on the top side. However, if you then play it out then after f17, kill corner, g18 h18 f18 (actually f18 can cut) then it realises white doesn't need another move to kill the corner and can play m17 for 83%.
P.S. black h18 as a followup to f17 instead of normal f18 or g18 is interesting.
Black to play, I didn't mark white's last move to avoid that influencing you:
$$Bc
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O O . O . . . . . . . . . . . . |
$$ | . . . , X X O . . , . . . . . X . . . |
$$ | . . X . . . X O . . . . . . . . . . . |
$$ | . . . . . . X . . . . . . . . . X . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . O , . . . . . , . O . . . X . . . |
$$ | . . . . . . . X O O O . O . . X . . . |
$$ | . . . . . . X O . . . O . . O X . . . |
$$ | . . . . . . . X O O O . O . O X X X . |
$$ | . . X . . . . O . X . X . . . O X O X |
$$ | . . . . . . . . X O . X O O O O X O . |
$$ | . X O O . . . . . X . X O X O X O O . |
$$ | . . X X . O . . X O . X O X X X O . . |
$$ | . . . . . . . . . . O X X X X O O O . |
$$ | . . . . . . . . . . . . . O . X . . . |
$$ ---------------------------------------
[go]$$Bc
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O O . O . . . . . . . . . . . . |
$$ | . . . , X X O . . , . . . . . X . . . |
$$ | . . X . . . X O . . . . . . . . . . . |
$$ | . . . . . . X . . . . . . . . . X . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . O , . . . . . , . O . . . X . . . |
$$ | . . . . . . . X O O O . O . . X . . . |
$$ | . . . . . . X O . . . O . . O X . . . |
$$ | . . . . . . . X O O O . O . O X X X . |
$$ | . . X . . . . O . X . X . . . O X O X |
$$ | . . . . . . . . X O . X O O O O X O . |
$$ | . X O O . . . . . X . X O X O X O O . |
$$ | . . X X . O . . X O . X O X X X O . . |
$$ | . . . . . . . . . . O X X X X O O O . |
$$ | . . . . . . . . . . . . . O . X . . . |
$$ ---------------------------------------[/go]
The strength of current top bots lies in global evaluation and search. The strength of humans lies in local evaluation and search, and in global depth first search of one lane roads, such as ladders.
I'm unclear as to whether global and local here apply just to 'evaluation' or to both 'evaluation and search'.
I'm not sure how it was meant, but fairly sure it applies to both. In fact I would stress global search more than global evaluation (bot evaluation itself is not necessarily better than pros, even Alphago was top amateur only without search). But global evaluation AND (more or less / reasonably) correct global search is something humans don't have an answer for.
The reason I raise it is that I'm also unclear how Elf manages to see the 'shocking' R11 narabi earlier on. It doesn't seem like the sort of move it would see very often even with millions of self-training games, but even if it did have this as a globally good candidate move on the basis of self training, it would still need to be good at local search to justify this move (and maybe good enough to find a long ladder?). Or have (a) I completely misunderstood how Elf finds it moves or (b) wrongly assumed it has chosen the narabi on the basis of a search whereas it was chosen solely on the grounds of global evaluation?
Usually it only searches in moves chosen by global evaluation / policy (which is why it still has blind spots). But IMO it doesn't justify candidates - it refutes them. Moves that drop winrate are phased out, what's left is played. It's much harder to phase in new candidates as that takes much more search effort.
This particular case may also have to do with generalizability: some things generalize better, so easier to learn even from different positions, others may need more specialized (so more rare) occurrences.
(bot evaluation itself is not necessarily better than pros, even Alphago was top amateur only without search)
Be carefull, it's "Alphago without reading" vs "top amateur with reading"
The "problem" is that humans will automatically read (and ponder on their opponent time), so it's impossible to really have "humans without reading" as a comparaison.
(bot evaluation itself is not necessarily better than pros, even Alphago was top amateur only without search)
Be carefull, it's "Alphago without reading" vs "top amateur with reading"
The "problem" is that humans will automatically read (and ponder on their opponent time), so it's impossible to really have "humans without reading" as a comparaison.
NNs will also do some kind of reading (without search), and I think raw AG net strength is similar enough to "top pro without reading" to say that the gap is larger on the global search side at least.
(bot evaluation itself is not necessarily better than pros, even Alphago was top amateur only without search)
Be carefull, it's "Alphago without reading" vs "top amateur with reading"
Indeed.
The "problem" is that humans will automatically read (and ponder on their opponent time), so it's impossible to really have "humans without reading" as a comparaison.
Not in that comparison. However, as I recall, De Groot managed long ago to study human first impressions at chess. In his research pros usually not only saw the best move, they saw a main line of play. That is not what neural networks can do, but such seeing is not the same as the conscious process called reading, either.
The Adkins Principle: At some point, doesn't thinking have to go on?
— Winona Adkins
I've failed to keep up with which engine is doing what, but I thought it was pretty settled that both Leela Zero and Elf could reliably beat professionals. Is that not the case?
hyperpape wrote:I've failed to keep up with which engine is doing what, but I thought it was pretty settled that both Leela Zero and Elf could reliably beat professionals. Is that not the case?
Yes (on good hw). The question here was if most of bot advantage comes from it's global evaluation (NN strength) or it's global search.
Hence the comparison hack: top raw net without search ~= top pro without reading ~= top ama with reading.
Bill Spight wrote:Not in that comparison. However, as I recall, De Groot managed long ago to study human first impressions at chess. In his research pros usually not only saw the best move, they saw a main line of play. That is not what neural networks can do, but such seeing is not the same as the conscious process called reading, either.
You're right, I didn't elaborate, but I just wanted to add a caveat to the original comparaison (it is still an usefull comparaison if you're aware of its shortcomings).
And personally, I have "automatic subconscious reading", by that I mean that, the more I look at the board, the more I see/feel sequences, even if I don't do any conscious reading.
@moha Thanks. That's what I thought, but Uberdude was cautious enough to make me doubt it:
Of course it's possible the pros had some good response that Elf didn't see (particularly involving ladders), but when it comes to judgement of positions, particularly quiet ones or even fighting without low liberties I tend to think Elf is better than the pros.
I have no doubt that with good hardware both LZ and LZ-Elf will spank even top pros. On WBaduk I've seen them playing some Japanese pros (mostly young so not quite top except Shibano) and won every game, often before move 100. Even on my PC I expect it's stronger overall, but on certain positions Elf can struggle and have blindspots, mess up ladders and life and deaths as we see. So if Elf says a pro is making a mistake probably 9+ times out of 10 it's right but occasionally Elf is stupid too.
despite not looking like it does, and the ladder works