lightvector wrote:
Just some unorganized notes for anyone who might be interested in buying Zen 7 and looking for info:
By my understanding and impressions of both, Zen 7 is significantly stronger than Leela 0.11 when using equal hardware, but Zen 7 commercial release doesn't support GPU, unlike Leela. So if you have a laptop like mine without any good graphics card anyways, Zen 7 is definitely stronger, but if you have a decent graphics card, Leela might well be as strong or stronger because Zen will be unable to use that hardware.
On CPU, stylewise Zen 7 so far seems on average a little more stable and self-consistent in its evaluations and very steady. Leela is more dynamic and prone to change its mind as you play down a variation, and continues to have occasional blind spots or misevaluations just like the older 0.10 Leela, although they are somewhat less frequent. I don't have any experience with GPU-accelerated Leela, but I would expect with the large speedup of a strong GPU it would continue to have rare large misevals in situations where the search isn't sufficient to correct it, but I could easily see the deeper search making it much sharper and competitive with Zen 7 in fights that it doesn't miseval.
Zen 7 doesn't appear to have a nice "offline" review interface right now, or at least I haven't found it, but I've found it okay for interactive review of games. The interface is a bit lacking through. By contrast, with Leela 0.11 GTP version there are scripts you can find online that will let you do things like have Leela deep-search your game SGF overnight and annotate it with what it thinks your mistakes were, as well as plug them into GUIs that let you navigate and make reviewing easy.
However, with Leela I always have to take its evals with a grain of salt when they get in certain kinds of capturing races or life and death due to its blind spots. From experience I have a sense now of when it's more likely to do this and in those cases will often interactively go to the position and "test" Leela by playing the moves to resolve a situation or that I think might expose a blind spot, before I can trust the eval. I've used Zen less, but I haven't noticed such major problems with Zen.
Also, one last detail - Leela has an interesting behavior when reviewing handicap games. For example, in no-komi games it thinks *white* has the advantage early on, and in low-handicap games it thinks Black has none or only a small advantage. This is presumably because it was trained on human games where in practice, no-komi games often are won by white since they are underhandicapped, etc. But this makes it hard to use Leela to "objectively" evaluate the position sometimes.
And yet I don't think you can always rely on it to give you a reasonable value of things like "how likely is black to win given that he is 2 stones weaker than white" either. Because from what I can tell from what gcp has said, Leela doesn't actually know that for example black is 2 stones weaker than white. It simply infers this from seeing the board position with black having the extra hoshi stone. But as the board fills up and captures happen, it's no longer clear from the board position alone that it was originally a handicap game, so I think Leela in some unknown and hard-to-predict way will gradually transition to no longer evaluating as if the players are different in strength.
Zen seems to report values closer to objective, at least in 2 and 3 stone games unlike Leela it reports a strong advantage for Black at the start. I haven't tried using Zen on a no-komi game yet.
Thanks for this very useful post. Wish I had read it before I bought zen7.
I think what you call "a nice "offline" review interface" was what I had in mind when Zen7 was advertised as having a "review mode". Its a bit disappointing that there is no such option available, when freely available software (eg Leela + GRP) can do so. It is really time consuming to review a game using the analyze function alone. At the very least Zen 7 should be able to highlight the moves that most require scrutiny: there is a similar function when reviewing games played against zen7, where it highlights moves that require review in
red.
As things stand, i am not sure whether to review my games with Zen or Leela+GRP (i dont have GPU). Perhaps the choice will in time be an obvious one, once Leelazero powers up
Login
Register
FAQ
Search
johnsmith
