Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo

Javaness2 · **#41**

You can also read some more respectable papers on score estimation, for example, a bit of a daddy

https://webdocs.cs.ualberta.ca/~mmueller/ps/goeval.pdf

pnprog · **#42**

Javaness2 wrote:

You can also read some more respectable papers on score estimation, for example, a bit of a daddy

https://webdocs.cs.ualberta.ca/~mmueller/ps/goeval.pdf

Thanks for the link!

I made a bit of research regarding the bot mentioned inside (Explorer). My guess is that those score evaluation methods are similar to what is been used (and probably improved) into Gnugo and Fuego.

Maybe I should have a better look at Fuego for a score estimation.

roy7 · **#43**

Once upon a time I think the Leela author told me pacci has the best end game scoring.

Javaness2 · **#44**

https://webdocs.cs.ualberta.ca/~mmueller/ps/goeval.pdf is not too bad either

pnprog · **#45**

Javaness2 wrote:

https://webdocs.cs.ualberta.ca/~mmueller/ps/goeval.pdf is not too bad either

Is that the same document as in your former message?

By the way, I have finished implementing my todo list for v0.5
I will test a bit more, then release an update tomorrow.

pnprog · **#46**

Hi everybody!

So here I come with v0.5, the two main features added to v0.4 are:

Possibility to display Leela's sequences move after move using mouse wheel of keyboard up/down keys. It was necessary because Leela often comes up with very long sequences (try reviewing a 9x9 games with 60s thinking time...).
Possibility to focus the analysis in either black or white player

I also added the neural network value. But I am not sure what it really means (at least values on 9x9 seem meaningless).

For next v0.6 I do not plan to add new features (well, I am still opened to suggestions indeed). My target for v0.6 is to completely separated GnuGo and Leela, and then offer an GnuGo review of the game. My reasons for this are:

For me it's important to be able to offer a completely opensource solution
GnuGo moves are ok for 15kuy and weaker players
For those players, on modern multi-threaded computers, GnuGo should be able to perform an analysis very quickly, this is probably more important
Most importantly, this is paving the way to adding support to additional bots later:
- By forcing me to restructure the code in a cleaner way
- Also, by having a generic code that works with minimal GTP only bots (and no useful information available otherwise). It make it possible to support any bot that support at least the GTP undo command, but will provide only one alternative variation per move (and at the expense of very very long analysis time). Currently, Rayon falls in that category, but somebody has started to implement some dedicated GTP commands to help extract more information from Rayon (see the antifermion branch on GitHub). So let's hope for the best.

By the way, I feel bad about something: I noticed I received a "pull request" on my Github project, that was one months ago, and I hadn't noticed until yesterday. In fact, I rarely go on Github, mostly working on my computer, alternating commit and push commands.
Honestly, I have no idea how to handle a pull request, and I missed the opportunity to learn.
This pull request was about making GoReviewPartner working with Leela 0.9, something I did by myself later, unknowingly about this pull request. As far as I understand, this is an anonymous pull request ("guest093843"), so in case this is a L19 member, I would like to apologized to that contributor if I appeared rude by ignoring him.

Download for windows at the usual place, code source on Github.

roy7 · **#47**

The neural networks are only used in 19x19 and should be ignored for other board sizes.

Leela is far stronger in 19x19.

roy7 · **#48**

BTW there's another open source bot out there called Matilda:

https://github.com/gonmf/matilda

johnsmith · **#49**

pnprog wrote:

I also added the neural network value. But I am not sure what it really means (at least values on 9x9 seem meaningless).

As far as I know, it's an evaluation that's much more precise than MCTS in terms of who's leading at certain point of the game... and that's actually what Deepmind team is really proud of. It happens quite often that Monte Carlo is around 50% and NN value (Net Win %) is above 75% which is basically game over (almost sure win) :bow:

Edit: May I notice that on the right screen "Neural network value" is actually Net Prob %, which is probability of a professional player playing that certain move.

goame · **#50**

pnprog wrote:

Hi!
I finally had a quick try with Ray (rn-3.11.1) under Windows.

As it stands now, Ray seems to not offer that much useful information, see by yourself the result of GTP command "genmove black":

Attachment:

ray_stderr.png

I think the interesting part is the "Best Sequence" that should shows what Ray expects to be the following moves and the different winning percentages (I do not know the meaning).

For comparison, Leela always propose a several sequences with winning percentage, that's much more information useful to game review.

You can tell them here https://github.com/zakki/Ray/issues what you need :tmbup:

Or just ask https://sjeng.org/ gcp at sjeng dot org

goame · **#51**

goame wrote:

-show white and black moyo.
-show white and black territory.

I do not know any method to do that in fact. The closest I got so far is asking Gnugo to quickly play until the end of the game to find out, but it's slow, and does not really work when there are group with unclear life and death status.

goame wrote:

-click in the right window on variation a, b, c,... and let me see every move with the arrow keys, so it is easier to remember and better to learn.

Yes, I plan to do that soon. I plan to use the mouse wheel instead of the keyboard, but that's the idea. Yeah, maybe I should use both keyboard and mouse wheel, for laptop users with out mouse wheel.

goame wrote:

-use different colors for best, second ... variation.

Currently, the letters indicate the order from best (A) to not so good (B, C, D) and blue/red to differentiate winning rate >50% I do not have much margin to change the colors in fact

goame wrote:

-take only the 6 moves (variations) with the highest playouts, it doesn't make sense the have one variation with 64400 playouts and another mit 47 playouts.
-display variation 2-6 only if they have at least 25% playouts from the first variation with the most playouts.
-an infinite analysis so I can still analyse the game when looking on it after your first analysis is finished.
-also the number of playouts/simulations of the first analysis for each move should be saved. When I analyse the game later and get higher playouts, the data should change automatically.

Those are idea that have more to see with Leela internal algorithm. Not something I can change at my level. It will probably evolve as Leela improve in the near future.

goame wrote:

There is a new version rn-3.11.1 https://github.com/zakki/Ray/releases see also the issues and ask there if you need help;)

Are you a member of Ray developer team????[/quote]

Do you have a clue how to fix the show moyo and territory problem?
Lots of people will be happy to have different colors, because for example:
first variation red, second variation red, = equal
but first variation has 75% chance to win and second has 51% chance to win = not equal.
I'm not a member of Ray developer team.

goame · **#52**

pnprog wrote:

Hi everybody!

For next v0.6 I do not plan to add new features (well, I am still opened to suggestions indeed). My target for v0.6 is to completely separated GnuGo and Leela, and then offer an GnuGo review of the game. My reasons for this are:
[list]
[*]For me it's important to be able to offer a completely opensource solution
[*]GnuGo moves are ok for 15kuy and weaker players
[*]For those players, on modern multi-threaded computers, GnuGo should be able to perform an analysis very quickly, this is probably more important

Wouldn't it be easier to let the users decide which gtp engine they want use? Leela, Rayon, later Crazystone and Zen.
GnuGo is to weak. Every Go beginner is after a month 15 kyu or stronger.
It is important to have a good analysis, fast is second point, because the go players want to learn something. And even more improtant it is for beginners because when they learn also the bad moves from GnuGo they will repeat it and after a year with training on GnuGo blunders, they will need 10 years to fix them.

pnprog · **#53**

roy7 wrote:

BTW there's another open source bot out there called Matilda:

https://github.com/gonmf/matilda

Thanks, I will have a try too.

pnprog · **#54**

goame wrote:

Do you have a clue how to fix the show moyo and territory problem?

Not at all, really, this is not an easy task. The fact that all Go servers score estimators suck just shows that. At least, I am not the one who will come up with a practical solution. But if somebody designs an algorithm that is reasonably accurate, I would be happy to try to implement it. In my own experience, The Many Faces Of Go had a pretty good one for my level.

goame wrote:

Lots of people will be happy to have different colors, because for example:
first variation red, second variation red, = equal
but first variation has 75% chance to win and second has 51% chance to win = not equal.

For the color of the variation, I implemented the proposal from pleiade67:

pleiade67 wrote:

Would be nice to have two different color when going from win (winrate > 50% --> blue) to loss (winrate > 50% --> red) for the current color (black or white).

What would be your proposal exactly?

goame wrote:

Wouldn't it be easier to let the users decide which gtp engine they want use? Leela, Rayon, later Crazystone and Zen.

For sure it would be easier. But currently, all bots are unequal in what they offer that can be used for review purpose:

Most of them propose the basic set of GTP command, but one cannot go that far with only that (GTP was not designed for that).
Then, different bots propose more information but in different ways (private GTP command, stderr channel) so it make it impossible to have a generic solution to extract that information
Some of them do not propose anything, like Zen or Crazy Stone that do not even have a GTP mode (or they do?)

The implication is that I will have to make dedicated codes to support different bots. At least I will propose a generic solution that is GTP only, and should make it possible to get, for each moves of the game, one (and only one) alternative sequence (no winning probability).

goame wrote:

GnuGo is to weak. Every Go beginner is after a month 15 kyu or stronger.

At least I was not

And for some of us, Leela is weak :lol:

If I remebmer well, on KGS, GnuGo 3.8 level is that of a SDK (8k?). I thinks it has a lot to teach to 15kyu and weaker players. But nobody will be forced to use it.

goame wrote:

It is important to have a good analysis, fast is second point, because the go players want to learn something.

For those new players that are still in the "loose 100 games as soon as possible" period, then a fast analysis by GnuGo will be more useful than a long analysis from Leela.

goame wrote:

And even more important it is for beginners because when they learn also the bad moves from GnuGo they will repeat it and after a year with training on GnuGo blunders, they will need 10 years to fix them.

Yes, this is a well known risk.

Dontbtme · **#55**

Hi.
Just passing by to say I tried the v0.5 and think it's awesome! ^^
Thanks a lot for everything.

lobotommy · **#56**

I have a small problem on macOS with Leela and GRP
When I try to run sgf analysis in GRP, after I point the sgf file I get this error:

Code:

Exception in Tkinter callback
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1536, in __call__
    return self.func(*args)
  File "/Users/tomaszp/Downloads/goreviewpartner-0.5/leela_analysis.py", line 694, in start
    self.popup=RunAnalysis(newtop,self.filename,move_selection)
  File "/Users/tomaszp/Downloads/goreviewpartner-0.5/leela_analysis.py", line 94, in __init__
    self.initialize()
  File "/Users/tomaszp/Downloads/goreviewpartner-0.5/leela_analysis.py", line 373, in initialize
    leela=gtp(leela_command_line)
  File "/Users/tomaszp/Downloads/goreviewpartner-0.5/gtp.py", line 8, in __init__
    self.process=subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Homebrew installed, Leela works, and GRP also open, both can be run separately with no error. Any ideas whats the problem here?

pnprog · **#57**

lobotommy wrote:

I have a small problem on macOS with Leela and GRP
When I try to run sgf analysis in GRP, after I point the sgf file I get this error:

Apparently, an issue appears when GoReviewPartner is launching Leela. A few questions to help me identify the issue:

What is the command line set for Leela in setting? The GTP version of Leela has to be used, not the GUI version (it's a separate download)
Did you add the two parameters? --gtp --noponder
Does it work when you copy it and run it directly into a terminal?

pnprog · **#58**

Hi!

This is just to give an update on GoReviewPartner. I am still working on completely separating analysis for Leela and Gnugo, with the later destined for 15kuy and weaker players. The analysis part by GnuGo is more or less working, it's using threads to speed up the work, although I think it could be sped up even more in the future (as they say: make it work first, then make it work fast)
The main purpose of all of this is to make it possible for me to add support of other bots later (I am having Ray in mind).

I am taking advantage of Gnugo extended set of GTP command to make it possible to display the territory during the game, as estimated by Gnugo. It looks like this:

Attachment:

Screenshot0000.png [ 54.99 KiB | Viewed 22313 times ]

In fact, is used the standard sgf properties [TB] and [TW] to mark the territories during the analysis. When such properties are available, the the "territories" button appears. So if other bots add support for this function, it should be pretty easy to add it into GoReviewParter.

I am not so sure the score estimation of Gnugo is consistent with it's territories estimation, so I might drop the former.

djhbrown · **#59**

after a recent system rebuild, GRP no longer works; presumably a configuration error. i don't have gnugo installed - could that be it?

i vaguely remember this error turning up once before, but have forgotten the fix

using latest edition of GRP from GitHub

Settings:
/home/d/go/clients/leela_080_linux_x64 --gtp --noponder
gnugo-3.8\gnugo.exe --mode gtp

d@d-HP-Pavilion-dv6700-Notebook-PC:~/go/clients/goreviewpartner-master$ python main.py
refreshing
refreshing
/home/d/go/clients/goreviewpartner-master/sgf/na1line.sgf
gamename: /home/d/go/clients/goreviewpartner-master/sgf/na1line
filename: /home/d/go/clients/goreviewpartner-master/sgf/na1line.sgf
========= move selection
[116]
Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1545, in __call__
return self.func(*args)
File "/home/d/go/clients/goreviewpartner-master/leela_analysis.py", line 694, in start
self.popup=RunAnalysis(newtop,self.filename,move_selection)
File "/home/d/go/clients/goreviewpartner-master/leela_analysis.py", line 94, in __init__
self.initialize()
File "/home/d/go/clients/goreviewpartner-master/leela_analysis.py", line 379, in initialize
gnugo=gtp(gnugo_command_line)
File "/home/d/go/clients/goreviewpartner-master/gtp.py", line 8, in __init__
self.process=subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1024, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

pnprog · **#60**

djhbrown wrote:

Settings:
/home/d/go/clients/leela_080_linux_x64 --gtp --noponder
gnugo-3.8\gnugo.exe --mode gtp

Under *ubuntu, first install gnugo:

Code:

sudo apt-get install gnugo

then the setting for gnugo is simply

Code:

gnugo --mode gtp

Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo

Who is online