Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo

Tell the community about tournaments, new go sites, software updates, etc.
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

Hi!
snippy wrote: - The probability on the left goban is of the current position and not of the move actually played, isn't it? I would find it much more usefull to display the latter, so you can directly compare what you did with what the bot figures to be right (especially if you played a move that the bot has not displayed as a variation).
It's hard to convey the meaning of that probability on the left board. Basically, it's the winning probability before the game actual move, and so it's the probability of Leela best move for that configuration. What I could do is to display something like:
goreviewpartner wrote:Move 2 White to play, in the game, white played q4
Value network black/white win probability before q4: 47.6%/52.4%
Value network black/white win probability after q4: 45.4%/54.6%
Would you find this more meaningful? Or maybe a different wording?
snippy wrote: - Would it be possible to sort the calculated variations (A,B,C,...) by probability? That would simplify things greatly since you could evaluate the possibilities without hovering the mouse over every move.
That is what the first versions of GRP were doing. Until I realized that more often than not, Leela does not select the variation with the higher win rate. Look carefully, you will noticed that the latest variation (by alphabetical order) may have sightly higher win rate, but much lower play-out number. As I understand it, Leela first feeling about those moves are good, but she has not enough time to investigate the follow up moves to be confident enough. And so she discards those possible variations for other variations that are more thoroughly investigated (slightly lower win-rate, but higher confidence). In the end, the alphabetical order truly reflects Leela's best moves order, it's just that Leela does not only consider the value network win rate to make her final decision. So I don't think I should change this.
snippy wrote: - In the same sense, I would much prefer a coloration where (for example) blue meant better than the actual move and red worse then the actual move, instead of >50% and <50%. The latter is just helpful in an even game situation, where the other is helpful in every part of the game. The colotation could even indicate the probabilities like in leela (redder -> higher prob.)
IIRC, this is also the behaviour of the early versions of GRP (at least that's my favorite behaviour), but L19 readers asked me change this behaviour (the full story is in this thread). Recently, a reddit user asked me if it's possible to have only the best move in blue as well. I plan to make the 3 ways possible with a setting in fact.
snippy wrote: - I can't seem to get the text windows with the probs to scroll down. And the textual output is not that easy to read. Perhaps a extra window with the output in table form would be easier and you could even display all the variations inside.
OK for the extra windows (maybe a popup opens automatically on clicking inside the textual area). Those textual area are a hell to deal with resizeable windows anyway, I am not happy with the current result...

For the table, I guess you mean something similar with what is used inside the Leela GUI ? I need to consider this carefully, because other bots do not provide the same type of information, but I need a solution that is consistent across all bots. I plan to somehow standardize all those data into extra SGF properties. I started already in fact and it's the way the graphs are displayed. When this is completed, displaying that table should be as easy as displaying the graphs.
snippy wrote: Again, I really enjoy GRP and find it enourmously helpful and don't want my text to sound like nagging. Just that everything that is good, can still be made better :)
No problem! I guess in a few years, there will be plenty of "GRP-like" software, but at the moment GRP is more like an experimental software paving the way for "next generation" software. So it's both interesting and important to try on all those idea :)
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

djhbrown wrote:when puzzling over history, it would be so helpful if (a) you didn't tell me where i didn't go wrong, and (b) when i do go wrong, you tell me by how much and where i should have gone instead.
You probably have a point here. In fact, with those three things:
  • a simple option labelled "Only keep variations when actual game move and bot move differ" (<- that's where my poor English kicks in)
  • Setting the "maximum number of variations to record during analysis" to 1
  • Modifying the left size comment as I proposed to snippy above
would do the trick I guess
then, you could strip off ALL the other statistics and variations which to my way of thinking are more clutter than information - but since there are plenty of others asking for every bell and whistle under the sun, maybe they could be made user-option choices.
Different users, different use case. Maybe some GRP users use it to improve their bot, not their go skill?
djhbrown wrote:cos it's just a couple of "If" statements
:roll:
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by johnsmith »

pnprog wrote:My current feeling is that I should try to make that left goban more useful. One possibility would be to reuse it for the "open position" mode. Problem is that I like to keep open more than one "opened position" opened at the time. Maybe a tab system, à la Firefox?
That is a good idea for the left board, I like it too.
pnprog wrote:In the end, the alphabetical order truly reflects Leela's best moves order
I agree again. Leela's (and LZ) output is how it should be sorted. That's how the program was made.
User avatar
djhbrown
Lives in gote
Posts: 392
Joined: Tue Sep 15, 2015 5:00 pm
Rank: NR
GD Posts: 0
Has thanked: 23 times
Been thanked: 43 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by djhbrown »

pnprog wrote:a simple option labelled...
Let me try to be constructive: as a card-carrying atheist fundamentalist, i wholeheartedly believe that cleanliness is next to godliness applies to user interface design just as much as it does to Thomas Crapper's innovation (oops, no, it was John Harrington's).

With that in mind, may i suggest something like:

Code: Select all

      File          View                Bot                                Moves
      Open         Simple              Leela-> Options->  Heatmap          Live
      Save          Full                                  Time             Number-> start-end
      Save As      Extras-> Swim:)      L0           ditto                           all  
                            Stats      YAAC1 (= Yet Another Alfie Clone)
                                       YAAC2
                                        ...
                                       YACCn
                                       i can't bring myself to add Gnugo! 
In Live mode:
Esc (or spacebar) interrupts analysis and displays whatever it's got
another spacebar continues analysis
right arrow moves on to next move
up arrow moves to start of previous variation
etc
i can see value in having two windows for side-by-side comparison without having to run 2 copies of GRP
i shrink, therefore i swarm
snippy
Beginner
Posts: 4
Joined: Sat Feb 10, 2018 4:21 am
GD Posts: 0

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by snippy »

pnprog wrote:
goreviewpartner wrote:Move 2 White to play, in the game, white played q4
Value network black/white win probability before q4: 47.6%/52.4%
Value network black/white win probability after q4: 45.4%/54.6%
Would you find this more meaningful? Or maybe a different wording?
That sounds good. The basic idea was just to have a better comparison between the move I played, with the moves the bot would have played. Another possibility might be to output some kind of delta statement in the variations ("+x.y pp on actual move") like you show in the graphs.
pnprog wrote:That is what the first versions of GRP were doing. Until I realized that more often than not, Leela does not select the variation with the higher win rate. Look carefully, you will noticed that the latest variation (by alphabetical order) may have sightly higher win rate, but much lower play-out number. As I understand it, Leela first feeling about those moves are good, but she has not enough time to investigate the follow up moves to be confident enough. And so she discards those possible variations for other variations that are more thoroughly investigated (slightly lower win-rate, but higher confidence). In the end, the alphabetical order truly reflects Leela's best moves order, it's just that Leela does not only consider the value network win rate to make her final decision. So I don't think I should change this.
I did not realize this. Sure, then it's no good to change it.
pnprog wrote:IIRC, this is also the behaviour of the early versions of GRP (at least that's my favorite behaviour), but L19 readers asked me change this behaviour (the full story is in this thread). Recently, a reddit user asked me if it's possible to have only the best move in blue as well. I plan to make the 3 ways possible with a setting in fact.
Great!
pnprog wrote:For the table, I guess you mean something similar with what is used inside the Leela GUI ? I need to consider this carefully, because other bots do not provide the same type of information, but I need a solution that is consistent across all bots. I plan to somehow standardize all those data into extra SGF properties. I started already in fact and it's the way the graphs are displayed. When this is completed, displaying that table should be as easy as displaying the graphs.
Yeah, exactly. I quite like the way leela displays it. But I understand your reservations. In case your idea of standardization does not work out, would it be so bad to have slightly different looking tables for different bots? The output in the textual areas must be changed according to the bot as well, doesn't it? Of course, I only use one bot. It might be annoying for users, that use several bots simultaneously.

One more thing. Could you display the player names in the review mode?

Thanks again!
Last edited by snippy on Sun Feb 11, 2018 10:17 am, edited 1 time in total.
User avatar
Dontbtme
Dies with sente
Posts: 97
Joined: Sun Jan 04, 2015 10:30 am
GD Posts: 0
KGS: 2k
Tygem: 2d
IGS: 1k
Wbaduk: 2d
OGS: 1K
Has thanked: 26 times
Been thanked: 20 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by Dontbtme »

pnprog wrote:What I can do for you instead, it a python script that takes the RSGF file and "strip" the extra variations and reorganize the comment to get the SGF you need. This would not require too much time for me to do. Then if more people find it useful it could become a standard feature of GRP.
That would be fantastic, but if you don't think it's worth doing, please don't. I certainly wouldn't want to make you work for my benefit only ^_^
Bored with skillful play? Watch my blunders on my YouTube Channel
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

Hi!

Here is another update, regarding my former concerns about how to set the bots in different situations.

So I went ahead and implement my idea to have 2 profiles for each bot, one "slow game" and one "fast game".
Then both profiles can be made available in all circumstances:
bot_setting.png
bot_setting.png (55.09 KiB) Viewed 19140 times
Here for example, the selections available for Analysis:
analysis_setting.png
analysis_setting.png (41.62 KiB) Viewed 19140 times
And below for Line Analysis:
live_seting.png
live_seting.png (33.16 KiB) Viewed 19140 times
The same idea will be use if I add more analysis modes in the future (I am considering adding an OGS analysis mode).

So after one year of existence, GRP has support for 5 bots, and we can expect to have support for more bots in the future (the next two I am considering are pachi and minigo) and it might become a bit cluttered inside GRP. So now, one can afford to enable both profiles for all situations, but in the future, one may have to be more selective.

At least, it's important today to be selective for the "open position" mode, because too much bots would require too much memory.

By the way, I just want to mention GRP received it's first formal contribution: a github gentleman send me a patch to improve the aesthetic of the file selector dialogues (when wxPython is available). He also sent me a German translation for GRP :clap:
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
snippy
Beginner
Posts: 4
Joined: Sat Feb 10, 2018 4:21 am
GD Posts: 0

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by snippy »

Hi again :)

Might there still be a bug concerning handicap stones?

I played a high handicap game against leela 0.11 and lost (White wins by -9 + 0.5 Komi + 14 Handicap).

But after analyzing the game with GRP (with leela as bot) the win graph always stays around 100% win for black and for white leela even resigns close before endgame and keeps resigning every move just until the game ended. How can that be?

All the best
Finn
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

snippy wrote:Hi again :)

Might there still be a bug concerning handicap stones?

I played a high handicap game against leela 0.11 and lost (White wins by -9 + 0.5 Komi + 14 Handicap).

But after analyzing the game with GRP (with leela as bot) the win graph always stays around 100% win for black and for white leela even resigns close before endgame and keeps resigning every move just until the game ended. How can that be?

All the best
Finn
This reminds me of this issue: https://github.com/pnprog/goreviewpartner/issues/18
If possible, try with the latest version of GRP (from github) otherwise, please share the SGF with me (through private message if you prefer) so that I can have a try.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.8 (graphs & user manual)

Post by pnprog »

jeromie wrote:
pnprog wrote:
ez4u wrote:I like the work you are doing on this. However, should the 'grain' of your texture be oriented top to bottom rather than left to right?
Now that you mention it, I seem to only find representation of Goban with vertical "grain" (wood fibber), and parallel with the Goban grid.

Is there some sort of aesthetic convention for that?
When sitting at a real go board, it is conventional to align the board so that the grain traces lines between you and your opponent. In Japanese style boards that are not perfectly square, this also maintains the correct orientation of the board.

At a computer, it is natural to think of yourself as playing from the bottom and your opponent from the top, so vertical grain maintains this convention.
By the way, I noticed the AlphaGo teaching tool has the grain "wood fibber" horizontal, not vertical :lol:
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by johnsmith »

I downloaded current (beta) version on github, and there are couple of things I'd like to know so I can use it in Haylee's game today, regarding Live Analysis:

When I chose players in live analysis, what is the difference between:
a) Leela Zero (fast profile)
b) Bot used for analysis: Leela Zero (fast profile)
Third option is "Human", but that's obvious :)

Couldn't find where is the game saved? Edit, solved: "Saving RSGF file 2018-03-04_01:48_Human_vs_Human.rsgf" but the file saved is "2018-03-04_01". Colon needs to be removed from filename.

When I choose live game and choose LZ as white for example, LZ is going to calculate and play its move, and then it's going to analyze it again in the analysis? Maybe time can be saved if the first (previous) calculation was saved. Though it might be a headache to code it :)

As always, thx for magnificent tool ;)
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

Hi !
johnsmith wrote:I downloaded current (beta) version on github, and there are couple of things I'd like to know so I can use it in Haylee's game today, regarding Live Analysis:
Sorry for late reply, I am in the opposite time zone with Haylee, so could not see your message on time. Apparently, AQ did a pretty good job :tmbup:
When I chose players in live analysis, what is the difference between:
a) Leela Zero (fast profile)
b) Bot used for analysis: Leela Zero (fast profile)
Third option is "Human", but that's obvious :)
You need to have at least one instance of Leela Zero to perform the analysis.
But then, if you want to also play against Leela Zero, you can decide to play against that same instance ("Same bot as for the analysis") or open up yet another instance.

The main use case would be to use the slow profile for the analysis, and the fast profile for playing. Having two separate instances of fast profile of the same bot (or two instances of slow profile of the same bot) should not present any interest. Except that you don't have to wait for the analysis of both colours for the game to proceed (for now, there is still no option to analyse only black moves or only white moves).
When I choose live game and choose LZ as white for example, LZ is going to calculate and play its move, and then it's going to analyze it again in the analysis? Maybe time can be saved if the first (previous) calculation was saved. Though it might be a headache to code it :)
So if you choose the same bot for the analysis and for white player, then, when white plays, the analysis is performed at the same time, as you suggest. The "analysis" only consists in asking Leela to play as white and collect the extra data at the same time, it's not harder to code in fact. When Leela is analysing a black move, Leela is also asked to play as black in this position, but the move she generates is discarded (undoed and replace by you move as black).
Couldn't find where is the game saved? Edit, solved: "Saving RSGF file 2018-03-04_01:48_Human_vs_Human.rsgf" but the file saved is "2018-03-04_01". Colon needs to be removed from filename.
Do you means with Windows, file names can't include colon?
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by johnsmith »

I've managed to do live analysis, but unfortunately discovered how to save the game after I had closed the window. So I don't have this game to share. About the colon - yes, I'm on windows and filenames can't include colons it seems. Dashes instead of colons work just great :)
So if you choose the same bot for the analysis and for white player, then, when white plays, the analysis is performed at the same time, as you suggest. The "analysis" only consists in asking Leela to play as white and collect the extra data at the same time, it's not harder to code in fact
It's my bad then, it doesn't calculate twice for the same move. What confused me was that the move played and first suggested move (in the review) were different. Is that expected result?
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

johnsmith wrote:I've managed to do live analysis, but unfortunately discovered how to save the game after I had closed the window. So I don't have this game to share. About the colon - yes, I'm on windows and filenames can't include colons it seems. Dashes instead of colons work just great :)
I plan to add the possibility to select where to save the file, and change the name through a file picker, but had not time for it so far. Then the RSGF file is saved after each analysed move automatically. I am not sure I am saving the SGF file at the moment :mrgreen:
It's my bad then, it doesn't calculate twice for the same move. What confused me was that the move played and first suggested move (in the review) were different. Is that expected result?
No, the first suggested move should be the game move. Where you using Leela or Leela Zero?
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by johnsmith »

the first suggested move should be the game move
Here is the game saved by GRP:
2018-03-04_13-37_Human_vs_Human.rsgf (replaced colon, and changed from rsgf to sgf so I can upload it)

I was using Leela Zero (both as analysis engine and as opponent) and the last W move was played at B, not at A. This is second time I try this position and the result is the same. Args: --gtp --noponder -v 700 -d --timemanage off -w 6615567eaa3adc8ea695682fcfbd7eaa3bb557d3720d2b61610b66006104050e

Right after it played the last move, I immediately opened "start review" but there were only 7 moves analyzed. So I closed it, let the anlysis finish and then opened it again. And just then there were all the moves. So I was confused... it first calc-ed answers to my moves (up to move 12), then went back to move number 7 and calc-ed 7,8,9,10,11,12? Does it lose previous calcs that way? Cuz it may had an idea when played certain move, but had to find it again? Or it just cal-ed 7,9,11 beacuse 8,10,12 were stored somewhere? :)

Edit: I should probably just use human vs human in live analysis (just like last night) and everything is just perfect :)
Attachments
2018-03-04_13-37_Human_vs_Human.sgf
(25.71 KiB) Downloaded 639 times
Post Reply