Page 2 of 3

Re: Kaya Open-sources the rating system & updates

Posted: Tue Sep 27, 2011 5:06 am
by Kaya.gs
yoyoma wrote:I have Glicko working. It is rough, for example I'm not sure how I should be doing my own local constants. Also I need the player class to have some more attributes, such as "RD" the rating deviation, and also a DateTime of the last game a player was in. I just added them to the system.rb file, maybe I should have added them from my own glicko.rb file.

Also I am having some trouble with git. I'm running Windows, and it's having trouble with the file ".rvmrc.09.25.2011-00:43:51". I would bet it's a compatibility issue with the filename being strange... Maybe you could just get rid of it if it's not necessary to help me workaround that? It looks like some sort of unneeded backup.



Its fine i shouldnt have uploaded that fine either.

I merged your code to the branch "glicko" on the official repository. I will be cleaning it up a bit and mainly adding some tests. I run the code and it made an output with negative ranks, and rating "0". So i will add some automated tests and verifications to have some sanity checks.

Maybe i should ask wms for a db-dump of some sort.

BTW, I contacted Remi and he sent me his WHR implementation on C/C++. He wasnt interested in doing a ruby version himself, but i can forward it to you if you want to compare it to your python impl.

Also, you are AWESOME. :).

Re: Kaya Open-sources the rating system & updates

Posted: Tue Sep 27, 2011 3:00 pm
by uPWarrior
I've searched this in a few places but I haven't found one that covers this with enough depth.
Also, looking at the rules didn't quite help as this are situations which give the same point difference to both players. The question is: what's the score in this games?

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . X X O O |
$$ | X X X O O |
$$ | X X X O O |
$$ | X X X O O |
$$ | X X . O . |
$$ -------------[/go]

Obvious seki. 0 points for each.

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . X X O O |
$$ | . X X O O |
$$ | X X X O O |
$$ | X X X O . |
$$ | X X . O . |
$$ -------------[/go]

Obvious seki. 0 points for each

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . X X O O |
$$ | . X X O O |
$$ | . X X O . |
$$ | X X X O . |
$$ | X X . O . |
$$ -------------[/go]

No idea. Each player is able to either create a seki or to make life, but this certainly seems an endgame position.

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

Re: Kaya Open-sources the rating system & updates

Posted: Tue Sep 27, 2011 3:04 pm
by Jordus
Diagram 2 and 4 are not seki..

*edit* nvm... :oops:

Re: Kaya Open-sources the rating system & updates

Posted: Tue Sep 27, 2011 4:30 pm
by gurujeet
Under AGA rules and in Japanese scoring, points fully enclosed by a single color within a seki are scored.

Re: Kaya Open-sources the rating system & updates

Posted: Tue Sep 27, 2011 4:38 pm
by xed_over
gurujeet wrote:Under AGA rules and in Japanese scoring, points fully enclosed by a single color within a seki are scored.

I thought it was Chinese rules, not Japanese
http://senseis.xmp.net/?Seki#toc10

(so AGA rules are basicly same as Chinese rules)

Re: Kaya Open-sources the rating system & updates

Posted: Tue Sep 27, 2011 4:53 pm
by gurujeet
Thanks, I got that reversed between and Chinese and Japanese scoring.

Re: Kaya Open-sources the rating system & updates

Posted: Wed Sep 28, 2011 12:06 am
by Laman
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

this is actually not a seki, because "strange things happen at the 1-2 point":

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)

Re: Kaya Open-sources the rating system & updates

Posted: Wed Sep 28, 2011 9:30 am
by Ootakamoku
I would gladly provide you with a ranking system, BUT.. never programmed ruby not used GIT. Long time since I did any open source stuff. Ill at very least take a look at GIT however. In the meantime maybe I could draw your attention to this post..

viewtopic.php?f=10&t=4743

Based on the analysis a ranking system could be designed, where players can choose whatever komi and handi they want to, and the system can accurately estimate the probability of each player winning. Based on this probability you can then adjust their ranking. Also based on these probabilities you can deduce how accurate a players rank is by calculating mean square error for his recent games. This number could then be used as a multiplier when changing his rank, and conversely its reverse should be used when adjusting opponents rank based on the game result. And the same in pseudo code.


rank_weight=( 1 - mean_square_error( last_few_games ) ) ** cool_constant

if( won_game ) {
rank+= ( 1 - probability_of_winning ) * ( 1 - rank_weight ) * opponents_rank_weight
} else {
rank-= probability_of_winning * ( 1 - rank_weight ) * opponents_rank_weight
}


That should produce a ranking system where rank changes fast when its uncertain and becomes accurate when players result look consistent.

Re: Kaya Open-sources the rating system & updates

Posted: Wed Sep 28, 2011 3:38 pm
by uPWarrior
Laman wrote:this is actually not a seki, because "strange things happen at the 1-2 point":


That's right, but how should we score it in case both players pass? It's certainly possible, I've just overlooked it.

Re: Kaya Open-sources the rating system & updates

Posted: Wed Sep 28, 2011 3:59 pm
by Solomon
uPWarrior wrote:
Laman wrote:this is actually not a seki, because "strange things happen at the 1-2 point":


That's right, but how should we score it in case both players pass? It's certainly possible, I've just overlooked it.
Black's group would still be considered dead, because White doesn't need to make a move to ensure Black's death.

Re: Kaya Open-sources the rating system & updates

Posted: Wed Sep 28, 2011 4:46 pm
by Jordus
Laman wrote:
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

this is actually not a seki, because "strange things happen at the 1-2 point":

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)


Black only dies if he plays... so if he passes its seki.... Play isnt mandatory here right?

Re: Kaya Open-sources the rating system & updates

Posted: Wed Sep 28, 2011 5:19 pm
by Solomon
Jordus wrote:
Laman wrote:
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

this is actually not a seki, because "strange things happen at the 1-2 point":

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)


Black only dies if he plays... so if he passes its seki.... Play isnt mandatory here right?
It's a semeai; if Black doesn't play first, then he'll only be more liberties behind if he passes. Either way, Black will die.

Re: Kaya Open-sources the rating system & updates

Posted: Thu Sep 29, 2011 8:42 am
by Kaya.gs
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)


You guys are missing the point. The correct answer to the question is 0 points, in all cases.

It doesnt matter what could've been, it matters what the players agree on. If they agree its seki, its 0 points under japanese rules, and the territory is counted as points in chinese rules (+all the stones).


typohh wrote:I would gladly provide you with a ranking system, BUT.. never programmed ruby not used GIT. Long time since I did any open source stuff. Ill at very least take a look at GIT however. In the meantime maybe I could draw your attention to this post..

viewtopic.php?f=10&t=4743

Based on the analysis a ranking system could be designed, where players can choose whatever komi and handi they want to, and the system can accurately estimate the probability of each player winning. Based on this probability you can then adjust their ranking. Also based on these probabilities you can deduce how accurate a players rank is by calculating mean square error for his recent games. This number could then be used as a multiplier when changing his rank, and conversely its reverse should be used when adjusting opponents rank based on the game result. And the same in pseudo code.


rank_weight=( 1 - mean_square_error( last_few_games ) ) ** cool_constant

if( won_game ) {
rank+= ( 1 - probability_of_winning ) * ( 1 - rank_weight ) * opponents_rank_weight
} else {
rank-= probability_of_winning * ( 1 - rank_weight ) * opponents_rank_weight
}


That should produce a ranking system where rank changes fast when its uncertain and becomes accurate when players result look consistent.


I scanned over the post, and it would be great to be able to compare what you did to our current win-by-walk-over champion, Yoyoma's Glicko.

Specially, if you have your dataset in a comfortable format, we would really appreciate it. I got my hands on some chess results, but Go ones should be better as we can calculate with handi/komi.

It is not so difficult to use git for this purpose: you have to Fork the project( you can do that online), work on it, and then make a pull request. I will receive that and see your changes: after i review them, i merge them to the official branch (master).

All that you should be able to do online. Then, from the command line you have to do :

git clone git@github.com:conanbatt/OpenKaya.git
#work on your stuff
git add files_you_changed_or_created
git commit -m "i just made something really cool"
git push

There are several GUI's if you are a windows guy, but i cant recommend you one by experience.


Regarding Ruby, for such mathematically intensive code, you shouldn't have that many issues adapting yourself to it. You can see under rating/strategies/glicko to see how yoyoma implemented his ,its fairly straight forward.

If you need pointers or help, don't hesitate to ask. I'm available to help code-contributors. You can join #kaya.gs channel on FreeNode.

Re: Kaya Open-sources the rating system & updates

Posted: Thu Sep 29, 2011 9:10 am
by yoyoma
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X a O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?

Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.


Here is a brief description from wms on how KGS does seki detection:
http://groups.google.com/group/rec.game ... 8f79?pli=1

Another related link:
http://senseis.xmp.net/?WmssDameSekiChallenge

From the rgg link:
wms rgg post wrote:Since I was only identifying seki for the purposes of scoring, it became much easier. The hard part of identifying seki is spotting dead groups. Cgoban already knows exactly
which stones are alive and dead, because the users have either told the program or else resolved through the dispute mechanism of cgoban which groups are alive. From here, cgoban just fills in the dame (filling fake eyes as needed), then looks at what is left. If a shape has only one eye, and can not make a second, then it is seki. Figuring out whether you can make a second eye is pretty simple, you just need to recognize a few shapes.


If you put the above position into KGS it declares it to be a seki since the square-four shape cannot make 2 eyes. I believe the only eye shapes it will declare to be seki are 1-space, 2-space, and the square-four. All others such as 3-space can make an eye with sente, therefore the assumption is that since the player did not choose to do so it indicates that even if the opponent plays in, there must be a way to win a semeai.

Re: Kaya Open-sources the rating system & updates

Posted: Thu Sep 29, 2011 12:17 pm
by Kaya.gs
yoyoma wrote:
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X a O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?

Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.


Here is a brief description from wms on how KGS does seki detection:
http://groups.google.com/group/rec.game ... 8f79?pli=1

Another related link:
http://senseis.xmp.net/?WmssDameSekiChallenge

From the rgg link:
wms rgg post wrote:Since I was only identifying seki for the purposes of scoring, it became much easier. The hard part of identifying seki is spotting dead groups. Cgoban already knows exactly
which stones are alive and dead, because the users have either told the program or else resolved through the dispute mechanism of cgoban which groups are alive. From here, cgoban just fills in the dame (filling fake eyes as needed), then looks at what is left. If a shape has only one eye, and can not make a second, then it is seki. Figuring out whether you can make a second eye is pretty simple, you just need to recognize a few shapes.


If you put the above position into KGS it declares it to be a seki since the square-four shape cannot make 2 eyes. I believe the only eye shapes it will declare to be seki are 1-space, 2-space, and the square-four. All others such as 3-space can make an eye with sente, therefore the assumption is that since the player did not choose to do so it indicates that even if the opponent plays in, there must be a way to win a semeai.



excellent contribution.I will add the links on the repo for score.