Kaya Open-sources the rating system & updates

General conversations about Go belong here.
Kaya.gs
Lives with ko
Posts: 294
Joined: Fri Aug 12, 2011 10:52 am
Rank: 6d
GD Posts: 0
KGS: Dexmorgan
Wbaduk: c0nanbatt
Has thanked: 25 times
Been thanked: 78 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by Kaya.gs »

yoyoma wrote:I have Glicko working. It is rough, for example I'm not sure how I should be doing my own local constants. Also I need the player class to have some more attributes, such as "RD" the rating deviation, and also a DateTime of the last game a player was in. I just added them to the system.rb file, maybe I should have added them from my own glicko.rb file.

Also I am having some trouble with git. I'm running Windows, and it's having trouble with the file ".rvmrc.09.25.2011-00:43:51". I would bet it's a compatibility issue with the filename being strange... Maybe you could just get rid of it if it's not necessary to help me workaround that? It looks like some sort of unneeded backup.



Its fine i shouldnt have uploaded that fine either.

I merged your code to the branch "glicko" on the official repository. I will be cleaning it up a bit and mainly adding some tests. I run the code and it made an output with negative ranks, and rating "0". So i will add some automated tests and verifications to have some sanity checks.

Maybe i should ask wms for a db-dump of some sort.

BTW, I contacted Remi and he sent me his WHR implementation on C/C++. He wasnt interested in doing a ruby version himself, but i can forward it to you if you want to compare it to your python impl.

Also, you are AWESOME. :).
Founder of Kaya.gs
uPWarrior
Lives with ko
Posts: 199
Joined: Mon Jan 17, 2011 1:59 pm
Rank: KGS 3 kyu
GD Posts: 0
Has thanked: 6 times
Been thanked: 55 times

Re: Kaya Open-sources the rating system & updates

Post by uPWarrior »

I've searched this in a few places but I haven't found one that covers this with enough depth.
Also, looking at the rules didn't quite help as this are situations which give the same point difference to both players. The question is: what's the score in this games?

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . X X O O |
$$ | X X X O O |
$$ | X X X O O |
$$ | X X X O O |
$$ | X X . O . |
$$ -------------[/go]

Obvious seki. 0 points for each.

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . X X O O |
$$ | . X X O O |
$$ | X X X O O |
$$ | X X X O . |
$$ | X X . O . |
$$ -------------[/go]

Obvious seki. 0 points for each

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . X X O O |
$$ | . X X O O |
$$ | . X X O . |
$$ | X X X O . |
$$ | X X . O . |
$$ -------------[/go]

No idea. Each player is able to either create a seki or to make life, but this certainly seems an endgame position.

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.
User avatar
Jordus
Site Admin
Posts: 1125
Joined: Fri Dec 04, 2009 6:06 pm
Rank: KGS 9k
GD Posts: 0
Universal go server handle: Jordus
Location: Allegan, MI, USA
Has thanked: 16 times
Been thanked: 116 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by Jordus »

Diagram 2 and 4 are not seki..

*edit* nvm... :oops:
I'm thinking...
gurujeet
Dies in gote
Posts: 52
Joined: Wed Jul 28, 2010 10:55 am
Rank: AGA 7K
GD Posts: 0
KGS: gurujeet
DGS: gfromdc
Location: Chevy Chase, MD
Been thanked: 8 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by gurujeet »

Under AGA rules and in Japanese scoring, points fully enclosed by a single color within a seki are scored.
xed_over
Oza
Posts: 2264
Joined: Mon Apr 19, 2010 11:51 am
Has thanked: 1179 times
Been thanked: 553 times

Re: Kaya Open-sources the rating system & updates

Post by xed_over »

gurujeet wrote:Under AGA rules and in Japanese scoring, points fully enclosed by a single color within a seki are scored.

I thought it was Chinese rules, not Japanese
http://senseis.xmp.net/?Seki#toc10

(so AGA rules are basicly same as Chinese rules)
gurujeet
Dies in gote
Posts: 52
Joined: Wed Jul 28, 2010 10:55 am
Rank: AGA 7K
GD Posts: 0
KGS: gurujeet
DGS: gfromdc
Location: Chevy Chase, MD
Been thanked: 8 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by gurujeet »

Thanks, I got that reversed between and Chinese and Japanese scoring.
User avatar
Laman
Lives in gote
Posts: 655
Joined: Thu May 06, 2010 10:24 pm
Rank: 1d KGS
GD Posts: 0
KGS: Laman
Location: Czechia
Has thanked: 29 times
Been thanked: 41 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by Laman »

uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

this is actually not a seki, because "strange things happen at the 1-2 point":

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)
Spilling gasoline feels good.

I might be wrong, but probably not.
User avatar
Ootakamoku
Lives with ko
Posts: 126
Joined: Wed Sep 28, 2011 6:44 am
Rank: EGF 2 dan
GD Posts: 0
Universal go server handle: Ootakamoku
Has thanked: 1 time
Been thanked: 63 times

Re: Kaya Open-sources the rating system & updates

Post by Ootakamoku »

I would gladly provide you with a ranking system, BUT.. never programmed ruby not used GIT. Long time since I did any open source stuff. Ill at very least take a look at GIT however. In the meantime maybe I could draw your attention to this post..

viewtopic.php?f=10&t=4743

Based on the analysis a ranking system could be designed, where players can choose whatever komi and handi they want to, and the system can accurately estimate the probability of each player winning. Based on this probability you can then adjust their ranking. Also based on these probabilities you can deduce how accurate a players rank is by calculating mean square error for his recent games. This number could then be used as a multiplier when changing his rank, and conversely its reverse should be used when adjusting opponents rank based on the game result. And the same in pseudo code.


rank_weight=( 1 - mean_square_error( last_few_games ) ) ** cool_constant

if( won_game ) {
rank+= ( 1 - probability_of_winning ) * ( 1 - rank_weight ) * opponents_rank_weight
} else {
rank-= probability_of_winning * ( 1 - rank_weight ) * opponents_rank_weight
}


That should produce a ranking system where rank changes fast when its uncertain and becomes accurate when players result look consistent.
uPWarrior
Lives with ko
Posts: 199
Joined: Mon Jan 17, 2011 1:59 pm
Rank: KGS 3 kyu
GD Posts: 0
Has thanked: 6 times
Been thanked: 55 times

Re: Kaya Open-sources the rating system & updates

Post by uPWarrior »

Laman wrote:this is actually not a seki, because "strange things happen at the 1-2 point":


That's right, but how should we score it in case both players pass? It's certainly possible, I've just overlooked it.
User avatar
Solomon
Gosei
Posts: 1848
Joined: Tue Apr 20, 2010 9:21 pm
Rank: AGA 5d
GD Posts: 0
KGS: Capsule 4d
Tygem: 치킨까스 5d
Location: Bellevue, WA
Has thanked: 90 times
Been thanked: 835 times

Re: Kaya Open-sources the rating system & updates

Post by Solomon »

uPWarrior wrote:
Laman wrote:this is actually not a seki, because "strange things happen at the 1-2 point":


That's right, but how should we score it in case both players pass? It's certainly possible, I've just overlooked it.
Black's group would still be considered dead, because White doesn't need to make a move to ensure Black's death.
User avatar
Jordus
Site Admin
Posts: 1125
Joined: Fri Dec 04, 2009 6:06 pm
Rank: KGS 9k
GD Posts: 0
Universal go server handle: Jordus
Location: Allegan, MI, USA
Has thanked: 16 times
Been thanked: 116 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by Jordus »

Laman wrote:
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

this is actually not a seki, because "strange things happen at the 1-2 point":

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)


Black only dies if he plays... so if he passes its seki.... Play isnt mandatory here right?
I'm thinking...
User avatar
Solomon
Gosei
Posts: 1848
Joined: Tue Apr 20, 2010 9:21 pm
Rank: AGA 5d
GD Posts: 0
KGS: Capsule 4d
Tygem: 치킨까스 5d
Location: Bellevue, WA
Has thanked: 90 times
Been thanked: 835 times

Re: Kaya Open-sources the rating system & updates

Post by Solomon »

Jordus wrote:
Laman wrote:
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

this is actually not a seki, because "strange things happen at the 1-2 point":

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)


Black only dies if he plays... so if he passes its seki.... Play isnt mandatory here right?
It's a semeai; if Black doesn't play first, then he'll only be more liberties behind if he passes. Either way, Black will die.
Kaya.gs
Lives with ko
Posts: 294
Joined: Fri Aug 12, 2011 10:52 am
Rank: 6d
GD Posts: 0
KGS: Dexmorgan
Wbaduk: c0nanbatt
Has thanked: 25 times
Been thanked: 78 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by Kaya.gs »

uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X . O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?


Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.

Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . 2 X 4 O O |
$$ | 3 6 X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O 5 . O |
$$ | X X O 1 . O |
$$ -------------[/go]

(black has to play 3, otherwise white plays it himself)


You guys are missing the point. The correct answer to the question is 0 points, in all cases.

It doesnt matter what could've been, it matters what the players agree on. If they agree its seki, its 0 points under japanese rules, and the territory is counted as points in chinese rules (+all the stones).


typohh wrote:I would gladly provide you with a ranking system, BUT.. never programmed ruby not used GIT. Long time since I did any open source stuff. Ill at very least take a look at GIT however. In the meantime maybe I could draw your attention to this post..

viewtopic.php?f=10&t=4743

Based on the analysis a ranking system could be designed, where players can choose whatever komi and handi they want to, and the system can accurately estimate the probability of each player winning. Based on this probability you can then adjust their ranking. Also based on these probabilities you can deduce how accurate a players rank is by calculating mean square error for his recent games. This number could then be used as a multiplier when changing his rank, and conversely its reverse should be used when adjusting opponents rank based on the game result. And the same in pseudo code.


rank_weight=( 1 - mean_square_error( last_few_games ) ) ** cool_constant

if( won_game ) {
rank+= ( 1 - probability_of_winning ) * ( 1 - rank_weight ) * opponents_rank_weight
} else {
rank-= probability_of_winning * ( 1 - rank_weight ) * opponents_rank_weight
}


That should produce a ranking system where rank changes fast when its uncertain and becomes accurate when players result look consistent.


I scanned over the post, and it would be great to be able to compare what you did to our current win-by-walk-over champion, Yoyoma's Glicko.

Specially, if you have your dataset in a comfortable format, we would really appreciate it. I got my hands on some chess results, but Go ones should be better as we can calculate with handi/komi.

It is not so difficult to use git for this purpose: you have to Fork the project( you can do that online), work on it, and then make a pull request. I will receive that and see your changes: after i review them, i merge them to the official branch (master).

All that you should be able to do online. Then, from the command line you have to do :

git clone git@github.com:conanbatt/OpenKaya.git
#work on your stuff
git add files_you_changed_or_created
git commit -m "i just made something really cool"
git push

There are several GUI's if you are a windows guy, but i cant recommend you one by experience.


Regarding Ruby, for such mathematically intensive code, you shouldn't have that many issues adapting yourself to it. You can see under rating/strategies/glicko to see how yoyoma implemented his ,its fairly straight forward.

If you need pointers or help, don't hesitate to ask. I'm available to help code-contributors. You can join #kaya.gs channel on FreeNode.
Founder of Kaya.gs
yoyoma
Lives in gote
Posts: 653
Joined: Mon Apr 19, 2010 8:45 pm
GD Posts: 0
Location: Austin, Texas, USA
Has thanked: 54 times
Been thanked: 213 times

Re: Kaya Open-sources the rating system & updates

Post by yoyoma »

uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X a O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?

Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.


Here is a brief description from wms on how KGS does seki detection:
http://groups.google.com/group/rec.game ... 8f79?pli=1

Another related link:
http://senseis.xmp.net/?WmssDameSekiChallenge

From the rgg link:
wms rgg post wrote:Since I was only identifying seki for the purposes of scoring, it became much easier. The hard part of identifying seki is spotting dead groups. Cgoban already knows exactly
which stones are alive and dead, because the users have either told the program or else resolved through the dispute mechanism of cgoban which groups are alive. From here, cgoban just fills in the dame (filling fake eyes as needed), then looks at what is left. If a shape has only one eye, and can not make a second, then it is seki. Figuring out whether you can make a second eye is pretty simple, you just need to recognize a few shapes.


If you put the above position into KGS it declares it to be a seki since the square-four shape cannot make 2 eyes. I believe the only eye shapes it will declare to be seki are 1-space, 2-space, and the square-four. All others such as 3-space can make an eye with sente, therefore the assumption is that since the player did not choose to do so it indicates that even if the opponent plays in, there must be a way to win a semeai.
Kaya.gs
Lives with ko
Posts: 294
Joined: Fri Aug 12, 2011 10:52 am
Rank: 6d
GD Posts: 0
KGS: Dexmorgan
Wbaduk: c0nanbatt
Has thanked: 25 times
Been thanked: 78 times
Contact:

Re: Kaya Open-sources the rating system & updates

Post by Kaya.gs »

yoyoma wrote:
uPWarrior wrote:
Click Here To Show Diagram Code
[go]$$B
$$ -------------
$$ | . . X a O O |
$$ | . . X O O O |
$$ | X X X O O O |
$$ | X X O O O O |
$$ | X X O . . O |
$$ | X X O . . O |
$$ -------------[/go]

No idea. Same situation?

Can we just say that if the eye is bigger than or equal to 3 spaces, then it's alive?
These are simple positions, I might post a few more complicated ones.


Here is a brief description from wms on how KGS does seki detection:
http://groups.google.com/group/rec.game ... 8f79?pli=1

Another related link:
http://senseis.xmp.net/?WmssDameSekiChallenge

From the rgg link:
wms rgg post wrote:Since I was only identifying seki for the purposes of scoring, it became much easier. The hard part of identifying seki is spotting dead groups. Cgoban already knows exactly
which stones are alive and dead, because the users have either told the program or else resolved through the dispute mechanism of cgoban which groups are alive. From here, cgoban just fills in the dame (filling fake eyes as needed), then looks at what is left. If a shape has only one eye, and can not make a second, then it is seki. Figuring out whether you can make a second eye is pretty simple, you just need to recognize a few shapes.


If you put the above position into KGS it declares it to be a seki since the square-four shape cannot make 2 eyes. I believe the only eye shapes it will declare to be seki are 1-space, 2-space, and the square-four. All others such as 3-space can make an eye with sente, therefore the assumption is that since the player did not choose to do so it indicates that even if the opponent plays in, there must be a way to win a semeai.



excellent contribution.I will add the links on the repo for score.
Founder of Kaya.gs
Post Reply