Life In 19x19

Another 3d game (picked from browsing kgs game archives until I found 3d vs 3d that didn't end by time, didn't look at the game beforehand):
This one is rather interesting as there was an early ko fight and exchange (though white could have lived without ko at c1!). Should we say the opening ended at move 26, or after the exchange to 46. The latter is near the 50 mark, and might actually give the first game for these statistics. Judging this game takes more time than I have now. Will edit later...

Edit: Lower left black has 20 solid points, lower right shimari let's call 12. r7 area is not points, and top right black group could be attacked by a white invasion at r13/r11 so can't really give black many points there, but equally a white invasion is weak too. Perhaps say of the r11 and r7 area black will get one so lets give him another 8 points on that side somewhere. So black has 40 points territory plus the outside influence of that lower left (and sente). How to value that wall? Hard to say. g4 is not dead stone and could be sacrificed, but as is helps k4/n3 a bit from severe attack so perhaps say black has an extra 5 points in the j2 area for that. d10 can also be isolated by black's 47 so some value in that influence direction too, but also black's invader is weak too. Maybe another 5 points? So let's say black has 50.

White's lower side group is sandwiched between two thick black groups so won't make much, let's say 5 points. White's top right is 16 solid to O row. k18 to n18 area is also likely to be points, about 10? How to value the 3 hoshi stones in top left? Given the game continuation a reasonable estimate would be the top side becomes points, d10 becomes dame. Top side is about a 3x10 block so 30. So white's total is 5+16+10+30+6.5 = 67.5. So white leads by about 17 points +/- 10 or so, but black sente. So I say this game counts as a white lead of about 10 or so. And white won it. So adding this to my tally of results in the 1st post.

Do people agree with my judgement? Maybe I was too generous to white at the top left, and too stingy to black on the right? My gut feeling was white boobed by allowing the ko, but also black made a mistake to ignore the threat as white's compensation was larger.

ez4u wrote:Consider how many games we will need in order to test the hypothesis if we are talking about an advantage of 60%-40%. Add in the fact that there is no agreement on when the opening ends or what the advantage is. This shapes up to be a rather long thread!

Well, Often suggested 10,000 games! I reckon we might need 500 KGS 3d games to get 100 with an opening imbalance of >=10 points (which is a relaxation of my 10-15 points in the bet so should increase the win rate). And even then if the hypothesis is a 10+ lead results in a 0.6 win chance rather than 0.5, then the chance of >50 successes in 100 trials of that binomial distribution is 97.3%, versus 46% for 0.5 win chance. And >55 is 86% for 0.6 win chance, versus 13.6% for 0.5 win. And >60 is 46% for 0.6, 1.7% for 0.5.

emeraldemon wrote:How about my game? Who was ahead in the opening?

I had made the two categories of KGS 5k and KGS 3d, with my predicted win rates of 55-60% for a 10 point lead for the 5ks and 65% for the 3ds. If we relax the kyu category to KGS 5k to 1k then I'd expect the win rate for 10+ point leads to increase as 1ks are quite a bit stronger than 5ks, but it's more inclusive. As for your game I agree white is leading at 46, but gut feeling is not sure if 10+? So some counting:

White top left 10, lower side 5, komi 6, right side moyo = ???. Open at s6. Let's say white gets 2 of the 3 areas of corner, side above r10, side below r10 so about 30 points. 51 total.
Black lower left 20, lower right n3 might work so 18, top left only 2 and weak, but also white's group weak so about cancel, top middle 7. 47 total. So white leads, but not enough to count in this experiment. Or was 30 for the white moyo too stingy? Black has basically no potential to make more points other than 2 point invasion groups and l5 to make an extra 10 or so by patching up n3 or e2/h2 to grow a little there and hassle white a bit, whereas the 30 I gave for white's moyo had quite a large error on it. If black goes for reduction and can get p10 for q12 and r17 for r16 then p18 white is reduced to about 35 points on the side. But maybe a deeper invasion can succeed. So if that 30 was 35 then white leads by about 10, and if 40 then more than 10. Do people think white is leading this game if no komi? So I'm not sure whether to count this game in the >=10 lead category.

Ok, here is my experimental design:

1) Get 10 (?) volunteers. Each submits an sgf with the result scrubbed out (and maybe player name/rank)
2) Each of the volunteers goes through each sgf ONLY up to move 50. (This may not be the end of the opening, but we have to pick a line.)
3) Each volunteer predicts Black or white to win.

We now have 10 blind predictions on 10 games = 100 predictions. The null hypothesis is: the opening doesn't matter, so the predictions will be approx. 50% correct. If the predictions are significantly (80%? Who knows) better than random, it would provide support for the "opening matters" hypothesis.

Any volunteers?

emeraldemon wrote:Ok, here is my experimental design:

1) Get 10 (?) volunteers. Each submits an sgf with the result scrubbed out (and maybe player name/rank)
2) Each of the volunteers goes through each sgf ONLY up to move 50. (This may not be the end of the opening, but we have to pick a line.)
3) Each volunteer predicts Black or white to win.

We now have 10 blind predictions on 10 games = 100 predictions. The null hypothesis is: the opening doesn't matter, so the predictions will be approx. 50% correct. If the predictions are significantly (80%? Who knows) better than random, it would provide support for the "opening matters" hypothesis.

Any volunteers?

I volunteer. But I think volunteers should submit the games only to move 50 to avoid accidentally seeing later parts of the game.

Also, submitted games should be chosen at random. Participants shouldn't choose a game they think is hard or easy to predict.

Seeing some experiments done would be cool.

Here's my prediction (although perhaps it's too hard to directly test): my guess is that the amount that the advantage/disadvantage coming out of the opening affects the winning rate for a game between two evenly matched players is within a factor of 1.5 of the amount you would estimate as follows:

* Evaluate the position (say, around move 40-50) as accurately as possible determining the number of points ahead one player is over another (for example, by finding out at what value of komi a few different strong players or pros would switch from preferring to play black to preferring to play white, or vice versa)

* Convert that into ranks at the rate of 12 points = 1 rank.

* Take the number of ranks and estimate the winning rate using the chart at http://gemma.ujf.cas.cz/~cieply/GO/statev.html, doing a little bit of regression to locally estimate the slope depending on the strength of players.

Eyeballing the statistics, I would guess a 10 point lead is about +5% (55%) for 5k-10k, +9% (59%) for 1k, +14% (64%) around 3d, and +21% (71%) around 5d. With a tolerance of about * or / 1.5.

emeraldemon wrote:Ok, here is my experimental design:

1) Get 10 (?) volunteers. Each submits an sgf with the result scrubbed out (and maybe player name/rank)
2) Each of the volunteers goes through each sgf ONLY up to move 50. (This may not be the end of the opening, but we have to pick a line.)
3) Each volunteer predicts Black or white to win.

We now have 10 blind predictions on 10 games = 100 predictions. The null hypothesis is: the opening doesn't matter, so the predictions will be approx. 50% correct. If the predictions are significantly (80%? Who knows) better than random, it would provide support for the "opening matters" hypothesis.

Any volunteers?

My only difficulty with this is that accuracy of predictions also has to do with go strength. The predictions of a weak player, one whose counting is poor and who does not know how to evaluate thickness, can hardly be better than random, so it's basically only interesting what the strong players predict.

daal wrote:

emeraldemon wrote:Ok, here is my experimental design:

1) Get 10 (?) volunteers. Each submits an sgf with the result scrubbed out (and maybe player name/rank)
2) Each of the volunteers goes through each sgf ONLY up to move 50. (This may not be the end of the opening, but we have to pick a line.)
3) Each volunteer predicts Black or white to win.

We now have 10 blind predictions on 10 games = 100 predictions. The null hypothesis is: the opening doesn't matter, so the predictions will be approx. 50% correct. If the predictions are significantly (80%? Who knows) better than random, it would provide support for the "opening matters" hypothesis.

Any volunteers?

My only difficulty with this is that accuracy of predictions also has to do with go strength. The predictions of a weak player, one whose counting is poor and who does not know how to evaluate thickness, can hardly be better than random, so it's basically only interesting what the strong players predict.

I kind of agree, but on the other hand, I think it's interesting to evaluate the results of people having strength similar to those involved in this "opening vs middle game" debate.

emaraldemon, your experiment could let us see how consistent the volunteers are at judging positions after the opening, but it's too small to test my/often's hypothesis of 'does an opening lead increase the win rate?'. The problem is you have only ten games. Multiple predictions on the same game are not new independent data points. Also when picking games at random the majority will probably not have a significant opening lead (10+ points) so your ten games and one hundred predictions would only give perhaps 3 data points which is far too few to test my hypothesis that a modest opening lead increases the win rate from 50% to 55% - 65% for the 5k - 3d range. Hence my idea of many people submitting games with their predictions and the forum can do peer review to help ensure those opening counts are reasonable. But that doesn't seem popular.

Also I don't see why you need to cut the game to the first 50 moves. It makes a load of effort for participants as you can no longer just directly link to a game in the KGS archive as I have, but need to edit the sgf.

P.S. lightvector's predictions are remarkable close to mine, it's almost as though we used the same source

To me, it is vital that the predictor doesn't know the outcome of the games. No matter how objective or unbaised you try to be, if you know the outcome is W+15 that will change how you evaluate the opening. Humans can't ignore information they've been given, so we need blind tests.

I agree, the 10 predictors don't generate 100 data points, they generate 10 data points with hopefully a strong predictor: the 10 independent evaluations should do a better job at predicting the outcome than 1 strong player evaluating. (Haven't we played that malkovich?). Of course data 10 points isn't enough, but a small experiment is better than no experiment at all. I would happily scrub the results from 25 or 50 KGS games if 10 people would commit to evaluate all of them. And 10 was an arbitrary number I chose (human bias again), a panel of 5 might also be strong enough. We could of course see how each individual judge performs too, just for fun

emeraldemon wrote:I agree, the 10 predictors don't generate 100 data points, they generate 10 data points with hopefully a strong predictor: the 10 independent evaluations should do a better job at predicting the outcome than 1 strong player evaluating. (Haven't we played that malkovich?). Of course data 10 points isn't enough, but a small experiment is better than no experiment at all. I would happily scrub the results from 25 or 50 KGS games if 10 people would commit to evaluate all of them. And 10 was an arbitrary number I chose (human bias again), a panel of 5 might also be strong enough. We could of course see how each individual judge performs too, just for fun

It seems to me you are overly focused on getting a strong predictor versus a large enough sample: to tease out my hypothesised 55% win rate for 5ks requires a large number of games to get any statistical significance, so rather than 10 people analysing the same 10 games, I'd have thought it better than 10 people analyse 10 different games. All that needs to be done is group them into a discard pile of "basically even" versus "one player leads 10+". By including the game forum members can jump in with corrections (people are probably more likely to participate when they can say someone else is wrong ). A nice group counting exercise for sure. Useful to disprove often's "opening leads don't help you win" hypothesis? No.

emeraldemon wrote:To me, it is vital that the predictor doesn't know the outcome of the games. No matter how objective or unbaised you try to be, if you know the outcome is W+15 that will change how you evaluate the opening. Humans can't ignore information they've been given, so we need blind tests.

You can always just not scroll the eidogo player all the way down and just click through the opening if you don't want to see the result. Plus if you do a proper count like I did I don't see that being influenced much by knowing who won. I agree blind is better, but not much given the coarseness of categorization needed. IF you want to link a load of trimmed games be my guest and I'll use those instead, but I doubt we will get enough people to commit to this.

In the meantime, another 3d game.
Hard to judge this one and say when the opening ended. One could say at move 18, in which case evenish to then. But I decided to say the opening ended at move 62. Even though I peeped and saw black won this game I have no gut idea who is leading. There's a lot of thickness so probably the error bars on my count will be large. So counting: white top left 13, komi 6, left 10, lower left to j 10 points given b1 tesuji, m area another 5, upper right still open but if white first is 15, if black first maybe 5 each so let's say 7.5 white, right side not solid, perhaps s8 area is 10 and give white half of r12 so another 5. No influence really. White total about 67. Black much harder to count, has a lot of thickness. Top side isn't points yet and an invasion might try to attack h16 but with that thickness to support white in more danger from a running fight. So I'll give black 20 there. Lower right is hard to count but if we say black does p3 and q5 in sente then 10. Now the hard part is valuing the wall. f5 is not actually captured so it's not super thick. In the h6 area maybe 12 points of territory could be expected. f12 area (plus some help from f16) is maybe 10. So then how much in the centre? If black uses his sente to get p12 for r12 (gives white 6ish more) then o10 he sketches out another 6x6 box or so from before plus bits mroe round the edges. The lower right corner is a little weak which could hamper moyo ambitions, but not much and maybe he can settle that in sente first. Is that +30 points more for black or can white live inside? Maybe give black half, so another 15 for his centre potential, but error bars are over 10 for this. So total black is also 67! (I didn't fiddle this, honest!) So this game is basically even, with large uncertainty. Probably I would bet on black to win as he has more potential and chances to make more territory, but Cho Chikun would probably prefer white. I wonder what CrazyStone's count is.

So updating this as another discarded game, current talies, 3 discards, 1 leader win, 0 leader lose.

Uberdude wrote: Useful to disprove often's "opening leads don't help you win" hypothesis? No.

I don't think an emperical study is necessary to show that the claim that "opening leads don't help you win" is bogus.

Herman already pointed out a great illustration of why this is the case through komi. Komi is a numerical quantification of black's lead in the opening, in playing the first move. For any game result, where black or white leads by X, changing the value of komi can change the result. Therefore, an opening lead that's large enough is enough to change the result of the game.

My suspicion is that often's intent was mostly to point out his view that the middle game typically has a greater impact on the game result than the opening, which I think many fewer people would disagree with.

But let's all step back a moment and ask ourselves, "Who cares?". Why not try to improve all aspects of your game and play well each step of the way?