Experimenting on myself

[emeraldemon]

I've been thinking about the constant churning & disagreement about the fastest methods of improvement: tsumego vs fast games vs slow games vs pro games and so on. I said somewhat jokingly in another thread I'd like to have a set of players who let me assign them to different schedules so I could see who improves fastest. But then I was thinking, why not experiment on myself?

The idea would be to train a very specific regime for a certain amount of time, and see how my skill improves under each. Something like:
month 1: lots of tsumego
month 2: memorize pro games
month 3: play lots of fast games
month 4: play only slow games with review
Repeat

So my question to the forum-members: If you were designing such an experiment, how would you do it?
1) What types of training are worth comparing?
2) Is 1 month a good time period?
3) Is it worth trying cross-training (e.g. tsumego+pro games), or will it confuse results?

[Marcus]

The problem I see with such an experiment is that you cannot isolate the effect of the previous month's studying easy from the next month's studying. There is a natural period of time where your studies sink in. How are you going to evaluate your improvement? How, during periods of non-game-playing study, are you going to apply your studies? I've always been of the opinion that studying isn't very useful unless you try to apply it to your games, and gain the experiences to go with the theories ...

[daal]

Just try month 3.

[Joaz Banbeck]

It is an interesting idea, but there is no way that you will get statistically valid results with data sets as small as 1 person or 1 month. ( I think that the minimum is a person and a half in a month and half )

[palapiku]

If a person and a half can improve a stone and a half in a month and a half...

[Joaz Banbeck]

With a person and a half, do pair go rules apply?

[robinz]

I think Marcus has put his finger on the major problem with this as an experiment. If you find you suddenly improve by a stone in month 4, say, will that be due solely to the study regime of month 4, or the previous month's or months' (one of the rare cases where the positioning of the apostrophe makes a clear difference in meaning ) study having finally managed to sink in.

I think, if you really do want to run an experiment, you'd need to try each method on a separate student, or ideally, a largeish group of students. Hey, I might even volunteer to be a guinea pig myself, if you get enough volunteers to run it - although I do enough independent reading on this site and a few others (I doubt I'd be able to keep myself away from goproblems.com, for example) that it'd be hard to properly isolate the various factors...

[emeraldemon]

Thanks for the constructive criticism, you've all made good points.

For evaluation, I think I need a consistent test that occurs for every period, maybe something like 5 serious (slow) KGS games. For the question of overlap, I can see two possible solutions:

Solution 1: Changing order

The experiment would look something like:
month 1: lots of tsumego
month 2: memorize pro games
month 3: play lots of fast games
month 4: play only slow games with review
then
month 5: memorize pro games
month 6: play only slow games
month 7: lots of tsumego
month 8: lots of fast games

The idea is that I can try to control for previous months' effect by rearranging when they occur. In this example, if tsumego is very helpful, but the effect shows up a month later, I would expect the most improvement in months 2 and 8. It might also be valuable to add in a dead month (i.e. no study) for control.

Solution 2: changing time periods
Here the experiment would look something like:
2 weeks tsumego
2 weeks pro games
1 month tsumego
1 month pro games
2 months tsumego
2 months pro games
...
It would be harder to do this with more than two training strategies. If it's necessary to alternate to achieve good learning, I'd expect fast progress in the early stages of the experiment, and slower progress when I'm just doing one training type. On the other hand if tsumego are clearly more helpful than pro games, I would expect 2 months solid tsumego to be the best.

--
There's another problem with using long (multi-year) experiments on a single person: go improvement gets slower as you get better. I'm 2 kyu now, pushing on 1 kyu. Say I try one strategy and move quickly to 1 kyu and then 1 dan (I wish ). Then I try a different strategy, but don't improve to 2 dan. Is it because the strategy was worse, or because it's harder to go 1d->2d than it is to go 1k->1d ? But I may have to cross that bridge when I come to it: there's really no way to avoid it,

--
Of course if I could get volunteers it would be really interesting to have an experiment where multiple students get assigned to each training strategy, and compare results. But you have to be careful, because it introduces another big variable: different players are going to study differently and improve at different rates, even if you assign them the exact same tasks. My guess is even if you could take ten 2-kyu players, and have them all play an identical number of games and do the exact same problems, you would end up with ten different strengths. That said, if we could get some people interested I'm full of ideas to try

[Chew Terr]

When it comes down to it, whether or not the experiment actually proves anything, focusing on trying lots of different methods seems a good way to improve all areas of your game. Obviously, there are benefits to each method that have all been mentioned before. Since you'll be doing enough of each method to really get into it, maybe you'll benefit really well, regardless! Good luck!

[daniel_the_smith]

This won't give you any objective information on which is the best way to improve, for a number of reasons:

1. There's likely to be a delay before the results of study makes it into your game.
2. There's likely to be periods where study actually hurts your game in the short term until you've synthesized your new and old knowledge.
3. The amount of time you're planning on spending on each method may not give enough improvement to be distinguishable from noise in your rating graph.
4. If the first thing you do (seems to) helps you improve, then the next thing you do will appear to be less effective no matter what-- the better you are, the harder it is to improve further.
5. If you have a deficiency in some area, studying that area is likely to help more than studying other areas. So even if you do find something that makes you improve faster than other methods, you won't know if it's really a better way to study or if it's just addressing your weakness. What's working for you today might not work as well tomorrow.

Of course that doesn't mean it's a bad idea, subjective information is better than no information. Maybe. If you find your favorite way to study, that's worth something.

[judicata]

Meh, studies can be criticized to bits, but a data is data. In other words--go for it, the results could be interesting even if not definitive.

[Toge]

Meh, studies can be criticized to bits, but a data is data.

- Unfortunately only scientific data is worth anything for conclusions. If we cannot isolate the phenomenon we're interested in, "data" is not about that phenomenon, but hodgepodge of dozens of related phenomena.

[judicata]

Meh, studies can be criticized to bits, but a data is data.

- Unfortunately only scientific data is worth anything for conclusions. If we cannot isolate the phenomenon we're interested in, "data" is not about that phenomenon, but hodgepodge of dozens of related phenomena.

...so? My point is that it will be interesting anyway. Just take it for what it is.

[emeraldemon]

So I've decided I'm going to go with basically the original plan:

month 1: lots of tsumego
month 2: play only slow games with review
month 3: play lots of fast games
month 4: memorize pro games

I randomly shuffled the order of the months. The last week of each month I will play 5 games on KGS, 25min + 5*0:30 (the "medium" setting on automatch). If I make it through all four months I'll decide what to do then. I'm a few days in, but I'll start with December as tsumego month.

This isn't a valid experiment, in the sense that my conclusions will not be able to disprove any hypothesis. But I still think the data gathered could be useful, if nothing else it may give an idea of what would really be necessary to perform such an experiment. As Chew Terr said, at the very least maybe I'll get a good feel for what the different types of study feel like.

So wish me luck, I'll report in on findings!

[SoDesuNe]

Good luck and have fun.

I'm in the Tsumego fraction! : ) Actually I'd love to see some sort of "theory month". I just finished "Attack and Defense" for the second time and I feel like not being a weak 3-kyu anymore.

But well, I'm very curious to see how this turns out =)