Spaced Repetition Software for Tsumego

Bill Spight · **#21**

Well, I did a quick online search for spaced repetition. It appears that it is used for things that are memorized, like foreign vocabulary.

But there are several memory systems in the brain, and more than one is relevant to go problems. Skill learning is not the same as memorization.

When I was starting out I did not have much in the way of problem material. It was out there (I was living in Japan), but I did not buy any Japanese go books until I was SDK, and I bought them in department stores, which did not provide much of a selection.

I might have only a couple of dozen tsumego problems that were not too easy (I could solve at a glance) or not too hard (I could spend several minutes without solving). The trouble was that if I reviewed a problem after a week, I had memorized the answer. So I waited at least a month before review, so that I had to work on many of the problems instead of just remembering them. I suppose that that is a kind of spaced repetition, although it was not in the service of memorization.

I do think that memorization is more important for go skill than I gave it credit for back then. Especially for elementary material. Why not memorize nakade shapes, for instance? Why not memorize the empty triangle? Why not memorize the snapback? Why not memorize the basic ladder? the basic net? the basic connection shapes?

When it comes to tsumego problems, however, I think that things get past the elementary fairly quickly.

Click Here To Show Diagram Code: [go]$$ Flash card? $$ | . . . . . . . $$ | . . . . . . . $$ | . X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | . . O O X . . $$ | . . . . . . . $$ --------------[/go]

Experienced players will recognize that Black to play can make ko. I had no qualms about showing it to Boidhre in a review, even though I doubt if he could solve it as a problem. If he sees it as a problem, he will probably recognize it. My bad.

Is it useful in a game to know this position? You bet. Not that it comes up so much, but you may play to avoid it. It is lurking in many games.

Would I make it as a flash card for beginners? No. For SDKs? I don't know.

Click Here To Show Diagram Code: [go]$$ Flash card? $$ | . . . . . . . $$ | . . . . . . . $$ | . X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | . . O O X . . $$ | . . . X . . . $$ --------------[/go]

How about this position? White to play can live. This is a better flash card for beginners, I think. It helps them see the vital point.

Click Here To Show Diagram Code: [go]$$ Flash card? $$ | . . . . . . . $$ | . . . . . . . $$ | . X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | . . O O X . . $$ | . . O X . . . $$ --------------[/go]

I think that this would be a good flash card for beginners.

Live or die.

Click Here To Show Diagram Code: [go]$$ Flash card? $$ | . . . . . . . $$ | . . . . . . . $$ | . X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | . . O O X . . $$ | . O . X . . . $$ --------------[/go]

How about this one? Not a beginners' flash card, I think. And maybe not at all, because it is an uncommon position. Still, it is not a bad tsumego, combining the eye stealing tesuji with shortage of liberties. It arises when White makes a bad play in the previous position two diagrams ago. (And that arose from a mistake by Black in the original position.

) It is something that I have never seen, because I saw the correct play for White right off the bat, years ago. But it is something that I could have learned by trying to solve the problem. Why it is a bad move is harder to understand than why the correct move is right.

You can learn something from problems, even when you do not solve them.

Let's go back to learning the snapback. The prototypical snapback would make a good flash card, IMO. But learning that specific shape does not mean that much, even though it comes up or threatens to come up fairly often. The snapback is more abstract. It is the sacrifice of a single stone to reduce the dame of an opposing string to one, and then capture. (There is a multi-stage snapback, as well.

) The abstract snapback cannot be put on a flash card of a go position, but it is easy to learn from examples. You get spaced repetition of the flashback simply by playing go. It turns up, at least as a threat, in game after game. It also turns up as an element in problem after problem. Do we need to have a spaced repetition schedule for it?

Boidhre · **#22**

Mike Novack wrote:

I am questioning some of the premises in this discussion. I think we need to step back just a bit and discuss those.

What is an "easy" problem? What is a "hard" problem? How do we determine the difference?

There seems to be dissatisfaction with the rather simple approach of going by results. If you keep answering a particular problem correctly then it has become easy for you (and it should be presetnnted to you less frequently) If you continue to make mistakes answering it, then it has remained hard for you (and you should be presented with it more frequently). That is independent of any other a priori considerations of what should make a particular problem hard or easy for you.

Why should "it ought to be easy" or "it ought to be hard" be considered more valid/correct than the experienced results? Were I designing a "flash card" learning system with automated adjustment of the frequency of repetition (and of inserting new material) I would use the actual results and consider any preconceived notions of difficulty irrelevant

Well there's no reason one can't use a system like Anki that way if one chooses to, or to focus on a particular level of difficulty for you. The idea is, say solving a problem is trivial for you, it gets a long gap before being shown next, it is still feels trivial next time it gets a longer gap again and so on so long as it keeps feeling trivial for you. If it starts feeling hard it starts getting shown more often and if you get it wrong even once it resets back to the beginning frequency again since it clearly isn't easy anymore. I agree that there are philosophical issues with whether someone can mark something as easy or hard to solve in general but there is a feeling of a problem feeling trivial or not. If you've overlearned it beyond a certain point so that you can nearly instantly solve it and (in my opinion) it's time to see that problem less often relative to other problems in the set and other problems more often so long as the problem remains this easy for you (assuming of course one is not aiming to do as many very easy problems as possible).

As in, at the start, my approach is simply to mark problems wrong or correct. It's only on my third or more time getting a problem right first time that I might start mark it as easier than that based on the subjective feeling of difficulty. I do the same with language learning.

Again though, as a beginner, I'm loath to argue strongly here as I may be missing some fundamental point (which I'd be glad to have pointed out to me) and my opinions are preliminary at best.

Boidhre · **#23**

Bill Spight wrote:

Let's go back to learning the snapback. The prototypical snapback would make a good flash card, IMO. But learning that specific shape does not mean that much, even though it comes up or threatens to come up fairly often. The snapback is more abstract. It is the sacrifice of a single stone to reduce the dame of an opposing string by one. (There is a multi-stage snapback, as well.

) The abstract snapback cannot be put on a flash card of a go position, but it is easy to learn from examples. You get spaced repetition of the flashback simply by playing go. It turns up, at least as a threat, in game after game. It also turns up as an element in problem after problem. Do we need to have a spaced repetition schedule for it?

Well, the idea of spaced repetition is that it works with the brain's ability to learn something through repetition but is more efficient for memorisation than repeating every element every day or at the same frequency. It allows more material to be memorised more quickly than a simpler system of fixed gap repetition for languages. I don't know whether it'd be as useful in go as I'm very new, thus the thread asking if people had any luck with the method. Oh for a sample study with groups of people doing the same problem set using different methods (of course this wouldn't address the question of whether we do tsumego in order to solve them correctly first time or for some other reason...).

At the moment, I am nearly always doing simple Life and Death problems. I think SRS is very amenable to this since what it comes down to is memorisation of certain shapes. With complicated examples, I've no idea. Sorry, I can't develop my ideas more, minding a very active toddler at the moment.

Bill Spight · **#24**

Boidhre wrote:

Well, the idea of spaced repetition is that it works with the brain's ability to learn something through repetition but is more efficient for memorisation than repeating every element every day or at the same frequency. It allows more material to be memorised more quickly than a simpler system of fixed gap repetition for languages. I don't know whether it'd be as useful in go as I'm very new, thus the thread asking if people had any luck with the method. Oh for a sample study with groups of people doing the same problem set using different methods (of course this wouldn't address the question of whether we do tsumego in order to solve them correctly first time or for some other reason...).

At the moment, I am nearly always doing simple Life and Death problems. I think SRS is very amenable to this since what it comes down to is memorisation of certain shapes. With complicated examples, I've no idea. Sorry, I can't develop my ideas more, minding a very active toddler at the moment.

As I indicated, I used spaced repetition when I started out, not so much to aid memorization as to defeat it.

But I do think that memorization, particularly at the elementary level, is useful.

The results of some search:

This site provides a good review: http://www.gwern.net/Spaced%20repetition

"Skills like gymnastics and music performance raise an important point about the testing effect and spaced repetition: they are for the maintenance of memories or skills, they do not increase it beyond what was already learned."

A very important point.

This site by Dr. Wozniak, a spaced repetition expert, gives "20 rules of formulating knowledge in learning." http://www.supermemo.com/articles/20rules.htm

Rule 1. "Do not learn if you do not understand."

Rule 2. "Learn before you memorize."

IMO, simply making flash cards (or the computerized equivalent) out of go problems is not such a good idea. It does not follow Wozniak's rules. I suspect that making flash cards based upon go problems is a good idea, and that the main value comes from making them. For instance,

Click Here To Show Diagram Code: [go]$$ Dead or alive? $$ | . . . . . . . $$ | . . . . . . . $$ | X X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | X O O O X . . $$ | . O . X . . . $$ --------------[/go]

I think that this would make a good flash card based upon this problem, but devising the card would be an important part of the learning.

Edit: In fact, I would suggest making status flash cards from tsumego problems. I. e., zero move tsumego.

Mike Novack · **#25**

I am getting confused by this discussion.

I thought it began with a question. A question about the existence of non-existence of software that presented a set of problems with the following properties (my interpretation)

a) The problems reapear in different orientatons and colr with a frequency dependent on the history of getting them correct or not.
b) As the number of problems being presented frequently becomes low (many currently in the set have been learned) new problems are introduced.

And the discussion then diverged into whether "learned" should be based upon some subjective opinion or as I suggest be just a matter of the statistics of getting it right or wrong.

I said I thought such software currently exists. Yes of course you can use old fashioned flashcards but a virtual flashcard deck managed by the computer is what I thought was meant.

Boidhre · **#26**

Mike Novack wrote:

I am getting confused by this discussion.

I thought it began with a question. A question about the existence of non-existence of software that presented a set of problems with the following properties (my interpretation)

a) The problems reapear in different orientatons and colr with a frequency dependent on the history of getting them correct or not.
b) As the number of problems being presented frequently becomes low (many currently in the set have been learned) new problems are introduced.

And the discussion then diverged into whether "learned" should be based upon some subjective opinion or as I suggest be just a matter of the statistics of getting it right or wrong.

I said I thought such software currently exists. Yes of course you can use old fashioned flashcards but a virtual flashcard deck managed by the computer is what I thought was meant.

a) is correct, b) is not, new problems are introduced at a fixed rate per day. At least with the implementations that I've seen anyway. I'm sorry if I was unclear about this. Such a system could be used with either traditional flashcards or virtual ones. I prefer virtual for convenience but it's relevant to people who would prefer to do it with books with numbered problems or traditional flashcards.

Yes, the conversation has diverged into a philosophical one/practical one on the best way to learn and/or what the use of tsumego are. I apologise for dragging it off-topic, I find this interesting and relevant but I can see how it might not be to someone.

Bill Spight · **#27**

Mike Novack wrote:

I said I thought such software currently exists. Yes of course you can use old fashioned flashcards but a virtual flashcard deck managed by the computer is what I thought was meant.

Just speaking for myself, I am using the term, flashcard, in a generalized sense, just as I say, capture that stone, instead of saying, click on the display so that the software will refresh the image of the board so that that point will appear as an empty intersection.

Ortho · **#28**

I used spaced repetition for language learning and it worked very well, and have dabbled with it in Go, but I am not convinced it's useful here in the way we're talking about.

In language there is one word for "chair", there are set forms for putting things together, etc. Mostly you just need to remember what they are, but not necessarily how to reason from them.

I agree that Go has some things like this, but mostly you have a set of principles or experiences or pattern recognitions and the challenge is to figure out which ones apply in the current context.

When I SRS Go problems there seem to just simply be too many cues based on the problem itself. I can't actually remember ever saying to myself "oh, this is like that problem I did" in a game and making a better move than I would've done.

I think maybe SRS would be useful in blindly memorizing joseki sequences, for example, but I am not sure that's a good idea anyway.

But I do think there is somewhere that memory plays a huge part, and that is in remembering the principles themselves. I don't usually lose games currently because I didn't know what to do---it's usually that I know the principle but didn't apply it. I can see an advantage in using SRS to remember things that you just have to remember, the mental checklist itself. But I'm not actually sure that the checklist is actually long enough that you need spaced repetition to remember it, especially if you're doing good game reviews after--the fact that you blew it again by not defending that cutting point just naturally turns up right when you have forgotten it, and even Anki couldn't remind you at a better time then when you're still smarting from messing it up again.

Anyway, I just make a list of the problems I fail or have trouble passing and look at them again a week later when I've hopefully forgotten them. I cross problems off the list when I think the problem is boring or when I immediately remember what the answer is.

Boidhre · **#29**

Bill Spight wrote:

As I indicated, I used spaced repetition when I started out, not so much to aid memorization as to defeat it.

But I do think that memorization, particularly at the elementary level, is useful.

The results of some search:

This site provides a good review: http://www.gwern.net/Spaced%20repetition

"Skills like gymnastics and music performance raise an important point about the testing effect and spaced repetition: they are for the maintenance of memories or skills, they do not increase it beyond what was already learned."

A very important point.

Indeed and one I agree with. One should read out the answers initially and only solve instantly if one remembers the reading. Is this any different to building memories of patterns through continuous playing of games?

Do not learn what you don't understand. Doesn't that go against what the received wisdom for beginners is? i.e. don't worry about theory, just play games. I agree that blindly memorising the solution to one move problems and never reading them out would be a very bad idea, but doesn't reading them out before answering get around this issue?

Bill Spight wrote:

This site by Dr. Wozniak, a spaced repetition expert, gives "20 rules of formulating knowledge in learning." http://www.supermemo.com/articles/20rules.htm

Rule 1. "Do not learn if you do not understand."

Rule 2. "Learn before you memorize."

IMO, simply making flash cards (or the computerized equivalent) out of go problems is not such a good idea. It does not follow Wozniak's rules. I suspect that making flash cards based upon go problems is a good idea, and that the main value comes from making them. For instance,

Click Here To Show Diagram Code: [go]$$ Dead or alive? $$ | . . . . . . . $$ | . . . . . . . $$ | X X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | X O O O X . . $$ | . O . X . . . $$ --------------[/go]

I think that this would make a good flash card based upon this problem, but devising the card would be an important part of the learning.

Edit: In fact, I would suggest making status flash cards from tsumego problems. I. e., zero move tsumego.

Hmm, dead I think since even if white wins the ko they only have one eye and white cannot do better than a ko here I think. I could be wrong though.

Do not learn what you do not understand. Does this not go against the received wisdom for beginners? i.e. do not worry about theory and just play lots of games?

Boidhre · **#30**

Ortho wrote:

I used spaced repetition for language learning and it worked very well, and have dabbled with it in Go, but I am not convinced it's useful here in the way we're talking about.

In language there is one word for "chair", there are set forms for putting things together, etc. Mostly you just need to remember what they are, but not necessarily how to reason from them.

I agree that Go has some things like this, but mostly you have a set of principles or experiences or pattern recognitions and the challenge is to figure out which ones apply in the current context.

When I SRS Go problems there seem to just simply be too many cues based on the problem itself. I can't actually remember ever saying to myself "oh, this is like that problem I did" in a game and making a better move than I would've done.

I think maybe SRS would be useful in blindly memorizing joseki sequences, for example, but I am not sure that's a good idea anyway.

But I do think there is somewhere that memory plays a huge part, and that is in remembering the principles themselves. I don't usually lose games currently because I didn't know what to do---it's usually that I know the principle but didn't apply it. I can see an advantage in using SRS to remember things that you just have to remember, the mental checklist itself. But I'm not actually sure that the checklist is actually long enough that you need spaced repetition to remember it, especially if you're doing good game reviews after--the fact that you blew it again by not defending that cutting point just naturally turns up right when you have forgotten it, and even Anki couldn't remind you at a better time then when you're still smarting from messing it up again.

Anyway, I just make a list of the problems I fail or have trouble passing and look at them again a week later when I've hopefully forgotten them. I cross problems off the list when I think the problem is boring or when I immediately remember what the answer is.

Hmm, see I it as useful for something like one move life and death problems where you just have to find the vital point. You're not trying to memorise it per se (thus you *want* long gaps between problems you solve instantly) but trying to get a lot of practice reading these kinds of things out. I've honestly found this kind of repetition (not SRS) very useful in games for quickly killing or living within certain eye shapes.

Bill Spight · **#31**

Boidhre wrote:

Do not learn what you don't understand. Doesn't that go against what the received wisdom for beginners is? i.e. don't worry about theory, just play games.

I never received that wisdom.

I think that the point about playing games is that there are things that come up again and again that you can just pick up. When I was learning there was no go material that I knew about at that level. I suspect that more is available now, I don't know.

Bill Spight wrote:

This site by Dr. Wozniak, a spaced repetition expert, gives "20 rules of formulating knowledge in learning." http://www.supermemo.com/articles/20rules.htm

Rule 1. "Do not learn if you do not understand."

Rule 2. "Learn before you memorize."

IMO, simply making flash cards (or the computerized equivalent) out of go problems is not such a good idea. It does not follow Wozniak's rules. I suspect that making flash cards based upon go problems is a good idea, and that the main value comes from making them. For instance,

Click Here To Show Diagram Code: [go]$$ Dead or alive? $$ | . . . . . . . $$ | . . . . . . . $$ | X X X . X . . $$ | . O X , . . . $$ | O . O X X . . $$ | X O O O X . . $$ | . O . X . . . $$ --------------[/go]

I think that this would make a good flash card based upon this problem, but devising the card would be an important part of the learning.

Edit: In fact, I would suggest making status flash cards from tsumego problems. I. e., zero move tsumego.

Quote:

Hmm, dead I think since even if white wins the ko they only have one eye and white cannot do better than a ko here I think. I could be wrong though.

You do not mention it, but I suspect that you read this possibility:

Click Here To Show Diagram Code: [go]$$W Dead or alive? $$ | . . . . . . . $$ | . . . . . . . $$ | X X X . X . . $$ | 1 O X , . . . $$ | O 2 O X X . . $$ | X O O O X . . $$ | . O . X . . . $$ --------------[/go]

Your comment pertains to this possibility:

Click Here To Show Diagram Code: [go]$$W Dead or alive? $$ | . . . . . . . $$ | . . . . . . . $$ | X X X . X . . $$ | 2 O X , . . . $$ | O . O X X . . $$ | X O O O X . . $$ | 1 O . X . . . $$ --------------[/go]

You are right. White is dead.

Perhaps this might make a good flash card.

Click Here To Show Diagram Code: [go]$$ Dead or alive? $$ | . . . . . . . $$ | . . . . . . . $$ | X X X . X . . $$ | X O X , . . . $$ | O . O X X . . $$ | . O O O X . . $$ | O O . X . . . $$ --------------[/go]

The 2-3 point is a false eye.

Boidhre · **#32**

Hmm, you know Anki has a really useful function where you can upload a deck for free to a central server and then anyone can freely download it. If we had a community effort making images from tsumego we could make quite a few zero move "alive or dead" tsumego relatively quickly and have a free resource for beginners up there. Ditto a more complicated multi move remaining deck for more advanced players.

I'd be worried about making mistakes in my reading though if I was doing it! (i.e. I don't think it's right copying someone else's tsumego en masse for public consumption)

Oh and yeah, I read the self atari if white filled 1,4. I should have mentioned that.

RBerenguel · **#33**

The problem is, it takes more time to do this in Anki than creating the problem in EasyGo or similar (I know because I created a deck for the proverbs in Opening Theory Made Easy... Not that useful)

Boidhre · **#34**

RBerenguel wrote:

The problem is, it takes more time to do this in Anki than creating the problem in EasyGo or similar (I know because I created a deck for the proverbs in Opening Theory Made Easy... Not that useful)

Sure but Anki is free and cross platform, OSX, Windows, Linux, BSD, Android, iOS, (paid for on mobile devices though), while EasyGo is iOS only.

RBerenguel · **#35**

Yes, I know it's relatively free (you can install the HTML based Anki in jailbroken iOS devices and Android devices for free, but it's not as nice,) but in this kind of things you have to take into account the price in time. Generating the images of 1-move problems takes a lot of time... And is then not that much useful after a while. I'd rather have a cross-platform SGF problem viewer with SRS but...

Boidhre · **#36**

RBerenguel wrote:

Yes, I know it's relatively free (you can install the HTML based Anki in jailbroken iOS devices and Android devices for free, but it's not as nice,) but in this kind of things you have to take into account the price in time. Generating the images of 1-move problems takes a lot of time... And is then not that much useful after a while. I'd rather have a cross-platform SGF problem viewer with SRS but...

Yeah, that's the ideal.

As a side question: Given the comments above by people, why do you want SRS?

RBerenguel · **#37**

I'd like SRS to be given more frequently problems I can't solve. But I want the full sequence: I may find the first move... This qualifies as a "2". I may read the perfect sequence, this is a "5". But if there are variations I missed? Of course images would also work, but the amount of work of generating the required images is "a lot". I've been "typesetting" a lot of problems, and even that takes a lot of work (around 15 minutes for every 10 problems)! Of course, I can live without SRS for problem practice, repeating solved problems is a way to get a thick "muscle memory".

yoyoma · **#38**

I've fiddled with doing this too. My goal for using Anki/SRS system would be for *memorizing* certain things. Things that would be good to know instantly without reading. Mostly this would be the status of standard corner and side shapes, possibly including memorizing the vital point and/or mainline of the problem. Also certain standard tesujis, such as tombstone. To make good anki cards of this you would make several cards where the tombstone barely works and barely doesn't work. In this case it may be more like be able to read the mainline almost instantly instead of outright have it memorized.

Taking the tombstone example, for a long time I could solve tombstone problems in books with pretty much 100% accuracy. The problem though is that it took me quite a long time to read the sequence. But if you focused on practicing reading that exact sequence over and over, you would be able to read it faster.

Josekis could be included with targeted questions, such asking do you need a ladder for this variation, where the position shown is right at the critical moment that you have to pick a variation before the ladder actually happens. For this case I would be aiming for almost instant recognition that there is a ladder involved, and also knowing which direction it goes.

So for me this is about pattern recognition. This is different from reading practice. I think software could help for that as well, but it would be more like a method Bill described. Trying to feed you problems at a certain rate based on their difficulty to you. Don't bother showing simple problems you get 100%, and don't burden you too much with problems that are too hard. This is to exercise reading, a separate skill from pattern recognition.

Mike Novack · **#39**

Boidhre wrote:

b is not, new problems are introduced at a fixed rate per day. At least with the implementations that I've seen anyway.

I am getting even more confused.

I can see a statement like "new problems are introduced as a fixed rate per day" with reference to some specific software (or site or whatever). I cannot even imagine somebody making a claim like that "in general". I cannot even imagine somebody making a claim that all go software even accesses time/date and so wouldn't know that a day had pased(not obvious that it would need to).

Nor did I realize that we were supposedly only talking about "find the first move" problems. In many cases that is a trivial part of the problem (in a problem, black to kill the first move might be obvious whenever there is a point that makes white unconditionally alive. When doing problems beyond the easy ones, I can often find the first move but still not solve the problem correctly.

Boidhre · **#40**

Mike Novack wrote:

Boidhre wrote:

b is not, new problems are introduced at a fixed rate per day. At least with the implementations that I've seen anyway.

I am getting even more confused.

I can see a statement like "new problems are introduced as a fixed rate per day" with reference to some specific software (or site or whatever). I cannot even imagine somebody making a claim like that "in general". I cannot even imagine somebody making a claim that all go software even accesses time/date and so wouldn't know that a day had pased(not obvious that it would need to).

Nor did I realize that we were supposedly only talking about "find the first move" problems. In many cases that is a trivial part of the problem (in a problem, black to kill the first move might be obvious whenever there is a point that makes white unconditionally alive. When doing problems beyond the easy ones, I can often find the first move but still not solve the problem correctly.

My apologies for being confusing and unclear. We're not only talking about first move problems, they're just the ones I'm tending to do at the moment, my question was to others who've more experience than I whether they found this approach useful with tsumego in general, the first move problem element was descriptive of my current practices not prescriptive of what I think people should be doing. For a multi-move problem I'd only count it as a success if every move and reply was correctly read, otherwise I'd mark it as wrong, since I do such problems to practice reading rather than to memorise at the moment (I've no idea whether people memorise multi-move tsumego in the sdk and dan ranks or just use them for reading practice).

Spaced Repetition Software for Tsumego

Who is online