The idea of presenting test questions whose difficulty matches the ability of the test taker is sound, both in terms of pedagogy and of accurate assessment of the ability of the test taker. However, when you start with questions of unknown difficulty and test takers of unknown ability, that is not an easy thing to do. The difficulty is further compounded by not knowing the answers to the questions with certainty.
There are two major properties of test questions, the difficulty of the question and how well it discriminates between test takers of different ability. The test takers have similar properties. For instance, a person may do better on hard questions than on easy ones. Neither the ability of a person nor the difficulty of a question is one dimensional. They can be reduced to single numbers only approximately or on average. (You see the same thing with go ratings or ranks.)
(;CA[ISO8859-1]AP[GOWrite:2.2.21]ST[2]FF[4]GM[1]SZ[19]PM[2]PB[ ]FG[259:]PW[ ]GN[ ]
;B[pd]
;W[dp]
;B[pp]
;W[dc]
(
;C[*** Go Seigen argues against this play. See variations for next and then come back to variations for this play.]B[ce]
(
;C[*** White has a choice among six different pincers, which Black must be ready for.]W[cg]
)
(
;W[dg]
)
(
;W[ch]
)
(
;W[dh]
)
(
;W[ci]
)
(
;W[di]
)
)
(
;C[*** My guess is that this is the most popular play.]B[de]
)
(
;C[*** But this is what Go Seigen likes. White does not have a severe pincer and his choices are limited.]B[cf]
)
)
This example is based on one by Go Seigen in his
21st Century Go series, vol. 4. Go to
. You will see six options for
. All are pincers. Let us assume that White should pincer. Even though Go Seigen may believe that one pincer is better than another, he did not address that question. Rather, he focused on the practical matter that
gave White many options. (I'll come back to that later.
)
To me this seems like an easy question. To pincer or not to pincer. (As Hamlet may have put it.
) My guess is also that it differentiates between SDKs and DDks pretty well, even though a lot of DDKs will pincer and some SDKs will not. So maybe it has a difficulty of around 12 kyu and a discrimination score of around 0.7 (on a scale of 0 to 1). Now, it may be, if we look closely at the specific answers, that it could discriminate between players who play on the second line and those who do not. That would mean rating the answers, and not just dividing them into pincers vs. non-pincers.
But if we present this question to people around 12 kyu (in terms of fuseki ability), we are not interested in sorting out those whose non-pincers are worse than the non-pincers of others. We want to present easier questions to them.
Nor are we interested in sorting out those who make different pincers. Go Seigen included in his example the three space high pincer. I do not like it, myself, and I suspect that few pros would play it, if any. But for a 12 kyu problem it should be one of the correct answers. Perhaps some 1 kyu or stronger players would choose it. If we consider it to be incorrect, then the discrimination score of this question may drop. It may become more difficult and less discriminating at the same time. We want to present harder questions to the 1 kyus.
What about
? Go Seigen does not like it, nor do I, nor, I expect, do most pros. My guess is that the one space high approach is popular. But Go Seigen does not fault
on theoretical grounds. He does not say that it is objectively worse than other approaches, and give a variation or two to show what is wrong with it. Rather, he makes the practical point that it gives White too many options. He likes the large knight's approach, which reduces White's options and reduces the severity of a pincer. So if the question were where to play
, I think that the small knight's approach should be among the correct answers.
Another play that has fallen out of favor is the large knight's response to the small knight's approach to the 4-4 stone. That was popular for many centuries. But in the 20th century Go Seigen argued that the small knight's response was OK. (If you start with one player on the 3-6 and the other on the 6-3, an obviously equal position in the corner, the 4-4 is a good play. So it must be OK to respond on the 3-6 if you have a stone on the 4-4 and the opponent plays on the 6-3.) That does not mean that responding on the 3-7 is bad, but it gives your opponent a whole lot of options! Again, without a theoretical argument against the large knight's response, it should be included among the correct answers in many positions, even if it does not show up in any recent databases.
There are not only differences in style in go, but differences in fashion. The Mini-Chinese was in the textbooks at the start of the 19th century, but went out of fashion, only to stage a comeback in the late 20th century.
Just because a play is not currently popular among pros does not mean that it is inferior.
Now I do not know how much statistical sophistication Ootakamoku's web site might need, but I do think that it would be advisable to try to weed out positions that do not discriminate well at their level of difficulty. I also think that if a pro makes a play in the database, it should be considered correct, even though some mistakes may be counted as correct. The site is not presenting any positions to pros, in an attempt to rate their abilities. Pro errors are better than amateur shodan errors, as a rule. Including them among the correct answers in positions at the amateur shodan level may well improve the discrimination scores of those positions. I also think that it would be advisable to extend the database back to the early 20th century, at least. Yes, we have made progress in the fuseki since then, but a large database is good for the discrimination scores of the questions. For dan level positions, it might be better to go back only to the mid-20th century, however.
I know well not to take this serious. But I’d really like to know whether strong players would recommend to use this as training … or perhaps better: up to which strength could this be useful? I have the impression that at my rank (13k EGF) it might help me train some “reflexes”.