Hi,

I quite like Go but never liked tsumego, finding them tedious and too small in scope, so I've made a website with what I think to be a more rewarding and fun approach to Go problems. They are whole board positions from pro games turned into problems by AI (curated by me). It's still a work in progress but should be able to handle some traffic. I'd love to know what you all think of the concept and the quality of the problems.

I assigned the initial difficulty ratings myself, and especially in the 2200+ (4d+) range they are not likely to be accurate. Some of them could be simply too difficult, but it's hard for me to judge what's too difficult for high dan players.

https://realgoproblems.com/

Would appreciate any and all feedback!

Nice idea, however I don't always understand why my wrong answers are wrong and why Katago's answers are correct. Perhaps show a few followup moves?

Is the concept similar to
https://neuralnetgoproblems.com/ ?

The link takes me to a login page. I can't see anything else without registering. Literally nothing, it's just a black background, not even a bit of basic info about the site.

I've already got more accounts on more sites than I want to keep track of. I'm not going to create another one before I've had a look around and decided if I'm interested in the content. Compare goproblems.com, tsumego-hero.com, 101weiqi.com...

I considered adding the AI variation but given the (low) amount of playouts per position it's likely to confuse the user more than help them. I will probably add a few extra moves as played in the game though, in case AI and pro played the same move (usually the case).

One feature I want to add is allowing users to comment on problems and add variations, which would become visible with the solution and could help clarify things.

Yes, though I don't know how they generated the problems. I suppose the biggest difference is the rating system, which I believe to be important because it gives users something to play for, a way to measure progress, and it allows the ratings of the problems to grow organically, which will eventually make them perfectly accurate.

Fair point, I'll add a link to the about page there.

You can now play without registering

Thanks, this is what I was looking for :-) I think the concept is promising but difficult to execute well. Please keep working on it!

Some low hanging fruit (*) for the UI:

• The thing I most want is a "download SGF" button. This is for playing out some variations on the board, and also asking my local KataGo installation (a) why my wrong answers are actually wrong; (b) what a good followup looks like after the right move.
• In similar vein, the ability to scroll through the next few moves of the game continuation would be nice.
• Browse problems by difficulty level. Some days I might be looking more for "easy target practice", other days for a serious challenge. I'm happy with the default being that the site chooses problems appropriate to your level, but it would be good to have other options too.
• The bottom of the board is right at the bottom of the page. Some space below would look better.
• Personally I'm not a fan of "fake realism". In particular, wood grain on a computer screen just looks wrong to me. A settings menu with the option of a plainer board style (something more similar to the diagrams on this site) would be nice.
• The artwork on the login page isn't exactly inviting :-) I guess it's still early days.

Also, I know this is much less easy, but I'd strongly prefer discussion forums integrated with the site rather than a discord channel. On-site discussion means you don't need a separate login, and it means that things get indexed by search engines so it's easier to find old conversations.

* I say that, but I know these things are often harder than they look!

The info is all there on the site. Just click the "about" button at top right.

For me the big difference is that most of the positions on neuralnetgoproblems are what I think of as basic instinct problems: things you can solve based on a sense of shape or direction of play, without reading or deep positional judgement. I think realgoproblems is already covering a wider variety of problem types. This has both pros and cons.

The "basic instinct" character comes from the fact that the right answer is the highest policy move in a particular neural net. This somehow captures the idea that the move is "right" based on general considerations, rather than needing to read out variations. It also means that once the right answer is on the board, you often have a sense that it feels right, rather than thinking "why would anyone want to play that move???"

Two other key differences:

• I believe neuralnetgoproblems is 100% automatically generated, whereas realgoproblems is manually curated (some automation for making a list of candidate positions, but a human has judged which ones make suitable problems, right?)
• neuralnetgoproblems does a couple of things to soften the feeling of "it's right or it's wrong, no shades of grey, no explanation". Most of the problems have not just a single right answer, but also a number of "near miss" choices. And you have the option of seeing the next 9 moves of the game, which sometimes (not always!) give an insight into the logic of the right answer.

I tried to do something similar early this year, but gave up!

The biggest issue I see with https://realgoproblems.com/ in its current state is that it's a bit unsatisfying to put just a single move on the board and be told "right" or "wrong" with absolutely no explanation. I believe you plan to address this in the long term by allowing users to submit comments and variations. If you get enough users and build up the right culture, this could work well. But it's a long road...

For a few weeks, when I reviewed my games with KataGo I was noting the positions where it surprised me. Then I went through the list looking for things I could post on goproblems.com. My criteria were:

• A unique best move.
• A continuation making it clear why the best move leads to a good result.
• Refutations of plausible but wrong attempts. If a solver plays the wrong answer, they should get some feedback as to why it's wrong.

The main issue I had was the continuation. Finding a problem position with a clear-cut best move is one thing. Having a unique and clear best move at each step of the variation is much harder to achieve! I kept finding positions with several almost-equally good alternatives, and thinking that in order for the problem to be "fair" (you shouldn't mark it wrong if the user picks a move that's 0.1 point worse than the "best") I needed to add variations for all of those options. So the number of variations kind of exploded.

Listing the other plausible attempts and refutations also caused a bit of an explosion of variations.

Ultimately I discarded three quarters of my original candidate positions, and spent 30-60 minutes per problem working out the variations for what was left. Then I uploaded three of them, waited for some feedback before posting the rest, and found that goproblems.com is no longer the lively discussion forum it used to be :-/

I've just uploaded the rest of the set in their current state. Chances are there are few errors, and a few important things missing. At this stage I'm unlikely to do more. If you want to see how far I got, check out https://goproblems.com/problems/groups? ... ection=247. Hopefully https://realgoproblems.com/ can get over or around these sorts of roadbumps and go further!

Thanks so much for the thorough feedback! I added most of it to my rapidly growing todo-list.

Initially there were more problems per game, so I decided against a "download SGF" feature early on. After all, users would be able to analyze the game, then go to the next problem and immediately know the answer if it's from the same game. But over time I've landed on a way to generate the problems that leads to much less problems per game (typically one or two), so it's safe to add now.

I like your problem set! I considered eventually adding a way for people to submit their own problems but wasn't sure if anyone would. This has given me renewed interest in the feature.

I think that's right, and it's a hard problem to solve. As you mentioned, eventually I want users to be able to add comments and variations but that is a long road indeed. Something else I recently thought of and you mentioned as well is, as more users attempt the problems, "common mistakes" might emerge. And I could have KataGo have an automated look at those common mistakes and give refutations.

I'm afraid that adding the recommended variations other than what was played in the game, or multiple-move refutations to these common mistakes, and doing this with any kind of confidence that the recommendation/refutation is actually correct all the way through, is simply too costly when it comes to GPU resources. But this might change in the future.

Yes, efficiency is an issue here. That's why I only made 11 problems before giving up, while you're at several hundred and counting :-)