Life In 19x19

Not Wild

It has often been said that KataGo played wild moves far beyond human understanding. During my first three months of applied KataGo usage, this has not been my observation. More precisely, if KataGo is given too little time, it can play wildly. If, however, KataGo has the time to stabilise its selection among top candidates, it plays somewhat more sophisticatedly than humans but reasonably so. This is often the case if KataGo may generate at least 100k visits per candidate of the top few candidates. Sometimes, significantly more than 100k is needed. 50k visits per candidate of the top few candidates is definitely not enough and can result in wild play. Now, I also understand why the observation of wild play has been rather frequent: most consumer hardwares and online services offer less within reasonable time. Luckily, my RTX 4070 is fast enough if I exercise patience until move selection stabilises on every next turn.

Tactics

KataGo tactics is close to human tactics with just these differences:
- deeper reading
- more global interaction
- KataGo finds every tesuji while lazy humans might sometimes miss some

Uncommon Play

KataGo has its regularly played moves or sequences. At first, one might think that others would not be played. However, everything is played: the rarities are played when exceptional positional environments demand them. E.g., a usually bad joseki might be perfect in a special position. Therefore, do not forget the usual mistakes - use related moves or sequences when they are appropriate.

KataGo has no prejudice against "bad" shapes. Instead, it plays whichever shape is the value-best in every current position.

KataGo's Mistakes

KataGo does make mistakes. A human mistake can lose several (or even many) points. A typical KataGo mistake loses about 0.1 points. This is the difference...!

KataGo's Search

KataGo searches depth-first much more than breadth-first. Therefore, it can overlook good alternative candidates in positions that also humans consider difficult. This infrequently leads to the following behaviour: after the top-most candidate has got 5 or 10 million visits, KataGo eventually wakes up and notices that another move with previously only a few visits is not a mistake but worth careful exploration.

How do you know its mistakes are 0.1 point if you don't have a stronger tool at your disposal?
By the way, did you test it on endgame positions for which you knew the answer? Did it make mistakes?

jlt wrote:How do you know its mistakes are 0.1 point if you don't have a stronger tool at your disposal?

When having evaluated a position and then evaluating the child position, the best score changes by +-0.1 (the percentage also changes favourably) and then KataGo explores and favours a previously dismissed ("overlooked") other candidate with only then significantly better percentage and score values, which previously had values of a mistake.

By the way, did you test it on endgame positions for which you knew the answer? Did it make mistakes?

No. So far, I have only studied openings. This keeps me more than busy at least for months. Checking endgames or other things familiar to me would be interesting for fun to see if KataGo does worse or empirically confirms, but they have little or much less impact on my strength.

Studying openings is much more relevant for me because traditional opening theory has been so weak that about the only thing I keep from it is the advice to always consider the whole board position. Every other traditional opening knowledge KataGo seems to dismiss, except that it still uses some of the human josekis while it disregards many others and uses many new ones.

Studying life and death is also important but I think practising problems is more relevant for improving related strength than admiring KataGo's "reading" skills.

Robert, I sent that PM not to personally get that information but to suggest that you add it to this discussion.

Time (real time) per move is relevant. We are discussing a very high level of play so it is reasonable to compare the average amount of time per move with that of humans playing at that same high level. And with computer, there is a tradeoff between time and crunch power. Typically, if qwith given hardware crunch power it takes an average of X seconds per move, with half the crunch power we would expect around 2X per move.

Mike Novack asks: How long ON AVERAGE to stabilize every next turn?

Answer:

Maybe 150k or 200k per candidate of the top few, as the necessary amount decreases logarithmically. I have not measured it in seconds but studying one opening of ca. 50 moves I take roughly 15 minutes so that would be ca. 18s per turn.

There are some variables in my estimate, whose exclusion would require a too great effort of measurement. So take this as a rough estimate. The average might be 10s or 30s on the desktop RTX 4070 with TensorRT, well chosen Nvidia libraries and good tuning.

EDITs.