KataGo on Mac

[Kirby]

In the interest of saving time investigating this myself, I'll ask a noob question.

I compiled katago source on a mac, and I want to use it with lizzie. I saw the announcement here that I can use katago with lizzie:
https://github.com/featurecat/lizzie/releases/tag/0.7

I have the `katago` executable that I compiled locally - what do I do with it to get it working with lizzie?

[fireproof]

Hmmm, I'm not sure -- the README is certainly sparse on details. For leelaz I simply made a symlink in the Lizzie directory. perhaps the same would work for KataGo?

[Kirby]

I use macOS and have one directory containing Lizzie 0.7.1, and another directory where I've compiled KataGo and have downloaded the g104-b20c256 network. Let's call this second directory /path/to/katago

Then I edited Lizzie's config.txt:

Code: Select all

"engine-command": "/path/to/katago/katago gtp -model /path/to/katago/g104-b20c256-s447913472-d241840887.txt.gz -config /path/to/katago/my.cfg"

where my.cfg is basically a copy of configs/gtp_example.cfg that comes with katago, but I've changed a few parameters, I've commented out "maxVisits = ..." so as to not limit the search and set:

Code: Select all

numSearchThreads = 10
chosenMoveTemperatureEarly = 0.9

Brilliant! You saved me a ton of time, and things are working, now. I can see that I can tweak things in the config, too, which is great.

Thanks a ton!

[wineandgolover]

I am pulling out my hair. I have deleted and reinstalled katago and given it its initial run. It seems to recognize my eGPU, so that's promising. But then it just stops there, and doesn't go back to a command line.

...
[ 98%] Building CXX object CMakeFiles/katago.dir/main.cpp.o
[100%] Linking CXX executable katago
[100%] Built target katago
Bradys-MacBook-Pro-2018:cpp bradydaniels$ ./katago gtp -model /Users/bradydaniels/KataGo/cpp/models/g104-b20c256-s447913472-d241840887/model.txt.gz -config /Users/bradydaniels/KataGo/cpp/configs/my.cfg
KataGo v1.2
Using OpenCL Device 1: AMD Radeon Pro 580 Compute Engine (AMD) OpenCL 1.2
Loaded tuning parameters from: /Users/bradydaniels/.katago/opencltuning/tune_gpuAMDRadeonPro580ComputeEngine_x19_y19_c256_mv5.txt
Loaded model /Users/bradydaniels/KataGo/cpp/models/g104-b20c256-s447913472-d241840887/model.txt.gz
GTP ready, beginning main protocol loop

How do I know if it is working?

Per Marcel's idea, I created a folder called Katafiles, and placed the three key KataGo files (KataGo exectutable, model.txt.gz, and my.config) in it. I then placed that in the Lizzie folder. I am not 100% sure where / how Marcel inserted his text in the lizzie config file. Here is what I tried.

{
"leelaz": {
"engine-preload-list": [
false,
false,
false,
false,
false,
false,
false,
false,
false
],
"engine-command-list": [
"engine-command": “katafiles/katago gtp -model katafiles/model.txt.gz -config katafiles/my.cfg",
"",
...

When I run lizzie configured like this it crashes."The Java JAR file "lizzie.jar" could not be launched. check the console for possible error messages."

If I delete katago from the config, lizzie runs fine. What am I doing wrong? How can I tell if katago is even working?

Thanks.

[lightvector]

I am pulling out my hair. I have deleted and reinstalled katago and given it its initial run. It seems to recognize my eGPU, so that's promising. But then it just stops there, and doesn't go back to a command line.

...
[ 98%] Building CXX object CMakeFiles/katago.dir/main.cpp.o
[100%] Linking CXX executable katago
[100%] Built target katago
Bradys-MacBook-Pro-2018:cpp bradydaniels$ ./katago gtp -model /Users/bradydaniels/KataGo/cpp/models/g104-b20c256-s447913472-d241840887/model.txt.gz -config /Users/bradydaniels/KataGo/cpp/configs/my.cfg
KataGo v1.2
Using OpenCL Device 1: AMD Radeon Pro 580 Compute Engine (AMD) OpenCL 1.2
Loaded tuning parameters from: /Users/bradydaniels/.katago/opencltuning/tune_gpuAMDRadeonPro580ComputeEngine_x19_y19_c256_mv5.txt
Loaded model /Users/bradydaniels/KataGo/cpp/models/g104-b20c256-s447913472-d241840887/model.txt.gz
GTP ready, beginning main protocol loop

How do I know if it is working?

"GTP ready, beginning main protocol loop" is a very good sign, seems like everything in KataGo is working - KataGo is saying it's ready to accept GTP protocol input, which is what all Go engines and GUIs speak. To give some example input, you can type "genmove b" <enter>, "genmove w" <enter>, in alternation to have it play black and white moves and "showboard" to have it show the state of the board.

At that point KataGo is fully working and any further problem you are encountering has nothing to do with it.

Per Marcel's idea, I created a folder called Katafiles, and placed the three key KataGo files (KataGo exectutable, model.txt.gz, and my.config) in it. I then placed that in the Lizzie folder. I am not 100% sure where / how Marcel inserted his text in the lizzie config file. Here is what I tried.

{
"leelaz": {
"engine-preload-list": [
false,
false,
false,
false,
false,
false,
false,
false,
false
],
"engine-command-list": [
"engine-command": “katafiles/katago gtp -model katafiles/model.txt.gz -config katafiles/my.cfg",
"",
...

When I run lizzie configured like this it crashes."The Java JAR file "lizzie.jar" could not be launched. check the console for possible error messages."

If I delete katago from the config, lizzie runs fine. What am I doing wrong? How can I tell if katago is even working?

Thanks.

Sounds like you should check the Java console like the error message says (I forget how to do that, but it should be Googleable). Probably the problem is that there is a syntax error in the config (maybe a missing or extra punctuation mark), so that Lizzie upon startup crashes when trying to parse the config?

[wineandgolover]

"GTP ready, beginning main protocol loop" is a very good sign, seems like everything in KataGo is working - KataGo is saying it's ready to accept GTP protocol input, which is what all Go engines and GUIs speak. To give some example input, you can type "genmove b" <enter>, "genmove w" <enter>, in alternation to have it play black and white moves and "showboard" to have it show the state of the board.

At that point KataGo is fully working and any further problem you are encountering has nothing to do with it.

Thanks Lightvector. Indeed, I am excited that this seemed to work:

Bradys-MacBook-Pro-2018:lizzie bradydaniels$ katafiles/katago gtp -model katafiles/model.txt.gz -config katafiles/my.cfg
KataGo v1.2
Using OpenCL Device 1: AMD Radeon Pro 580 Compute Engine (AMD) OpenCL 1.2
Loaded tuning parameters from: /Users/bradydaniels/.katago/opencltuning/tune_gpuAMDRadeonPro580ComputeEngine_x19_y19_c256_mv5.txt
Loaded model katafiles/model.txt.gz
GTP ready, beginning main protocol loop
genmove b
= Q16

genmove w
= D16

Per Marcel's idea, I created a folder called Katafiles, and placed the three key KataGo files (KataGo exectutable, model.txt.gz, and my.config) in it. I then placed that in the Lizzie folder. I am not 100% sure where / how Marcel inserted his text in the lizzie config file. Here is what I tried.

{
"leelaz": {
"engine-preload-list": [
false,
false,
false,
false,
false,
false,
false,
false,
false
],
"engine-command-list": [
"engine-command": “katafiles/katago gtp -model katafiles/model.txt.gz -config katafiles/my.cfg",
"",
...

When I run lizzie configured like this it crashes."The Java JAR file "lizzie.jar" could not be launched. check the console for possible error messages."

If I delete katago from the config, lizzie runs fine. What am I doing wrong? How can I tell if katago is even working?

Sounds like you should check the Java console like the error message says (I forget how to do that, but it should be Googleable). Probably the problem is that there is a syntax error in the config (maybe a missing or extra punctuation mark), so that Lizzie upon startup crashes when trying to parse the config?

I'm hoping somebody sees that I did something wrong in the Lizzie Config file. To see the console, it suggested I try to run the lizzie jar in the command line, but then it says "To use the "java" command-line too you need to install a JDK" which I think would open up a new can of worms.

Marcel? Kirby? How did I blunder the config?

Thanks again, Lightvector. I am getting closer!

[wineandgolover]

I don't use "engine-preload-list"; here is the top of my Lizzie config file:
...

Everything is the default except for the "engine-command" value.

Did you try to use absolute paths to katago, the model and the config? If so, try to run that line in the terminal; does KataGo start up ok?

Maybe even relative paths can work, but then you could try "./katafiles/..." instead of just "katafiles/...",

Also, try to set "print-comms" to "true"; does it print extra debugging output?

Hi Marcel, and thanks.

I don't even know what the engine preload is. I think this is just the way my config file came, and I don't think I've changed anything except this katago command.

When I run katago in the terminal it works fine, so that is good. (Note it takes nearly 10 seconds to load, could that be the problem?):

Last login: Mon Oct 21 15:37:35 on ttys000
Bradys-MacBook-Pro-2018:~ bradydaniels$ /Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/katago gtp -model /Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/model.txt.gz -config /Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/my.cfg
KataGo v1.2
Using OpenCL Device 1: AMD Radeon Pro 580 Compute Engine (AMD) OpenCL 1.2
Loaded tuning parameters from: /Users/bradydaniels/.katago/opencltuning/tune_gpuAMDRadeonPro580ComputeEngine_x19_y19_c256_mv5.txt
Loaded model /Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/model.txt.gz
GTP ready, beginning main protocol loop
genmove b
= D4

I've modified my lizzie config file to mimic yours above (but with the command that works in my terminal), but I get the same crash as before. Here is my entire lizzie config file (sorry for length):

Code: Select all

{
  "leelaz": {
    "max-analyze-time-minutes": 99999,
    "analyze-update-interval-centisec": 10,
    "network-file": "lznetwork.gz",
    "_comment": "note, network-file is obselete in Lizzie 0.7+, ignore network-file, kept for compatibility",
    "max-game-thinking-time-seconds": 2,
    "engine-start-location": ".",
    "avoid-keep-variations": 30,
    "engine-command": “/Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/katago gtp -model /Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/model.txt.gz -config /Users/bradydaniels/Desktop/Lizzie.0.7.Mac-Linux/Lizzie/katafiles/my.cfg",
    "print-comms": false,
    "show-lcb-winrate": false
  },
  "ui": {
    "comment-font-size": 0,
    "hold-bestmoves-to-sgf": true,
    "shadow-size": 100,
    "show-winrate": true,
    "autosave-interval-seconds": -1,
    "limit-best-move-num": 0,
    "stone-indicator-type": 1,
    "win-rate-always-black": false,
    "board-width": 19,
    "show-border": false,
    "show-move-number": false,
    "winrate-stroke-width": 3,
    "show-next-moves": true,
    "comment-node-color": [
      0,
      0,
      255
    ],
    "show-comment": true,
    "show-leelaz-variation": true,
    "show-bestmoves-by-hold": true,
    "min-playout-ratio-for-stats": 0.1,
    "fancy-stones": true,
    "resume-previous-game": false,
    "comment-font-color": [
      255,
      255,
      255
    ],
    "show-coordinates": false,
    "shadows-enabled": true,
    "show-katago-estimate-onsubboard": true,
    "show-variation-graph": true,
    "show-dynamic-komi": true,
    "gtp-console-style": "body {background:#000000; color:#d0d0d0; font-family:Consolas, Menlo, Monaco, 'Ubuntu Mono', monospace; margin:4px;} .command {color:#ffffff;font-weight:bold;} .winrate {color:#ffffff;font-weight:bold;} .coord {color:#ffffff;font-weight:bold;}",
    "katago-scoremean-alwaysblack": false,
    "katago-notshow-winrate": false,
    "minimum-blunder-bar-width": 3,
    "large-winrate": false,
    "confirm-exit": false,
    "show-katago-estimate-onmainboard": true,
    "scoremean-line-color": [
      255,
      0,
      255
    ],
    "show-katago-estimate": false,
    "show-best-moves": true,
    "board-color": [
      217,
      152,
      77
    ],
    "append-winrate-to-comment": false,
    "fancy-board": true,
    "color-by-winrate-instead-of-visits": false,
    "show-captured": true,
    "replay-branch-interval-seconds": 1,
    "panel-ui": false,
    "blunder-bar-color": [
      255,
      0,
      0,
      150
    ],
    "weighted-blunder-bar-height": false,
    "katago-estimate-mode": "small",
    "theme": "Default",
    "show-winrate-in-suggestion": true,
    "show-scoremean-in-suggestion": true,
    "new-move-number-in-branch": true,
    "winrate-line-color": [
      0,
      255,
      0
    ],
    "blunder-node-colors": [],
    "show-katago-boardscoremean": false,
    "show-playouts-in-suggestion": true,
    "limit-branch-length": 0,
    "blunder-winrate-thresholds": [],
    "board-position-proportion": 4,
    "show-blunder-bar": false,
    "only-last-move-number": 0,
    "board-height": 19,
    "winrate-miss-line-color": [
      0,
      0,
      178
    ],
    "show-status": true,
    "handicap-instead-of-winrate": false,
    "large-subboard": false,
    "comment-background-color": [
      0,
      0,
      0,
      200
    ],
    "dynamic-winrate-graph-width": false,
    "show-subboard": true,
    "show-katago-scoremean": true,
    "show-comment-node-color": false,
    "board-size": 19
  }
}

When I delete the engine-command line Lizzie / LZ works fine, and the its original language reappears.I am so confused.

I've read others say their config file was corrupted. Could this be it? Is there an easy way to download a clean one? Should I just delete lizzie and try to install it again?

Regarding setting print-comms to true, when lizzie runs (LZ only), there is now a black text box to the left that seems to update when I play moves, When it crashes because I try to use katago, the board and text box don't ever appear, it just crashes. I also can't find a log.

Getting closer! Thanks again.

[wineandgolover]

Thanks Marcel,

This too failed, same crash as usual. So I guess its not the config file, but my lizzie install. I will delete everything Lizzie related and reinstall. Fingers crossed.

Again, I appreciate your patience and help, as well as that of others.

-Brady

[Kirby]

With this release of Lizzie:
https://github.com/featurecat/lizzie/releases/tag/0.7

You can specify the engine and the commands through the graphical interface. That'll populate the engine-command in Lizzie's config.txt.

[wineandgolover]

I am such an idiot. I deleted and reinstalled everything, and still had the same problem.

I tried to run the lizzie jar from the command line, but was told I needed jdk, so with trepidation, I installed it and used the terminal.

Got this feedback in terminal:

Code: Select all

Bradys-MacBook-Pro-2018:lizzie bradydaniels$ java -jar Lizzie.jar
Exception in thread "main" org.json.JSONException: Expected a ',' or '}' at 377 [character 24 line 10]
	at org.json.JSONTokener.syntaxError(JSONTokener.java:505)
	at org.json.JSONObject.<init>(JSONObject.java:263)
	at org.json.JSONTokener.nextValue(JSONTokener.java:431)
	at org.json.JSONObject.<init>(JSONObject.java:244)
	at featurecat.lizzie.Config.loadAndMergeConfig(Config.java:131)
	at featurecat.lizzie.Config.<init>(Config.java:188)
	at featurecat.lizzie.Lizzie.main(Lizzie.java:31)

Counted lines and characters, and turns out I had a weird quotation mark in my config text.

Code: Select all

"engine-command": “/Users

And this was enough to screw up the app. I assume that my Mac did an auto-format? I dunno. Anyway, I changed it and it's working now.

Kirby, installing LZ (hopefully using the GUI) is my next step, I didn't realize that installing katago the way would drop LZ, but it makes sense. I think Lizzie 0.71 should be able to switch between LZ and Katago, without relaunching, right? That'd be cool. Maybe minigo, too? edit - I have both LZ and Katago running now, using the engine config in the gui. yay!

Oh yeah, lightvector, you'll be pleased to know that when I check the Activity Monitor, KataGo is using my external GPU, as I hoped. The auto-tune worked without my forcing it. It's nowhere near as fast as LZ, but I will take a look at suggestions on how to optimize it.

Thanks to lightvector, Marcel, and Kirby. Your help is truly appreciated. I am delighted to finally have katago up and running!

[Kirby]

This must be at least the third time I've seen bug due to different format of quotation mark. I have to remember that for when their are weird issues.

I believe you can switch back and forth between engines.

[Bill Spight]

I have a 2014 MacBook Pro, and this year is when I would usually get a new computer. However, looking around the web for what is available, it seems like a lot of things are changing about hardware to run neural nets, so it may be best to wait a while. One thing I am thinking of is getting an NVIDIA RTX 2080 Ti, which seems to be the thing to get these days, and using it as an external GPU. The Macbook has Thunderbolt, but not Thunderbolt 3. Apparently that can be handled with an adapter. If I go that route I have two concerns. First, I have only 16 GB of RAM, which might not be enough for good performance. I have seen 64 GB recommended. Second, one possible problem may be overheating the MacBook. OC, the external GPU will have its own cooling. But the one guy I found using an external GPU with a 2014 MacBook Pro had a silent video about his setup, which showed a temperature monitoring program on his machine.

I welcome any ideas and suggestions. Many thanks.

[wineandgolover]

I have a 2014 MacBook Pro, and this year is when I would usually get a new computer. However, looking around the web for what is available, it seems like a lot of things are changing about hardware to run neural nets, so it may be best to wait a while. One thing I am thinking of is getting an NVIDIA RTX 2080 Ti, which seems to be the thing to get these days, and using it as an external GPU. The Macbook has Thunderbolt, but not Thunderbolt 3. Apparently that can be handled with an adapter. If I go that route I have two concerns. First, I have only 16 GB of RAM, which might not be enough for good performance. I have seen 64 GB recommended. Second, one possible problem may be overheating the MacBook. OC, the external GPU will have its own cooling. But the one guy I found using an external GPU with a 2014 MacBook Pro had a silent video about his setup, which showed a temperature monitoring program on his machine.

I welcome any ideas and suggestions. Many thanks.

From https://9to5mac.com/guides/egpu/

eGPUs require a Thunderbolt 3 connection to work with Macs, meaning only Thunderbolt 3-enabled Macs are eligible.

Pretty sure this is for speed, not just a connection, which means an adaptor won’t do the trick, and you'd need a more recent MBP, sorry.

I bought the basic BlackMagic eGPU, and it works well with my 2018 MBP, using Thunderbolt 3. Regarding RAM, I have 8gb and it’s fine, but I’d get more given the choice.

One huge red flag about buying a new Mac for these bots, though. Apple has deprecated OpenCL support, meaning that it works fine now, but isn’t supported going forward, and future changes in either OpenCL or Mac OS could / will break it. I believe Apple doesn’t support CUDA either, all this in favor of their own Metal approach. This would be fine if anybody was developing these apps to support Metal, but I don’t think they are.

Even though I've built a highly Apple oriented ecosystem in my house, I suspect my next purchase might be a PC-gaming rig.

For a portable solution, the current generation iPad Air, Mini, and Pro are all very fast using “A Master of Go”, as are the iPhone 10S's and 11’s. (Anything post 10)

I wish I had a better answer for you, Bill.

[Bill Spight]

Marcel, Brady,

Thank you both very much. It sounds like I need to get a gamer desktop, which was my first thought, anyway. Maybe I can get a nice one on sale this month, quien sabe?

[Bill Spight]

Having failed in the spring, and doubtful that I'll find a good bargain GPU this month or next month, I am giving another try to implementing Lizzie an my 2014 MacBook Pro. Apple says my graphics card, NVIDIA GeForce GT 750M, is compatible with Open CL 1.2. But from the installation instructions it is unclear whether I have to do anything about Open CL or if it is taken care of. No problemo, or do I have to do something? Thanks.