| Life In 19x19 http://www.lifein19x19.com/ |
|
| Computer Specs for Go Review with AI http://www.lifein19x19.com/viewtopic.php?f=18&t=17287 |
Page 1 of 3 |
| Author: | dino1019 [ Mon Feb 24, 2020 7:04 am ] |
| Post subject: | Computer Specs for Go Review with AI |
Anyone can provide a reference desktop computer specs for go AI review purpose? My computer vendor asks me to provide one based on which we can decide the specs, analysis speed, and cost. I have a notebook with Intel i5 CPU and Windows 10 64 bit OS, and a note 8 phone. I want to run go AI for reviewing games and find bad moves and good moves, in terms of winrate delta or score delta. Typical use cases are: Ah-Q app on my phone that connects to LZ engine (with different weights) my notebook via SSH, but this is limited to LZ but not Katago, since the latter requires GPU. Run go review partner on my notebook, but this way I have to set with my notebook. The analysis speed of LZ264 (40b) is about 3.5 visits / secs., if I want to analyze a game with 250 moves with 10k visits / move, it takes about 177 minutes, if 1k, 17 minutes. I wonder if I have a GPU desktop, I can run it even faster. Anyone any advice? (Computer specs, and analysis speed) Thanks in advance. |
|
| Author: | Gomoto [ Mon Feb 24, 2020 7:27 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
Use Zbaduk for now. All other advice is related to your budget. How much do you want to spend. Desktop GPUS are much faster. |
|
| Author: | Bill Spight [ Mon Feb 24, 2020 9:10 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
dino1019 wrote: The analysis speed of LZ264 (40b) is about 3.5 visits / secs., if I want to analyze a game with 250 moves with 10k visits / move, it takes about 177 minutes, if 1k, 17 minutes. I wonder if I have a GPU desktop, I can run it even faster. One thing you might do is to run the analysis for 15 minutes. Then, based on the analysis, play moves that you are interested in understanding better to get LZ's opinion at 10k. It is quite possible that at 10k for a particular move, LZ will change its mind about a move you made or about the move it would choose. |
|
| Author: | RobertJasiek [ Mon Feb 24, 2020 9:28 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
How does Apple A13 compare to Desktop GPUs? |
|
| Author: | Jæja [ Mon Feb 24, 2020 10:06 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
dino1019 wrote: Ah-Q app on my phone that connects to LZ engine (with different weights) my notebook via SSH, but this is limited to LZ but not Katago, since the latter requires GPU. Katago can run on the Intel Integrated Graphics, such as the Intel Iris Graphics 6100 found in my MacBook Pro (early 2015 model). I've read something about Katago having issues with it, but I haven't run into this. Gomoto wrote: Use Zbaduk for now. I agree with this. Forum member spook is doing an amazing job with ZBaduk. It works very well, it's easy (no need to install anything) and relatively cheap! |
|
| Author: | dino1019 [ Mon Feb 24, 2020 8:43 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
Yes, zbaduk is a potentially useful go review tool, but it's not fully customized for mobile as ogs does yet, also I cannot see list of bad moves as AI sensei does. ogs only shows the first 3 bad moves before paying, and AI sensei provides free analysis for 3 games only. If there's a fast, mobile and low cost way of knowing bad moves, then I can use Ah-Q + LZ@PC remotely for live analysis on bad moves. |
|
| Author: | inbae [ Tue Feb 25, 2020 2:19 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
You can also consider setting a cloud instance and running KataGo/LZ on there. If you don't play computer games very much, it can be a reasonable choice, even in terms of budget. |
|
| Author: | explo [ Tue Feb 25, 2020 2:35 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
dino1019 wrote: The analysis speed of LZ264 (40b) is about 3.5 visits / secs., if I want to analyze a game with 250 moves with 10k visits / move, it takes about 177 minutes, if 1k, 17 minutes. I wonder if I have a GPU desktop, I can run it even faster. Anyone any advice? (Computer specs, and analysis speed) Thanks in advance. I don't know how much you can/want to invest. On my GTX 1660 Ti (around 300€), I get ~300-350 visits per second on LZ265 using Lizzie. |
|
| Author: | jlt [ Tue Feb 25, 2020 2:47 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
dino1019 wrote: The analysis speed of LZ264 (40b) is about 3.5 visits / secs., if I want to analyze a game with 250 moves with 10k visits / move, it takes about 177 minutes, if 1k, 17 minutes. How did you make that calculation? I got 250*10000/3.5/60 = 11905 minutes = 198 hours. If your computer is too slow, use 15-block networks instead, they are strong enough. |
|
| Author: | xela [ Tue Feb 25, 2020 4:22 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
dino1019 wrote: The analysis speed of LZ264 (40b) is about 3.5 visits / secs., if I want to analyze a game with 250 moves with 10k visits / move, it takes about 177 minutes, if 1k, 17 minutes. I wonder if I have a GPU desktop, I can run it even faster. Anyone any advice? (Computer specs, and analysis speed) Thanks in advance. 3.5 visits/sec, are you sure you don't mean 35? My Ryzen 5 (without using GPU) gives me 25-30 visits/sec for a 40b network. Any mid price GPU will take you up to 200-300 visits/sec, no need to fuss too much over the specs. I get a little over 200 on my GTX 1070. |
|
| Author: | dino1019 [ Tue Feb 25, 2020 8:51 am ] | ||
| Post subject: | Re: Computer Specs for Go Review with AI | ||
Attached is screenshot of Ah-Q app with remote connection to Windows 10 (Intel i5) and run LZ264 (40b). You see the 500 visits are received in 2 or 2.5 min., so the speed is 3.5 to 5 vps. (the default threads is 4)
|
|||
| Author: | Bill Spight [ Tue Feb 25, 2020 10:06 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
You understand that the win rate estimates based upon fewer than 100 visits are basically meaningless. In fact, the Elf commentaries on pro games do not report any winrate estimates based on fewer than 500 visits. |
|
| Author: | jlt [ Tue Feb 25, 2020 10:17 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
For a kyu player, 100 visits is fine if you use Katago (otherwise LeelaZero may occasionally misread a ladder). Not that the bot will never make mistakes, but it will not make more mistakes than a high-dan player anyway. P.S. My notebook computer doesn't have a GPU, and I get about 50 visits/second with 15-block Katago. |
|
| Author: | go4thewin [ Tue Feb 25, 2020 4:16 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
xela wrote: dino1019 wrote: The analysis speed of LZ264 (40b) is about 3.5 visits / secs., if I want to analyze a game with 250 moves with 10k visits / move, it takes about 177 minutes, if 1k, 17 minutes. I wonder if I have a GPU desktop, I can run it even faster. Anyone any advice? (Computer specs, and analysis speed) Thanks in advance. 3.5 visits/sec, are you sure you don't mean 35? My Ryzen 5 (without using GPU) gives me 25-30 visits/sec for a 40b network. Any mid price GPU will take you up to 200-300 visits/sec, no need to fuss too much over the specs. I get a little over 200 on my GTX 1070. The Ryzen 5 has vega graphics, much faster than intel hd graphics. Ryzen 5 desktop might have vega 11, which is faster than a gt 1030 for opencl! https://browser.geekbench.com/opencl-benchmarks |
|
| Author: | Bill Spight [ Tue Feb 25, 2020 4:53 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
jlt wrote: For a kyu player, 100 visits is fine if you use Katago (otherwise LeelaZero may occasionally misread a ladder). Not that the bot will never make mistakes, but it will not make more mistakes than a high-dan player anyway. Incorrect winrate estimates do not mean that the order of correct winrates would be any different. But it's not unusual that the order will change. That's why I recommend doing a quick first run and then comparing interesting options by making those plays. Doing so will produce high enough visits to be reasonably confident about the comparisons. In the example given, for instance, the keima might be interesting, even though it has a low winrate estimate and only 16 visits. Making that play could easily change the winrate estimate by 10% or more, with 500 visits, or 1,000 visits, or 10,000 visits. It's not like you do that for every option, just for the ones that look interesting after the first run. |
|
| Author: | xela [ Tue Feb 25, 2020 5:31 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
Bill Spight wrote: You understand that the win rate estimates based upon fewer than 100 visits are basically meaningless. I think "meaningless" is too strong a word. If you care about the difference between a 53% move and a 55% move in the fuseki, then 100 visits isn't enough. But for reviewing your games and spotting kyu level mistakes (and even dan players make a handful of kyu level mistakes per game: weeding those out is a good way to improve), or for suggesting "here's another move you might have looked at" or "here's the natural move in this shape", small numbers of visits will still get the job done. Remember that the newer networks can play at dan level even on just one playout. Of course more is better, and getting a GPU is a good idea :-) |
|
| Author: | Bill Spight [ Tue Feb 25, 2020 6:03 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
xela wrote: Bill Spight wrote: You understand that the win rate estimates based upon fewer than 100 visits are basically meaningless. I think "meaningless" is too strong a word. If you care about the difference between a 53% move and a 55% move in the fuseki, then 100 visits isn't enough. As you may have noticed, I regard a 2% difference as within the margin of error, except perhaps with millions of visits. Quote: But for reviewing your games and spotting kyu level mistakes (and even dan players make a handful of kyu level mistakes per game: weeding those out is a good way to improve), or for suggesting "here's another move you might have looked at" or "here's the natural move in this shape", small numbers of visits will still get the job done. Well, that's what I recommend doing. Take the low number of playouts as suggesting a move, but do not rely upon the winrate estimate; make the play and get a high number of visits. |
|
| Author: | dino1019 [ Tue Feb 25, 2020 7:06 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
(Again, how AI helps human review a game, in particular a kyu player, I re-posted my comments at reddit/cbaduk here, note that the 3.5 vps speed for 40b on my notebook computer is shown above with a screenshot) I have a few points to make here: I am not sure whether a best move suggested by AI is a best move for a kyu level player and win rate (or score) itself might not be as relevant as win rate delta (or score delta). Note that win rate delta is the difference between the best move (the move with the highest win rate) and the actual move. If the delta is huge (>20%) then this move is probably a big mistake, and in this case, this is the key lesson for the player to learn. That's the reason I think the filter for the win rate delta in AI sensei is useful. (but 0.5 to 1 euro for full review of a game is too expensive) The use case of game review with AI for me is Ah-Q app that connects to my notebook computer (Windows 10, via SSH) with CPU-only Leela Zero. Many people say 100 visits per move is good enough for a kyu player (15, 20, or 40 blocks?), this is not the case. First of all, we know that Leela Zero has a problem with ko, seki, ladder, and I also see a stupid mistake about life and death, if the visits per move is not high enough. In the meantime, I see most full game review services are with 1000 or 1600 visits per move, so I think this is a good reference number. The reason I am looking at a PC with GPU is because the speed is important not only for a full game review but also for a live analysis. The live analysis in the above use case sees an issue if the calculation is too slow (in my current settings, it's only about 3.5 to 5 visits per second for LZ264(40b)), and in the process the best move can change around, which will be even more confusing for a kyu player (if the AI stabilizes quickly on one best move. then at least I know what the AI thinks the best move is). I heard a 1060 GPU (4G) has about 200 visits per second with LZ 40b, this seems to a bit slow if 1000 to 1600 per move is preferred, so maybe a higher spec. GPU is better. (Note in this context, visits = playouts = nodes) |
|
| Author: | xela [ Tue Feb 25, 2020 7:41 pm ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
200 visits/sec is actually quite nice: you get to see LZ's "first thought", then see whether or not it changes its mind on further investigation over the next few seconds. So you get some insight into whether the best move should be obvious or whether it requires reading. Besides, if you're reviewing interactively, you probably can't take in information much more quickly than 5 seconds per move anyway. No harm in aiming for a faster GPU, but I'd say it's nice to have not essential. The ladder problem is only minor, and pretty much solved with the 40b networks. The policy net can "see" the ladder shape and any ladder-breakers without any playouts. (I'm in the middle of testing this and hope to write up some results soon.) Can the app work with KataGo, or is it LZ-only? It's already been said here, but I want to emphasise: even with CPU only, if you're getting 3 visits/sec then probably something is wrong with your setup. On an i5 it should be going 5-10x faster. I don't know if it's the config on the notebook itself, or something to do with communication between the notebook and the app. But you might want to look into that before spending money on hardware. |
|
| Author: | jlt [ Wed Feb 26, 2020 12:10 am ] |
| Post subject: | Re: Computer Specs for Go Review with AI |
I've self-reviewed a number of kyu games using 15-block networks, making sure that each move of each variation gets at least 100 visits (occasionally I let it run longer), and also had some games reviewed by strong players. The human's and the AI's suggestions can be divided into the following categories: A. The human and the AI agree that a move was a big mistake, and suggest the same better move. B. The human and the AI agree that a move was a big mistake, but the move suggested by the human would be considered as a mistake by the AI. Either the human didn't see the better move, or he saw it but chose not to mention it because he thought it was too complicated. C. The AI thinks a move is a small mistake (about -3%), so is hard to detect in a self-review, but the mistake can easily be avoided by using human reasoning, so is pointed out by the human. D. The AI notices a big mistake, and the human doesn't. E. The AI makes a mistake. Getting a fast computer eliminates "E", which occurs less than 1% of the time, but won't improve "C", which occurs much more frequently. So I still think that as far as kyu games are concerned, for teaching purposes humans are better than computers, and that the improvement we get from upgrading a computer is marginal. Actually I haven't seen AI mistakes recently. I did notice a small number of mistakes with earlier networks like LZ157 at low playouts like:
but I didn't notice such mistakes with Katago (since Katago reads ladders and counts liberties). It doesn't mean Katago doesn't make life-and-death mistakes, but they occur rarely enough so that it's not a problem. |
|
| Page 1 of 3 | All times are UTC - 8 hours [ DST ] |
| Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |
|