I made a few games, the optimized network is unexpectedly weak
https://lifein19x19.com/viewtopic.php?p=271178#p271178
EDIT Turned out to be the clock
| Life In 19x19 http://www.lifein19x19.com/ |
|
| New android app "BadukAI" http://www.lifein19x19.com/viewtopic.php?f=18&t=17784 |
Page 21 of 27 |
| Author: | go4thewin [ Tue Feb 22, 2022 4:00 pm ] |
| Post subject: | Re: New android app "BadukAI" |
I know its a lot to ask, but would it be possible to run 5-10 matches of the latest snapdragon optimized network vs the last non distributed training 40b release at 300 playouts each? It is so impressive what you have done. Probably the slight difference is just from the parrallel processing? |
|
| Author: | And [ Tue Feb 22, 2022 9:13 pm ] |
| Post subject: | Re: New android app "BadukAI" |
go4thewin wrote: I know its a lot to ask, but would it be possible to run 5-10 matches of the latest snapdragon optimized network vs the last non distributed training 40b release at 300 playouts each? It is so impressive what you have done. Probably the slight difference is just from the parrallel processing? the difference in strength is small, it will take a lot of games |
|
| Author: | And [ Wed Feb 23, 2022 3:13 am ] |
| Post subject: | Re: New android app "BadukAI" |
I made a few games, the optimized network is unexpectedly weak https://lifein19x19.com/viewtopic.php?p=271178#p271178 EDIT Turned out to be the clock
|
|
| Author: | go4thewin [ Wed Feb 23, 2022 8:27 pm ] |
| Post subject: | Re: New android app "BadukAI" |
Thanks And for the attempt, I really appreciate the effort! Of course the optimized version may be a little weaker than the same net, but i am very curious how it would compare to the older non-distributed training katago. You gave me an idea, maybe i will try two games at 1 playout. My guess is the optimized version is stronger. |
|
| Author: | akigo [ Fri Feb 25, 2022 1:13 pm ] |
| Post subject: | Re: New android app "BadukAI" |
go4thewin wrote: I know its a lot to ask, but would it be possible to run 5-10 matches of the latest snapdragon optimized network vs the last non distributed training 40b release at 300 playouts each? It is so impressive what you have done. Probably the slight difference is just from the parrallel processing? I ran 6 matches (with alternating colors) as you described, and the result was 4-2 in favour of the optimized network. |
|
| Author: | And [ Fri Feb 25, 2022 2:46 pm ] |
| Post subject: | Re: New android app "BadukAI" |
suppose the two networks are equal in strength. then the six game match will end with a score of 0:6 with a probability of 0.015625 (1/64) with a score of 1:5 with a probability of 0.09375 (6/64) with a score of 2:4 with a probability of 0.234375 (15/64) with a score of 3:3 with a probability of 0.3125 (20/64) with a score of 4:2 with a probability of 0.234375 (15/64) with a score of 5:1 with a probability of 0.09375 (6/64) with a score of 6:0 with a probability of 0.015625 (1/64) six games is not enough to say anything with certainty. maybe analyze these games and see how does the optimized network perform compared to the original one? Are there any weak or weird moves? can you upload sgf? I think the difference is negligible, BadukAI and optimized networks are amazing!!! thanks akigo!!! |
|
| Author: | go4thewin [ Sat Feb 26, 2022 9:18 pm ] |
| Post subject: | Re: New android app "BadukAI" |
I was getting this same feeling using the snapdragon optimized network against golaxy. it definitely felt at least as strong as the s509 network. What an achievement by all of you to create such a strong, small and fast network. You have made katago play at a professional level even on cheap mobile devices. The kyu mode works well too with the opening book turned on. Even at 10 kyu, the optimized network does not make egregious errors. I am just in awe of how well these projects went and the immense power of modern computing to solve problems that humans cannot. |
|
| Author: | lovego [ Sun Mar 06, 2022 7:20 am ] |
| Post subject: | Re: New android app "BadukAI" |
I have two mobile phones, one of which is Huawei mate30 and its processor is kirin990. But its speed is much slower than that of mobile phones using Qualcomm chips. Can it be optimized? |
|
| Author: | akigo [ Sun Mar 06, 2022 10:41 am ] |
| Post subject: | Re: New android app "BadukAI" |
lovego wrote: I have two mobile phones, one of which is Huawei mate30 and its processor is kirin990. But its speed is much slower than that of mobile phones using Qualcomm chips. Can it be optimized? Unfortunately not, since the optimized networks use per-channel quantization (otherwise it's impossible to achieve the accuracy that is needed). And the NPU on Kirin 990 doesn't support this according to https://github.com/tensorflow/tensorflow/issues/43128 |
|
| Author: | lovego [ Mon Mar 07, 2022 4:46 am ] |
| Post subject: | Re: New android app "BadukAI" |
What a pity, but can you provide a low precision but faster weight for huawei phones? I can test it |
|
| Author: | And [ Mon Mar 07, 2022 5:31 am ] |
| Post subject: | Re: New android app "BadukAI" |
lovego wrote: What a pity, but can you provide a low precision but faster weight for huawei phones? I can test it then what's the point? the network will be weak. try 20 or 15 block network |
|
| Author: | akigo [ Mon Mar 07, 2022 4:18 pm ] |
| Post subject: | Re: New android app "BadukAI" |
And is right: With other quantization methods the error will be larger than the difference between best and worst move, so the network will produce pretty random results (I tried this some time ago for a different purpose). Such a network would be useless, no matter how fast it is. |
|
| Author: | floating [ Thu Mar 10, 2022 11:54 am ] |
| Post subject: | Re: New android app "BadukAI" |
I decided to retry badukAI (last time I couldn't get it to work). I downloaded the apk 1.13 from https://aki65.github.io/ and also downloaded the bin.gz and .tflite from the assets (1.13.2) at github releases page. The 20b and 10b internal networks seem to work now which is great! Though once I load network from storage and choose those files I downloaded it says AI engine crashed or such when I turn the play B or Play W button blue as in instructing it to play it. I tried to go to the phone's settings page for the badukai and there it says the app has requested no permissions, was just wondering if it lacked some permissions but it seems to not ask any. I haven't changed any config settings in badukAI if that could make the downloaded networks work. No idea what settings I should try there if that's the case. By the way how much stronger this 1.13.2 network is compared to the 20b and 10b internal networks? I guess if I go through this thread there is info of the strength of the internal 20b and 10b |
|
| Author: | akigo [ Thu Mar 10, 2022 2:14 pm ] |
| Post subject: | Re: New android app "BadukAI" |
floating wrote: By the way how much stronger this 1.13.2 network is compared to the 20b and 10b internal networks ? According to https://github.com/breakwa11/GoAIRatings the 20b network has around 5050 Elo at 1200 visits on the human scale (so already far stronger than the best humans). According to https://katagotraining.org/networks/ the 1.13.2 network is around 500 Elo stronger at equal playouts, so around 300 Elo stronger at equal time. Therefore the 20b network should be ok for most use cases. Both networks have the same structure, just different size, so if the smaller one runs and the bigger one does not, it could be a memory problem (especially if there is a lot of background activity on the phone). |
|
| Author: | floating [ Thu Mar 10, 2022 2:59 pm ] |
| Post subject: | Re: New android app "BadukAI" |
Cool I see thanks! The 20b will be just fine for me then. Anyway I still tried a bit. Adding here notes. This is a redmi note 7 with 6GB ram version. I'm running Lineage OS 17. I closed all other apps and from developer options closed couple running services that used over 300MB ram so it shows now system 2G, Apps ~400M and free 3.6G. Then started badukAI, it initialized the 40b from the assets/releases I had downloaded but still crash after playing a move. |
|
| Author: | go4thewin [ Thu Mar 10, 2022 3:34 pm ] |
| Post subject: | Re: New android app "BadukAI" |
if you want the strongest net, the 60b net in this link should work fine for you https://media.katagotraining.org/upload ... 017.bin.gz |
|
| Author: | floating [ Thu Mar 10, 2022 3:59 pm ] |
| Post subject: | Re: New android app "BadukAI" |
That one crashes already while loading. Once I press "load" I get KG is dead/crashed. In case it worked, I guess that one wouldn't require an additional .tflite file? |
|
| Author: | akigo [ Fri Mar 11, 2022 1:09 pm ] |
| Post subject: | Re: New android app "BadukAI" |
@floating To sum up, you get 20b optimized (file size 25 MB): works fine 40b optimized (file size 50 MB): initializes, but crashes when it starts "thinking" 60b original (file size 400 MB): doesn't even initialize This series strongly suggests memory problems. 3.6 GB may sound a lot, but in the area of neural networks, it's really not that much. You could try reducing the cache that KataGo uses: Change "nnCacheSizePowerOfTwo" from 20 to 10 in settings. I don't know which percentage of KataGo's total memory usage is caused by the cache, but it's worth a try ... |
|
| Author: | lovego [ Sat Mar 12, 2022 7:39 pm ] |
| Post subject: | Re: New android app "BadukAI" |
Graph search is merged to katago, it can lead +100 Elo improvement, can you update the engine soon? 
|
|
| Author: | akigo [ Thu Mar 17, 2022 1:55 pm ] |
| Post subject: | Re: New android app "BadukAI" |
I made a new version (1.14) available on play store https://play.google.com/store/apps/deta ... r.baduk_ai and github https://github.com/aki65/aki65.github.i ... ag/v1.14.0 with the following changes: - support NPU on recent Mediatek devices - upgrade KataGo (including graphsearch) - load sgf from clipboard - load sgf with arbitrary encoding |
|
| Page 21 of 27 | All times are UTC - 8 hours [ DST ] |
| Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |
|