Life In 19x19 http://www.lifein19x19.com/ |
|
AI makes humans better - says University research http://www.lifein19x19.com/viewtopic.php?f=10&t=19084 |
Page 1 of 1 |
Author: | Knotwilg [ Tue Mar 14, 2023 12:14 pm ] |
Post subject: | AI makes humans better - says University research |
translated from an article in our national press "Decision Quality Index" The scientists, led by Minkyu Shin of Hong Kong's City University, collected data on 5.8 million moves made by professional Go players between 1950 and 2021. Then, with the help of AI trained in Go, they developed a benchmark, the " Decision Quality Index" (DSI). It assesses the quality of a move. A move was considered innovative if it had never been performed before in combination with the previous moves. The scientists found a significant difference between the period before and after AI. Between 1950 and 2015, the quality of the game barely improved, with an average DSI between -0.2 and 0.2. That means that in some years the quality even deteriorated. 2016, the year of AlphaGo's famous profit, was a turning point. The DSI started to rise. Between 2018 and 2021, it averaged 0.7 each year. Another example of the improvement in quality: in 2016, 62 percent of the games showed innovative strategies, in 2018 this was 88 percent. "Decision making process improved by AI" Part of it was about moves and strategies people had learned from Go-playing artificial intelligence. But they also came up with new moves that had never been seen before. They were not only new, but also of a high level, the scientists saw. "Our findings suggest that the development of superhuman AI may have prompted human players to deviate from traditional strategies and test new moves, which in turn may have improved their decision-making process," the scientists write in their conclusions. https://osf.io/xpf3q/#! |
Author: | jeromie [ Wed Mar 15, 2023 7:52 pm ] |
Post subject: | Re: AI makes humans better - says University research |
I saw that article, too. It’s an interesting result, and I’m curious about the answers to some of the questions the authors raise in the discussion section. What is the causal link between the increase in quality AND novelty and the rise of strong AI? Part of the growth in quality and novelty is surely the ability to get new ideas from a strong (AI) player. Top level players are really good at learning from game records. But I would guess that another significant factor is the ability for players to evaluate their moves after the game. The knowledge that you can get an answer on whether your idea was sound——besides the game itself, which can sometimes be deceptive if there was a problem in your execution or the opponent’s resistance——gives a player the freedom to try out new moves more frequently, knowing they can learn from the results. Do you have any guesses as to how players are learning from AI? I know that you’ve strived to do so in your own play. |
Author: | Knotwilg [ Thu Mar 16, 2023 1:24 am ] |
Post subject: | Re: AI makes humans better - says University research |
jeromie wrote: Do you have any guesses as to how players are learning from AI? I know that you’ve strived to do so in your own play. Your hypothesis is that players are more likely to experiment because they can more easily evaluate afterwards. I think it's a valid idea but it is not one that plays a role in my case. What I drew from the article is that AI has disproven or colored a lot of conventional wisdom and established patterns. So, on one hand, players will "follow" the new wisdom and patterns learned from AI, which is actually not a change in behavior, only a change of source. On the other hand, they may be more critical of any acquired wisdom and patterns, and be more on the lookout for novelty/experimentation. In my case it's about reinforced learning. IF I play better today - which I do feel and which is slowly noticeable in my online rankings - it is because I have acquired new heuristics which correct bad habits I used to have. Examples include: "avoid slow connections", "avoid heavy cuts", "defend while making territory", "surround/pressure large groups (in sente)" ... You could ask why AI does what human knowledge couldn't. The answer is 1) Personalized AI analysis reveals patterns in an individual's (bad)play 2) Frequent: AI analysis has a very high frequency. Before AI we didn't have access to pro level knowledge, immediately, for free and unlimited in time. The personalized learning at high frequency leads to more awareness and critical evaluation of my own moves and better ideas for alternatives. I can't tell whether that is true at the professional level. The pro world is a world apart. For amateurs, I assume my case can apply to many others. As a recent example, here's a good move I played: "To pincer or back off" is a classic decision we have to make. For the last decades, there has been a tendency to back off more often, or always, because the pincer is influence oriented and influence based play was "going out of fashion". Of course, professionals will always have known when a pincer is better, but for me as an amateur it was not so easy to assess. In this case, the argument is clear: the left side has a solid, low group extending to half of the board, even if there's some cutting aji. The lower side is open. Still, frequent AI evaluation has reinforced my understanding, that even if backing off is the common answer and usually better for making territory, this is a situation where building influence at the bigger side is better. |
Author: | gowan [ Thu Mar 16, 2023 9:56 am ] |
Post subject: | Re: AI makes humans better - says University research |
I'm not so sure that the AI players have "disproven" the old ways, mostly because I don't believe that the AIs play perfectly and there could be flaws in their "thinking". It was a failure of programming that led to the now well known performance of an amateur player winning multiple games against "super human" AI. AI go is not based on a conceptual framework understanding so it is difficult to judge the performance of the AIs. |
Author: | Knotwilg [ Thu Mar 16, 2023 4:34 pm ] |
Post subject: | Re: AI makes humans better - says University research |
gowan wrote: I'm not so sure that the AI players have "disproven" the old ways, mostly because I don't believe that the AIs play perfectly and there could be flaws in their "thinking". It was a failure of programming that led to the now well known performance of an amateur player winning multiple games against "super human" AI. AI go is not based on a conceptual framework understanding so it is difficult to judge the performance of the AIs. AlphaGo - pros 64 - 0 Could be sheer luck but I don't think so. I haven't dived into that particular amateur defeating AI story. I HAVE seen nonsensical moves by AI, especially when the game is almost over and nonsensical moves make just as little of a difference as normal moves, or when the difference is so high that it doesn't know how to differentiate at all. Despite it being a neural network I'm still cautious of calling what they do "thinking" let alone "reasoning". But they do come up with novel patterns and they have encouraged humans to increase their experimentation. That's what academic research says. I'm not inclined to argue with that. I find it encouraging that an amateur still finds a way through that neural network - says something about human capacity to find loopholes to exploit - but it doesn't shake my belief AI has transformed modern Go. |
Author: | PC_Screen [ Sun Mar 19, 2023 3:37 pm ] |
Post subject: | Re: AI makes humans better - says University research |
Well, to be precise, it wasn't a human who found the loophole but an adversarial AI trained against a frozen target (KataGo in this case). It by no means takes away from the AI's strength in a normal setting. Leela for example was known to be weak against ladders but that doesn't mean it would still be weak outside of that scenario. Katago is currently weak against cyclic groups possibly due to overcounting liberties but in a game where you don't know your opponent it is situational at best which is precisely why KataGo hasn't learned how to deal with it yet. Recent networks are being exposed to the exploit in training in very small amounts and are making some progress in fixing it |
Author: | jeromie [ Sun Mar 19, 2023 4:03 pm ] |
Post subject: | Re: AI makes humans better - says University research |
I think the emphasis on personalized and frequent analysis actually supports the idea that part of the rise in novelty is the ability for players to get feedback on moves that they have played. It’s not that I think that there is a conscious decision to try out novel moves in a serious game (which is surely where our game records come from) because it can be evaluated later. But I do think that the access to feedback will encourage experimentation in practice sessions, which will be reflected in games. (This is not to say that post game evaluation doesn’t have any impact at all.) Here’s my theoretical framework for how access to AI may not only allow for better moves, but allow for increased novelty in play. I also used this as an excuse to form an alliterative list, because I like playing with words almost as much as I like playing go. ![]()
I’m certainly not confident that this adequately explains the results of the paper! I just find it fun to think about how we think about the game, and how our methods of gaining knowledge show up on the goban. |
Author: | Kirby [ Sun Mar 19, 2023 6:50 pm ] |
Post subject: | Re: AI makes humans better - says University research |
gowan wrote: I'm not so sure that the AI players have "disproven" the old ways, mostly because I don't believe that the AIs play perfectly and there could be flaws in their "thinking". It was a failure of programming that led to the now well known performance of an amateur player winning multiple games against "super human" AI. AI go is not based on a conceptual framework understanding so it is difficult to judge the performance of the AIs. I would agree that AI players have not "disproven" old ways in a strict sense ("prove" is a strong word). However, they have provided very strong evidence for better alternatives. Judging the performance of an AI can at least be done empirically by having it play games against strong opponents. Even before AI, human performance has been judged in the same way - through competition. |
Page 1 of 1 | All times are UTC - 8 hours [ DST ] |
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |