Check the series of games between bigbadwolf 8d and zen19d 6d on July 11th. One game on three stones and six on two stones. Zen won the 3-stoner but bbw was 4-2 on two stones. Just a little counter example to you impression.
| Life In 19x19 http://www.lifein19x19.com/ |
|
| Bots really strong when receiving handicap? http://www.lifein19x19.com/viewtopic.php?f=18&t=6349 |
Page 1 of 1 |
| Author: | SpongeBob [ Fri Jul 20, 2012 5:14 am ] |
| Post subject: | Bots really strong when receiving handicap? |
When looking at bot-against-human games where the bot receives handicap stones according to the rank difference, it seems that the human player struggles hard and does not reach the 50% winning percentage. At least this is the impression I get when I look at some KGS games. When I think about it, I come up with this explanation: Bots are playing much more 'consistently' than humans, basically meaning they do not 'blunder' that often. When I play handicap with white against a human player, I would say that black could normally win these games if he pulls himself together and does not mess up someplace. These blunders from the black player occur less often when a bot is playing black. |
|
| Author: | ez4u [ Fri Jul 20, 2012 5:45 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
Check the series of games between bigbadwolf 8d and zen19d 6d on July 11th. One game on three stones and six on two stones. Zen won the 3-stoner but bbw was 4-2 on two stones. Just a little counter example to you impression.
|
|
| Author: | Mike Novack [ Fri Jul 20, 2012 5:57 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
SpongeBob wrote: When looking at bot-against-human games where the bot receives handicap stones according to the rank difference, it seems that the human player struggles hard and does not reach the 50% winning percentage. At least this is the impression I get when I look at some KGS games. Some games? Informal estimate? Doing statistics (correectly) isn't as easy as it might at first seem. If you have a background in it we could presume you knew how to take a proper sample from which you could have drawn a conclusion (that not 50%). But in that case you would have likely told us the "confidence level" to be attached to the conclusion (the confidence that your conclusion meant something, not pure chance). The point is that if you flip a fair coin 10 times the result will be something other than 5-5 mpre likely than not. Would you conclude from a result of 4-6 or 3-7 that the coin was weighted? |
|
| Author: | speedchase [ Fri Jul 20, 2012 7:43 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
ez4u wrote: Check the series of games between bigbadwolf 8d and zen19d 6d on July 11th. One game on three stones and six on two stones. Zen won the 3-stoner but bbw was 4-2 on two stones. Just a little counter example to you impression. I think this is more evidence of the theory that bots lose their strength when their weakness is studied. |
|
| Author: | ez4u [ Fri Jul 20, 2012 8:14 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
Is it known that bbw has studied the weaknesses of zen (or bots in general)? |
|
| Author: | topazg [ Fri Jul 20, 2012 8:17 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
ez4u wrote: Is it known that bbw has studied the weaknesses of zen (or bots in general)? I think the implication is that simply noticing habits of the program in the earlier games increased BBW's performance in the later ones. |
|
| Author: | ez4u [ Fri Jul 20, 2012 8:22 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
topazg wrote: ez4u wrote: Is it known that bbw has studied the weaknesses of zen (or bots in general)? I think the implication is that simply noticing habits of the program in the earlier games increased BBW's performance in the later ones. Hmmm.... You think the sequence 0 (3-stones) followed by 1,0,1,0,1,1 (2-stones) really shows that? 
|
|
| Author: | Mike Novack [ Fri Jul 20, 2012 10:00 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
Unfortunately (and in spite of my earlier comment) probably does believe that. What ez4u is saying (and I most certainly agree) is that if you see a sample of just 6 splitting 4-2 that does not justify any conclusion let alone the one implied. OK --- for six trials flipping a fair coin the chances are less than 1/3 for you to get 3 heads and three tails. Just under 1/2 for you to get either 4-2 or 2-4 (combined. It's ~1/5 might split as badly as 5-1 or 1-5 (combined) and ~1/30 might be as bad as all heads or all tails (combined). That's all just pure chance, not bias of the coin. And while the MCTS have some particular behaviors not quite proper to describe as weakness. More a matter of "style of play", loose, leaving positons unfinished with potential all over the board. Some humans play this way too. Now if you have not before encountered this style of play or encountered it only very rarely there may be a learning curve so you might improve after some experience. But that's less a matter of learning the program's weakness as eliminating one that you had (misjudging how to play against this style). |
|
| Author: | ez4u [ Fri Jul 20, 2012 4:56 pm ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
Mike Novack wrote: ... What ez4u is saying (and I most certainly agree) is that... "That is as well said as if I had said it myself." - Jonathan Swift. 1667-1745 Better actually 
|
|
| Author: | emeraldemon [ Sat Jul 21, 2012 10:32 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
Most modern Go engines try to find moves that maximizes the win percentage of playouts. They make no distinction between B+0.5 and B+10.5, only between B+ and W+ . While most humans understand that winning by more isn't the same as being more likely to win, I think we may not be quite as good at adjusting out play to be safe when we're ahead and dangerous when we're behind. In handicap games this translates to Bots being good at not taking risks against stronger players. |
|
| Author: | hyperpape [ Sat Jul 21, 2012 12:30 pm ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
It's not obvious, though, since at the beginning of the game, the bot is very far ahead, and so the engine may play very slack moves if it is just trying to maximize win probability. Thus, there's a lot of research into the question of whether you can use dynamic komi (i.e. pretending to give your opponent a large komi at the beginning of the game, and slowly reducing that as the game progresses) to get better results. I suspect your effect is real, Emeraldemon, but which effect predominates is unclear. |
|
| Author: | speedchase [ Sun Jul 22, 2012 6:37 pm ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
ez4u wrote: Hmmm.... You think the sequence 0 (3-stones) followed by 1,0,1,0,1,1 (2-stones) really shows that? Actually I misread the first post. sorry |
|
| Author: | ez4u [ Sun Jul 22, 2012 9:15 pm ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
emeraldemon wrote: Most modern Go engines try to find moves that maximizes the win percentage of playouts. They make no distinction between B+0.5 and B+10.5, only between B+ and W+ . While most humans understand that winning by more isn't the same as being more likely to win, I think we may not be quite as good at adjusting out play to be safe when we're ahead and dangerous when we're behind. In handicap games this translates to Bots being good at not taking risks against stronger players. I would think this would translate to bots giving away too much of their advantage early, when their lead is the greatest. In other words they are too willing to compromise on safe moves as long as their playouts show they are still ahead even if only by a minimal amount. |
|
| Author: | speedchase [ Wed Jul 25, 2012 1:48 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
ez4u wrote: I would think this would translate to bots giving away too much of their advantage early, when their lead is the greatest. In other words they are too willing to compromise on safe moves as long as their playouts show they are still ahead even if only by a minimal amount. I think they use adjusted komi to compensate |
|
| Author: | Mike Novack [ Wed Jul 25, 2012 4:32 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
ez4u wrote: [ I would think this would translate to bots giving away too much of their advantage early, when their lead is the greatest. In other words they are too willing to compromise on safe moves as long as their playouts show they are still ahead even if only by a minimal amount. You aren't looking at the consequences of what you have just said. If it is true that there are always safe moves that retain a small but positive advantage that eventually results in a won game. Not unlike a match sailboat race (race between two boats) where the one in the lead maneuvers not to increase its lead but to always remain between the boat behind and the next mark. Remember, the MCTS programs are not seeing how much the lead has increased or decreased form the next move but playing games out to the end and seeing who eventually won. Remember that it's up to the player behind to start taking chances (and the player ahead responding to that). |
|
| Author: | ez4u [ Wed Jul 25, 2012 5:12 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
Mike Novack wrote: ez4u wrote: [ I would think this would translate to bots giving away too much of their advantage early, when their lead is the greatest. In other words they are too willing to compromise on safe moves as long as their playouts show they are still ahead even if only by a minimal amount. You aren't looking at the consequences of what you have just said. If it is true that there are always safe moves that retain a small but positive advantage that eventually results in a won game. Not unlike a match sailboat race (race between two boats) where the one in the lead maneuvers not to increase its lead but to always remain between the boat behind and the next mark. Remember, the MCTS programs are not seeing how much the lead has increased or decreased form the next move but playing games out to the end and seeing who eventually won. Remember that it's up to the player behind to start taking chances (and the player ahead responding to that). I am assuming that the player giving the handicap is stronger than the bot, e.g. kgs 8d vs a 6d bot. Thus White will find superior moves during the course of the game. Or in other words, the bot's playouts will not hold up as predictions of the actual course of the game. Thus a safety first selection may result in giving away a necessary cushion. This may well be an incorrect understanding of the situation and I would be very interested in a more thorough and technically correct explanation. (hint, hint, Mike!) 
|
|
| Author: | hyperpape [ Wed Jul 25, 2012 5:52 am ] |
| Post subject: | Re: Bots really strong when receiving handicap? |
speedchase wrote: ez4u wrote: I would think this would translate to bots giving away too much of their advantage early, when their lead is the greatest. In other words they are too willing to compromise on safe moves as long as their playouts show they are still ahead even if only by a minimal amount. I think they use adjusted komi to compensate |
|
| Page 1 of 1 | All times are UTC - 8 hours [ DST ] |
| Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |
|