Page 1 of 2
Bruce Wilcox's AI "passes" turing test
Posted: Fri Oct 29, 2010 11:40 pm
by Mef
Looks like Bruce Wilcox's chatbot Suzette won the Loebner prize this year for chatbots. As I understand it a judge spoke with a human and a chatbot for 25 minutes, then had to choose which was which:
http://www.newscientist.com/article/dn1 ... nline-news
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 12:59 am
by Li Kao
I think the result of such a test mainly depends on how hard the human is trying to prove that he is human. You'll be able to tell a decently intelligent human from a bot until a true AI is invented. And shortly thereafter we arrive at the Singularity.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 1:15 am
by kirkmc
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 1:24 am
by Li Kao
As I understand the article questions in other topics were allowed in the final. But the human test candidate might have tried to imitate a bot...
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 2:12 am
by daal
kirkmc wrote:What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
the article wrote: "In final rounds, judges can ask the bots anything they want."
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 2:22 am
by topazg
kirkmc wrote:What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
I have never found a bot hard to tell, I'm always surprised by the poor quality of questions that get cited. If the purpose is to tell humans and bots apart, asking standard questions is not the way to do it. Ask something like "What do you suggest I should feed to my elephant in Winter?" and take the answer into an interesting abstract philosophical debate in the merits of having elephants as domestic pets with increasing house prices... people handle the "what?" factor way way better than bots.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 4:53 am
by DrStraw
kirkmc wrote:What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
Based on those two I would probably have been judged to be a bot.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 5:23 am
by ethanb
topazg wrote:kirkmc wrote:What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
I have never found a bot hard to tell, I'm always surprised by the poor quality of questions that get cited. If the purpose is to tell humans and bots apart, asking standard questions is not the way to do it. Ask something like "What do you suggest I should feed to my elephant in Winter?" and take the answer into an interesting abstract philosophical debate in the merits of having elephants as domestic pets with increasing house prices... people handle the "what?" factor way way better than bots.
I read about this - I didn't realize it was Bruce Wilcox's AI though. Nice. If I recall correctly, the human was not a paragon of responsiveness, and while the bot didn't get pop-culture references, when pressed on them it got angry and then claimed to be getting bored with the conversation. So the judge decided the bot was probably the person - at least it was able to evince some kind of emotional reaction.
Honestly, the transcript reads more like the human failed the test. It would make a good Onion headline.
There was a comment on Slashdot I thought was pretty funny - something to the effect of a bot that simply repeated tired old memes would have a pretty good chance of being taken for a human who thinks he's being HILARIOUS doing this for a Turing test.
Judge: What do you think about the Sopranos?
MemeBot: IT'S OVER 9000! LOLOLLOL!
Wait, I just got a great idea! "YourMomBot" - just like ALICE but it makes Your Mom jokes using the sentence keywords.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Sat Oct 30, 2010 12:13 pm
by kirkmc
DrStraw wrote:kirkmc wrote:What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
Based on those two I would probably have been judged to be a bot.
No, you would have been able to say why you don't have any opinion on them.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Mon Nov 01, 2010 9:56 am
by wms
kirkmc wrote:DrStraw wrote:kirkmc wrote:What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
Based on those two I would probably have been judged to be a bot.
No, you would have been able to say why you don't have any opinion on them.
Even simple chatbots are stocked with phrases like "I don't pay much attention to stuff like that. I prefer knitting."
Re: Bruce Wilcox's AI "passes" turing test
Posted: Mon Nov 01, 2010 12:28 pm
by hyperpape
Instantly might be a bit much, but the right move for the judge is to press the issue. "Don't you feel left out when everyone else is watching sports?" or some such. It's a bit more difficult to program the chatbot with stock phrases like "Well, my friends all prefer knitting too" and "Yes, even my male friends. What, are you surprised?"
Re: Bruce Wilcox's AI "passes" turing test
Posted: Thu Nov 04, 2010 10:37 am
by Time
hyperpape wrote:Instantly might be a bit much, but the right move for the judge is to press the issue. "Don't you feel left out when everyone else is watching sports?" or some such. It's a bit more difficult to program the chatbot with stock phrases like "Well, my friends all prefer knitting too" and "Yes, even my male friends. What, are you surprised?"
If someone response to that question with something as complicated as "Well, my friends all prefer knitting too," then they're probably either a) very strange or b) a robot. I think someone could easily make ApatheticBot and have it response pretty much like a human to non-standard questions.
It can't be all that difficult to program a bot to reply with something like:
"What do you think about the world series?"
"I don't really follow the world series"
"Do you feel left out when everyone else is watching sports?"
"Not really"
This sort of response would work for any questions of the form "what do you think about X?" or "do you X?"
Re: Bruce Wilcox's AI "passes" turing test
Posted: Thu Nov 04, 2010 11:05 am
by emeraldemon
Time wrote:"What do you think about the world series?"
"I don't really follow the world series"
"Do you feel left out when everyone else is watching sports?"
"Not really"
This sort of response would work for any questions of the form "what do you think about X?" or "do you X?"
I think this would get noticed pretty fast:
"What do you think about being here?"
"I don't really follow being here"
Re: Bruce Wilcox's AI "passes" turing test
Posted: Thu Nov 04, 2010 11:21 am
by Stable
I'm not sure I understand you fully.
Re: Bruce Wilcox's AI "passes" turing test
Posted: Thu Nov 04, 2010 12:10 pm
by Marcus
Heh, this reminds me of my chat days, where I was constantly mistaken for a Bot because I used full words and punctuation.
