Life In 19x19
http://www.lifein19x19.com/

Bruce Wilcox's AI "passes" turing test
http://www.lifein19x19.com/viewtopic.php?f=8&t=2267		 Page 1 of 2
Author:  Mef [ Fri Oct 29, 2010 11:40 pm ]
Post subject:  Bruce Wilcox's AI "passes" turing test
Looks like Bruce Wilcox's chatbot Suzette won the Loebner prize this year for chatbots. As I understand it a judge spoke with a human and a chatbot for 25 minutes, then had to choose which was which:

http://www.newscientist.com/article/dn1 ... nline-news
Author:  Li Kao [ Sat Oct 30, 2010 12:59 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
I think the result of such a test mainly depends on how hard the human is trying to prove that he is human. You'll be able to tell a decently intelligent human from a bot until a true AI is invented. And shortly thereafter we arrive at the Singularity.
Author:  kirkmc [ Sat Oct 30, 2010 1:15 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.
Author:  Li Kao [ Sat Oct 30, 2010 1:24 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
As I understand the article questions in other topics were allowed in the final. But the human test candidate might have tried to imitate a bot...
Author:  daal [ Sat Oct 30, 2010 2:12 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.


the article wrote:
"In final rounds, judges can ask the bots anything they want."
Author:  topazg [ Sat Oct 30, 2010 2:22 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.


I have never found a bot hard to tell, I'm always surprised by the poor quality of questions that get cited. If the purpose is to tell humans and bots apart, asking standard questions is not the way to do it. Ask something like "What do you suggest I should feed to my elephant in Winter?" and take the answer into an interesting abstract philosophical debate in the merits of having elephants as domestic pets with increasing house prices... people handle the "what?" factor way way better than bots.
Author:  DrStraw [ Sat Oct 30, 2010 4:53 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.


Based on those two I would probably have been judged to be a bot.
Author:  ethanb [ Sat Oct 30, 2010 5:23 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
topazg wrote:
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.


I have never found a bot hard to tell, I'm always surprised by the poor quality of questions that get cited. If the purpose is to tell humans and bots apart, asking standard questions is not the way to do it. Ask something like "What do you suggest I should feed to my elephant in Winter?" and take the answer into an interesting abstract philosophical debate in the merits of having elephants as domestic pets with increasing house prices... people handle the "what?" factor way way better than bots.


I read about this - I didn't realize it was Bruce Wilcox's AI though. Nice. If I recall correctly, the human was not a paragon of responsiveness, and while the bot didn't get pop-culture references, when pressed on them it got angry and then claimed to be getting bored with the conversation. So the judge decided the bot was probably the person - at least it was able to evince some kind of emotional reaction.

Honestly, the transcript reads more like the human failed the test. It would make a good Onion headline. :)

There was a comment on Slashdot I thought was pretty funny - something to the effect of a bot that simply repeated tired old memes would have a pretty good chance of being taken for a human who thinks he's being HILARIOUS doing this for a Turing test.

Judge: What do you think about the Sopranos?
MemeBot: IT'S OVER 9000! LOLOLLOL!

Wait, I just got a great idea! "YourMomBot" - just like ALICE but it makes Your Mom jokes using the sentence keywords.
Author:  kirkmc [ Sat Oct 30, 2010 12:13 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
DrStraw wrote:
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.


Based on those two I would probably have been judged to be a bot.


No, you would have been able to say why you don't have any opinion on them.
Author:  wms [ Mon Nov 01, 2010 9:56 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
kirkmc wrote:
DrStraw wrote:
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.


Based on those two I would probably have been judged to be a bot.


No, you would have been able to say why you don't have any opinion on them.
Even simple chatbots are stocked with phrases like "I don't pay much attention to stuff like that. I prefer knitting."
Author:  hyperpape [ Mon Nov 01, 2010 12:28 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
Instantly might be a bit much, but the right move for the judge is to press the issue. "Don't you feel left out when everyone else is watching sports?" or some such. It's a bit more difficult to program the chatbot with stock phrases like "Well, my friends all prefer knitting too" and "Yes, even my male friends. What, are you surprised?"
Author:  Time [ Thu Nov 04, 2010 10:37 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
hyperpape wrote:
Instantly might be a bit much, but the right move for the judge is to press the issue. "Don't you feel left out when everyone else is watching sports?" or some such. It's a bit more difficult to program the chatbot with stock phrases like "Well, my friends all prefer knitting too" and "Yes, even my male friends. What, are you surprised?"


If someone response to that question with something as complicated as "Well, my friends all prefer knitting too," then they're probably either a) very strange or b) a robot. I think someone could easily make ApatheticBot and have it response pretty much like a human to non-standard questions.

It can't be all that difficult to program a bot to reply with something like:

"What do you think about the world series?"
"I don't really follow the world series"
"Do you feel left out when everyone else is watching sports?"
"Not really"

This sort of response would work for any questions of the form "what do you think about X?" or "do you X?"
Author:  emeraldemon [ Thu Nov 04, 2010 11:05 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
Time wrote:
"What do you think about the world series?"
"I don't really follow the world series"
"Do you feel left out when everyone else is watching sports?"
"Not really"

This sort of response would work for any questions of the form "what do you think about X?" or "do you X?"

I think this would get noticed pretty fast:

"What do you think about being here?"
"I don't really follow being here"
Author:  Stable [ Thu Nov 04, 2010 11:21 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
I'm not sure I understand you fully.
Author:  Marcus [ Thu Nov 04, 2010 12:10 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
Heh, this reminds me of my chat days, where I was constantly mistaken for a Bot because I used full words and punctuation. :blackeye:
Author:  flOvermind [ Thu Nov 04, 2010 12:56 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
http://xkcd.com/329/
Author:  Time [ Sat Nov 06, 2010 12:33 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
emeraldemon wrote:
Time wrote:
"What do you think about the world series?"
"I don't really follow the world series"
"Do you feel left out when everyone else is watching sports?"
"Not really"

This sort of response would work for any questions of the form "what do you think about X?" or "do you X?"

I think this would get noticed pretty fast:

"What do you think about being here?"
"I don't really follow being here"


I mean, do you really expect me to come up with a solution to this in 5 lines of explanation?

I'm pretty sure if I was programming and studying AI for a living I could find a solution. As for this particular example, I guess you should preface X by "the." But do we really have to nit it up this hard? The point is basically that if you're studying and programming AI for a living, you can probably fill most holes that are easy to exploit, since you're obviously going to be testing and thinking about those first.
Author:  willemien [ Sat Nov 06, 2010 1:11 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
kirkmc wrote:
What a bogus test. You can only ask about certain topics. Of course it's eventually easy to get there. Ask the bot what it thinks of the World Series or the Sopranos and it fails instantly.

not for me i don't know anything about any of the two

(or is that enough proof that i am Human? :-?
Author:  hyperpape [ Sat Nov 06, 2010 4:27 pm ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
@time Why assume that you can fill up the holes that are easy to exploit? Human languages allow an infinite number of meaningful sentences, of which an average human can process a gobsmackingly large number. Why think any solution based on responding to isolated tricky questions can work, without producing something approximating intelligence? That's the primary reason the contest is interesting--simple strategies of the type that are floated here are thought to be unpromising.
Author:  flOvermind [ Sun Nov 07, 2010 10:20 am ]
Post subject:  Re: Bruce Wilcox's AI "passes" turing test
Time wrote:
I'm pretty sure if I was programming and studying AI for a living I could find a solution. As for this particular example, I guess you should preface X by "the." But do we really have to nit it up this hard? The point is basically that if you're studying and programming AI for a living, you can probably fill most holes that are easy to exploit, since you're obviously going to be testing and thinking about those first.


http://xkcd.com/793/
Page 1 of 2 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/