Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM's Watson AI trumps humans in "Jeopardy!"

nytimes.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#31

Post by Simucal »

Qz wrote: post #24

"The only way to program a computer to do this type of mathematical reasoning might be to do precisely what Ferrucci doesn’t want to do -- sit down and slowly teach it about the world, one fact at a time." I'm wondering what the resistance to this is. Each and every one of us has been through exactly that process. Do we really think that we can create a knowledgeable mind out of whole cloth? It seems what we need to…

Each and everyone of us has been through this but we as humans are already "Strong" intelligent. We possess the ability to learn and form connections which is what makes it effective.

The resistance to the approach you mentioned comes when you you proceed to build up a giant scaffolding of predicate logic without there being any true "intelligence". This was tried in the 80's with the promise of Strong AI around the corner. We all know how that turned out.

Admittedly, these expert logic systems can make what sometimes seem like surprisingly intelligent inferences based on the rules you give it. However, they only know the facts you give them and they don't seem to possess the ability to "learn". It turns out intelligence isn't just a summation of facts, rules and hierarchies.

I personally think that our best chance of creating a Strong AI entity is through whole brain emulation. If we can mimic the neurons, synaptic connections, chemical interactions and other processes of the mind to a certain level of accuracy I think we could achieve some level of AI. Kurzweil is always talking about brain-scan resolution and how it is advancing at an exponential rate (as is all technological progress).

So, if we were one day able to scan a persons brain and emulate it at a reasonable speed in a detailed enough model I think we could have a reasonable "copy" of that persons persona. Alternatively, we may just model the processes without directly taking a scan of a human first. In this way, we may truly "teach" the AI as you suggest. It would start out like an infant with no ability to communicate coherently. We would provide it with stimuli and it would advance as a human does (although perhaps not at the same rate).

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#32

Post by Jun8 »

timwiseman wrote: post #8
jerf wrote: post #5

Earlier quoted context omitted.

'"Jeopardy!"'s answer snippets are not like normal questions that, say, a tax software would encounter, I think.' I'm not sure, but are you saying that you think it's easier to answer Jeopardy questions than tax questions? I wouldn't think so. The tighter you constrain the domain, the better the computer will do.

But in some ways, Jeopardy is much more constrained than tax questions. You know that the question will have a relatively short statement, the answer will be in the form of a question with probably no more than 5 words being actually relevant with the rest being used to make it into a question. Furthermore, you know that breadth of knowledge is generally more significant depth. Having a database of every nation's cap…

You're right, Jeopardy questions are much more contrained than standard, domain-based questions; however, as the article also points out, these constraints may be very hard for the computer to pick up. In fact, the main reason that Watson is slow compared to humans competing against it is precisely this, that it cannot effectively prune the search space in most cases. Look at how the answer to the Michael Jackson video is generated; the final answer is correct. The runner ups are also relevant but in a very weird sense, surely nothing that a human would come up with.

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#33

Post by alextp »

alextp wrote: post #15
timwiseman wrote: post #8

Earlier quoted context omitted.

But in some ways, Jeopardy is much more constrained than tax questions. You know that the question will have a relatively short statement, the answer will be in the form of a question with probably no more than 5 words being actually relevant with the rest being used to make it into a question. Furthermore, you know that breadth of knowledge is generally more significant depth. Having a database of every nation's cap…

Not only that but you can probably parse a jeopardy "answer" into a series of roughly independent clauses, and then try to predict classes that rank high in these clauses. For example, in the "answer" "This action flick starring Roy Scheider in a high-tech police helicopter was also briefly a TV series" you can get it right just by looking for things that correlate highly with "action flick", "Roy Schneider", "police…

But they must surely be doing something fancier than this naive-bayes-style model, otherwise they'd have no use for a roomful of supercomputers.

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#34

Post by DanielBMarkham »

I think the coming decade will see the evolution and widespread adoption of true question-answering machines. If we could combine that with auto-drive cars, it could mean an incredible leap forward in magnifying the human brain.

But auto-drive is going to take quite a bit longer, I think.

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#35

Post by csmeder »

jcl wrote: post #29
yanowitz wrote: post #18

How is this possible? I was very impressed but could guess at how it worked with the first several answers/questions listed, but this one blew my mind: “Classic candy bar that’s a female Supreme Court justice” — “What is Baby Ruth Ginsburg?” [Of course, Google now knows the answer :) ] Can someone explain how it can do that? Can it solve cryptics too (way harder than crosswords)? Amazing...

As albertni suggests, they have a series of heuristics, some of which do word matching and others which are specialized to common Jeopardy idioms -- such as the before-and-after clue. Each heuristic gives a list of candidate answers and probabilities, and Watson replies with the highest-probability answer, if its certainty is high enough. How does Watson "know" with high probability to apply the before-and-after heur…

"which are specialized to common Jeopardy idioms" Exactly, any new/novel idiom would probably easily stump Watson.

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#36

Post by elblanco »

Qz wrote: post #24

"The only way to program a computer to do this type of mathematical reasoning might be to do precisely what Ferrucci doesn’t want to do -- sit down and slowly teach it about the world, one fact at a time." I'm wondering what the resistance to this is. Each and every one of us has been through exactly that process. Do we really think that we can create a knowledgeable mind out of whole cloth? It seems what we need to…

http://en.wikipedia.org/wiki/Cyc

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#37

Post by Jun8 »

Qz wrote: post #24

"The only way to program a computer to do this type of mathematical reasoning might be to do precisely what Ferrucci doesn’t want to do -- sit down and slowly teach it about the world, one fact at a time." I'm wondering what the resistance to this is. Each and every one of us has been through exactly that process. Do we really think that we can create a knowledgeable mind out of whole cloth? It seems what we need to…

I completely disagree. The classical approach you mention has been tried many times, most famously by the Cyc Project (http://en.wikipedia.org/wiki/Cyc) without great results. AFAIK, nowadays the "no-model, pure statistical" approach is the norm

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#38

Post by zach »

alextp wrote: post #33
alextp wrote: post #15

Earlier quoted context omitted.

Not only that but you can probably parse a jeopardy "answer" into a series of roughly independent clauses, and then try to predict classes that rank high in these clauses. For example, in the "answer" "This action flick starring Roy Scheider in a high-tech police helicopter was also briefly a TV series" you can get it right just by looking for things that correlate highly with "action flick", "Roy Schneider", "police…

But they must surely be doing something fancier than this naive-bayes-style model, otherwise they'd have no use for a roomful of supercomputers.

Well, naïve Bayesian inference is supercomputer-level when you use it on a huge universe of data.

As Peter Norvig often points out, these kind of tasks are highly data dependent. The supercomputers are probably more used for data access as for raw computation. I can totally imagine Peter writing a forty-line Python app that runs on Google's infrastructure that does about as well.

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#39

Post by benkant »

gokhan wrote: post #13

Two days ago, for some reason I was thinking that we would see IBM dead in the following couple of years. You know, SGI, Sun, IBM... It appears that I might be clueless. Maybe they will revamp the company into the default search engine of the internet. Maybe they will own a cloud service for companies to automatic data mining.

IBM isn't going anywhere. For a start their consulting arms (Global Business|Technology Services) make up the bulk of their revenue. And that business is booming.

IBM has always been at the forefront of research. IIRC they have more patents than anyone.

Trust me, they're not going anywhere soon. In fact, I'd wager we'll see the end of Microsoft before we see the end of IBM.

Re: IBM's Watson AI trumps humans in "Jeopardy!"

#40

Post by zach »

csmeder wrote: post #35
jcl wrote: post #29

Earlier quoted context omitted.

As albertni suggests, they have a series of heuristics, some of which do word matching and others which are specialized to common Jeopardy idioms -- such as the before-and-after clue. Each heuristic gives a list of candidate answers and probabilities, and Watson replies with the highest-probability answer, if its certainty is high enough. How does Watson "know" with high probability to apply the before-and-after heur…

"which are specialized to common Jeopardy idioms" Exactly, any new/novel idiom would probably easily stump Watson.

That's probably better than the typical contestant who simply has categorical weak spots like sports or opera. Although not being able to grok audio or picture clues on top of that would be a real problem.

Every season has more clever, non-traditional categories to make Jeopardy! more playful. For example, it wouldn't be unusual to have a category like "Monopoly Colors," where responding to a Daily Double clue of "The $1 Bill" with "green" could be catastrophic.

This seems really fun, though. I'm very glad to see this kind of high-profile project that has the potential to rouse the curiosity of potential computer scientists.

Post reply on HN