Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to build your own "Watson Jr." in your basement

ibm.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: How to build your own "Watson Jr." in your basement

#21

Post by sliverstorm »

aristus wrote: post #7
gojomo wrote: post #5

My hunch is that 90-99% of all Jeopardy questions can be answered with information in Wikipedia/Wiktionary, properly understood. So I'd start with Wikipedia: ~30GB uncompressed full article text. Break it into chunks; canonicalize phrasings to be more declarative, and include synonyms/hypernym/hyponym phrasings (via something like WordNet), so that various 'cluesy' ways of saying things still bring up the same candid…

"properly understood" is the whole point. That's the hard problem they are trying to solve.

Which is especially interesting, considering the other competitors are coming from exactly the opposite direction. They are playing the same game, but facing completely different challenges.

Re: How to build your own "Watson Jr." in your basement

#22

Post by joakin »

I hate when I enter a page Im interested in that makes an ajax call whenever I click something, since I read long texts clicking and selecting text.

Why man? Why? If you want to track clicks on links with javascript, dont trigger the ajax call when I do click in :not(a) ... -_-'

Re: How to build your own "Watson Jr." in your basement

#23

Post by maeon3 »

This is one of the great moments in the history of humanity, right up there with the first self-powered flying machine. North Carolina got a licence plate: "FIRST IN FLIGHT". Someone is going to get the credit for open ended question answering machine shortly. Who gets it?

The race is on, whoever creates the first reasonably good question-answering machine for demonstration will get their names etched into the sands of time for the next ten thousand years. Get to it!

This industry has the chance to be bigger than Google and Microsoft combined. Every person on the Earth will demand one of these. Those who won't have one will be at a remarkable disadvantage. This is going to turn into a trillion dollar industry.

Re: How to build your own "Watson Jr." in your basement

#24

Post by _delirium »

rudiger wrote: post #6
kirpekar wrote: post #4

Watson has become an unbelievable marketing tool for IBM.

Has become? From the beginning, Watson's purpose has been to advertise IBM's computers.

I believe they also sell some of the tech behind this in their business-intelligence products, as a kind of extended semantic version of information extraction / information retrieval.

Re: How to build your own "Watson Jr." in your basement

#25

Post by jokermatt999 »

knowledgesale wrote: post #14
gojomo wrote: post #13

Earlier quoted context omitted.

Sure, but it does put somewhat of a cap on the amount of reference material you need to import. And a fairly low cap in the tens of GB: Wikipedia/Wiktionary/WordNet/Freebase/J!Archive is probably enough. Beyond that, you want software/heuristics. You might find far more data helpful to initially create that software, but once it's created, the reference material to have at hand can come from a small set of sources.

there is a project on formalization of the human "common sense" with (partly) open source database. take a look: http://en.wikipedia.org/wiki/Cyc

For those who were intrigued by the story of Eurisko (that space fleet battle playing bot that completely crushed the competition and then was impossible to find decent info on) the developer of Eurisko, Douglas Lenat, is the one who started this program.

Re: How to build your own "Watson Jr." in your basement

#26

Post by pwnguin »

aristus wrote: post #7
gojomo wrote: post #5

My hunch is that 90-99% of all Jeopardy questions can be answered with information in Wikipedia/Wiktionary, properly understood. So I'd start with Wikipedia: ~30GB uncompressed full article text. Break it into chunks; canonicalize phrasings to be more declarative, and include synonyms/hypernym/hyponym phrasings (via something like WordNet), so that various 'cluesy' ways of saying things still bring up the same candid…

"properly understood" is the whole point. That's the hard problem they are trying to solve.

But properly understanding what? You can diagram the question all you want, but if you don't know the answer it's pointless, no? Wikipedia is just a source of data.

Given that Jeopardy focuses a lot on literature, I'd also throw in gutenburg project books. And probably a newspaper archive going way back, like the NYT.

Re: How to build your own "Watson Jr." in your basement

#27

Post by dailystatusrpt »

maeon3 wrote: post #23

This is one of the great moments in the history of humanity, right up there with the first self-powered flying machine. North Carolina got a licence plate: "FIRST IN FLIGHT". Someone is going to get the credit for open ended question answering machine shortly. Who gets it? The race is on, whoever creates the first reasonably good question-answering machine for demonstration will get their names etched into the sands…

open-ended question answering systems have been around for many years. there are stacks of research papers written about them, watson is an improvement.

Re: How to build your own "Watson Jr." in your basement

#28

Post by easp »

rudiger wrote: post #6
kirpekar wrote: post #4

Watson has become an unbelievable marketing tool for IBM.

Has become? From the beginning, Watson's purpose has been to advertise IBM's computers.

You sure its purpose isn't to reinstantiate the cryogenically frozen brain of IBM's founder? Maybe that's just a bonus.

Re: How to build your own "Watson Jr." in your basement

#29

Post by jsvine »

gojomo wrote: post #5

My hunch is that 90-99% of all Jeopardy questions can be answered with information in Wikipedia/Wiktionary, properly understood. So I'd start with Wikipedia: ~30GB uncompressed full article text. Break it into chunks; canonicalize phrasings to be more declarative, and include synonyms/hypernym/hyponym phrasings (via something like WordNet), so that various 'cluesy' ways of saying things still bring up the same candid…

For what it's worth, I scraped J-Archive.com and wrote a couple of articles for Slate Magazine about what I found. More for the purpose of learning about Jeopardy than learning how to win.

- The first piece: http://www.slate.com/id/2284678/

- The follow-up, in which I answer reader questions: http://www.slate.com/id/2287705/

Post reply on HN