IBM's "Watson" finally ready for prime-time Jeopardy

I played a lot of College Bowl (and its local high-school equivalent) back in my younger days. And there were many running gags about the strategy. For example, if a question contained the words "Name this artist..." there was a strictly limited set of possible answers, defined roughly as "artists whose names appear in a typical high school art appreciation class". Similarly, "name this composer" is more likely to be cluing Beethoven or Bach or Mozart than Grieg or Saint-Saens, and almost certainly not anybody more obscure than that. And Grieg is not that obscure.

There were a lot of common themes: It was a good idea to memorize the names of all the Greek, Roman, and Norse gods; the names of all the types of clouds; the capitals of countries and the names of political leaders. That sort of thing.

In College Bowl (unlike Jeopardy) it is legal to press the buzzer and interrupt the questioner at any point. The questioner just stops reading until you've taken your guess. And so every College Bowl geek knows a story like this: Someone accidentally presses the buzzer before the person reading the questions has said more than one or two words. So, with nothing else to do but guess, the player shouts something out. And that guess is correct.

I've never seen a zero-word guess work in person, but I've seen a few two and three-word guesses succeed. Given just a hint of the topic ("this scientist...") the odds are surprisingly good. The search space is just not that large.

One reason why trivia is easy is that questions that require even the smallest amount of actual thought tend to be too ambiguous for good trivia. Trivia questions are generally read, answered, and judged by people who have no actual expertise in their subject matter, so it's important that the answers be very clear and unambiguous. I used to dread the physics questions, because I was a student physicist, and I understood the questions too well. So it was sometimes difficult to avoid giving answers that were technically correct, but that did not match the one on the card.

There is an archive of past Jeopardy questions here :

http://www.j-archive.com/

Try choosing a question and typing the category + the clue verbatim into Google. I've tried this a few times and in most cases the correct answer was in the top couple of sites (usually in the summary text on the Google search page).

Of course there's still the problem of actually extracting the answer from the page and presenting it in the proper form.

It's all about having enough breadth that the other bits of information presented can help whittle down answers by way of association and relationships. Good trivia players rely quite a bit on intuition.

Ken Jennings mentioned (in his book) how Jeopardy! often has questions about rivers, so it was a fairly simple matter to just memorize river names and some basic associations with which to answer questions.

It seems like Watson is mostly a really sophisticated algorithm on top of lots of brute force (2880 processors?!). Though I guess it could also be said that humans are just ugly bags of mostly water.

That said: in something like medicine (differential diagnoses), such a system could also be extremely useful, given the frequent occurrence of a fairly limited number of "answers".