Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM's Watson starts on Jeopardy Tonight

jeopardy.com

Post reply on HN
Print view View original

91–100 of 154 posts

Re: IBM's Watson starts on Jeopardy Tonight

#91

Post by ryanc »

brown9-2 wrote: post #63
jarin wrote: post #61

Earlier quoted context omitted.

Since there are a lot of pun-based categories on Jeopardy, I can understand why they would have problems with it.

Exactly - the category label probably is very little help in narrowing down possible answers to a text-processing algorithm.

This is a significant disadvantage for some categories. For example one category from Friday was "WARE"-ING with each question containing "ware" in the response.

After typing out this comment it occurs to me that if you carefully consider each answer there is only only logical question for all the answers in this category. Although I don't have any specific examples, there are times when a question is ruled incorrect because of the category name (e.g The category called for a specific number of letters in the response).

Re: IBM's Watson starts on Jeopardy Tonight

#92

Post by rkudeshi »

kenjackson wrote: post #9
umjames wrote: post #4

Much of success on Jeopardy is not just deciphering clues in the answers, but your timing on ringing in to give the question. I'd imagine a machine could get really good at getting the timing down. Does Jeopardy have a way of varying who rings in first to keep things more fair?

Something I wrote from a previous thread. The one piece of info that wasn't in this comment is that Watson can NOT anticipate when Trebek is about to finish the question. Watson must wait until it gets a signal that the buzzer is now available, and only then can it begin the process to physically depress the button. Remember that Watson also has to depress a physical button (the same buzzer everyone else uses). The e…

Interesting fact: during Ken Jennings' original 74-game winning streak, he felt it was an unfair advantage he had so much experience with the buzzers. He asked the producers to allow the other contestants (competing for the first time each episode) time to practice with the buzzers, which they did.

Re: IBM's Watson starts on Jeopardy Tonight

#93

Post by jhamburger »

jarin wrote: post #61
icefox wrote: post #47

Earlier quoted context omitted.

The real question I have is according to that episode Watson does not take into account the category only the 'answer' and the already shown questions. It seems _really_ odd that they would ignore this bit of information.

Since there are a lot of pun-based categories on Jeopardy, I can understand why they would have problems with it.

They could at least factor it in selectively, in obvious situations. For example, all categories in jeopardy that contain a quoted string include that string in all of the answers. Wouldn't be too hard to use that information.

edit: furthermore, sometimes it's impossible to narrow down the answer if you don't understand the category. For example, if the category is '"C" you later' or something, there might be like 8 possible synonyms that fit, but only one starts with a "C".

Re: IBM's Watson starts on Jeopardy Tonight

#94

Post by kenjackson »

icefox wrote: post #84
brown9-2 wrote: post #63

Earlier quoted context omitted.

Exactly - the category label probably is very little help in narrowing down possible answers to a text-processing algorithm.

I would have to disagree. Every bit of information helps. I was even surprised that they didn't implement a voice to text system so that he could learn what the other contestable guessed incorrectly and utilize that information if only to not guess the same thing on his turn.

They have added this. This was actually a problem in earlier versions where it would sometimes guess the exact same wrong answer as a previous contestant.

Re: IBM's Watson starts on Jeopardy Tonight

#96

Post by random42 »

kenjackson wrote: post #28
kj12345 wrote: post #13

One thing I'm interested in is any skew in the questions from normal. In particular I hope they ask linguistically tricky questions where you can't even figure out what's being asked at first. I felt like they went a bit easy on that front in the preview round: http://www.youtube.com/watch?v=WFR3lOm_xhE

I doubt they are skewing the questions. That kind of defeats the point. I suspect Jeopardy is constantly adding to a pool of questions. My hope is that they simply grab from that pool just like any other episode. Note, the hardest questions for Watson are short questions -- the reduced time hurts Watson more than most human competitors.

Also, smaller questions provide, much lesser context for Watson to work with.

Re: IBM's Watson starts on Jeopardy Tonight

#97

Post by yumraj »

Charuru wrote: post #95

Shouldn't IBM start their own search engine if they have algorithms like this? Or conversely, maybe Google should buy them?

Or conversely, maybe Google should buy them?

I wish you had checked the market cap of IBM and Google before suggesting that :). IBM > GOOG

Re: IBM's Watson starts on Jeopardy Tonight

#98

Post by osipov »

Charuru wrote: post #95

Shouldn't IBM start their own search engine if they have algorithms like this? Or conversely, maybe Google should buy them?

Looks like Google and IBM are evenly matched in terms of market cap so a buy out is unlikely http://www.wolframalpha.com/input/?i=IBM+GOOG+market+cap

QA technology is very different than search because the former makes strong assumptions about quality of the underlying data. Google's search is much better at filtering out and dealing with low quality content (e.g. blogs) and spam.

Re: IBM's Watson starts on Jeopardy Tonight

#99

Post by jleader »

alanfalcon wrote: post #35
jimmyjim wrote: post #32

Earlier quoted context omitted.

I would think as soon as it computes an answer. Of course there is a penalty for getting an answer wrong, so it can't be pressing the button all the time. But the PBS documentary I think hinted that it will be a sent a signal to know what amount to wait before reading aloud an answer. That said, I wish the engineers would have implemented OCR technology into Watson so it could see the TV screen, decipher the text que…

What he means is that you aren't permitted to buzz in on Jeopardy until the host has completed reading the question and the buzzers are activated. Perhaps Watson is spamming the 'answer' button from the moment he knows the answer, or perhaps they let him cheat by sending him a signal when the buzzers become activated. Given that the process of buzzing in to answer a question is a large part of the strategy of the sho…

I'd been told by a competitor on the show that timing the button press correctly was an underappreciated part of the game. If you press the button before you're allowed to (before Mr. Trebek finishes reading the question) your button is locked out for long enough (maybe a second?) that you're unlikely to get a second chance at that question. So I assume Watson wouldn't be allowed to "spam" the button.

Re: IBM's Watson starts on Jeopardy Tonight

#100

Post by ugh »

SMrF wrote: post #81
lrm242 wrote: post #40

Earlier quoted context omitted.

No, it is fed the correct answer by text once the question is completed. They said "listen" in the documentary, but later, in passing, specified what they meant by that. In particular, the demonstrated the active learning when Watson failed to understand a category required dates as answers. After 4 it learned that dates were the type of answer required and got the last question correct. Again, the feeding of the cor…

So what translates the speech to text? I guess I just assumed they used software but perhaps there is a human doing this. I probably just missed when they mentioned this.

The correct answers are already known. Watson presumably gets the same answer the host has in front of him.

Someone would have to transcribe what the other contestants are saying only if Watson would also get their incorrect answers which doesn’t seem to be the case.

Post reply on HN