Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Day 1: IBM Watson Ties for first on Jeopardy

venturebeat.com

Post reply on HN
Print view View original

71–80 of 99 posts

Re: Day 1: IBM Watson Ties for first on Jeopardy

#71

Post by ugh »

jluxenberg wrote: post #48

Does anyone know why Watson was unable to buzz in immediately in some cases when he had 97% confidence in his answer? Seems odd that a computer would not be able to beat the human opponents at hitting the buzzer. (It's possible that he didn't have the answer until after the human opponents, but that seems unlikely.)

Watson needs a few seconds to calculate the answer and will only hit the buzzer when it’s finished calculating. Its opponents can buzz in even if they are not completely sure that they have an answer (or they simply might be faster than Watson). It’s important to note that it is only possible to buzz in after the question has been read out. (Someone behind the stage flips a switch or something like that. Lights indicate that the buzzers are open for the humans, they are locked out for a few hundred milliseconds when they press the buzzer too early, Watson never buzzes in prematurely.)

I would like to know whether it is possible to beat Watson to the buzzer even if you and Watson both already know the answer. Watson has probably better reaction times but humans can anticipate when the host is finished reading. That probably depends highly on the consistency of the person behind the stage flipping the switch and opening the buzzers.

Here are some stats from the first round: Watson was the first to buzz in 16 times (with two wrong answers). It was above its buzzing threshold and didn’t manage to buzz in seven times. (It would have been wrong three times if it had made it to the buzzer first.) It was below its buzzing threshold six times. (The remaining one is the daily double which Watson got correct.)

This needs to be in a table. Numbers in brackets indicate wrong or potentially wrong answers:

  Confident & first:  14+(2)
  Confident & beaten:  4+(3)
  Not confident:       6
  Daily Double:        1
Edit: Some think that Watson’s (probably) superior reaction time gives it an unfair edge. I don’t really agree because fast reactions are simply a part of the game but I can sort of see the point that it’s not really about reaction times. We already know that computers can be better than humans when it comes to those.

I propose the following modification: One of the contestants who manages to buzz in in the first 100ms (the human reaction time) is randomly selected, buzzing in after those 100ms works as usual. Contestants are also no longer locked out for buzzing in too early.

Re: Day 1: IBM Watson Ties for first on Jeopardy

#72

Post by fuzzythinker »

chaosmachine wrote: post #2

After not being able to find a suitable online stream of this event (I don't have cable), I gave up and built a rudimentary antenna for my TV. Basically, I just wound a bunch of copper wire around a piece of cardboard and stuck it in the window. The other end of the wire went into the cable port on the back of my TV. Surprisingly, I was able to pick up the game in crystal clear, digital 1080p!

The non-realtime/non-hacking way: http://www.youtube.com/watch?v=ZLdkJpAtt1I (part1, part2 link on screen)

Re: Day 1: IBM Watson Ties for first on Jeopardy

#73

Post by chaosmachine »

Swizec wrote: post #3
chaosmachine wrote: post #2

After not being able to find a suitable online stream of this event (I don't have cable), I gave up and built a rudimentary antenna for my TV. Basically, I just wound a bunch of copper wire around a piece of cardboard and stuck it in the window. The other end of the wire went into the cable port on the back of my TV. Surprisingly, I was able to pick up the game in crystal clear, digital 1080p!

That's pretty impressive, mind sharing a photo with us? I just want to understand what this level of awesome looks like.

Thanks, I just wrote it up and took some pictures:

http://news.ycombinator.com/item?id=2222827

Re: Day 1: IBM Watson Ties for first on Jeopardy

#74

Post by chaosmachine »

jordanlyall wrote: post #64
chaosmachine wrote: post #2

After not being able to find a suitable online stream of this event (I don't have cable), I gave up and built a rudimentary antenna for my TV. Basically, I just wound a bunch of copper wire around a piece of cardboard and stuck it in the window. The other end of the wire went into the cable port on the back of my TV. Surprisingly, I was able to pick up the game in crystal clear, digital 1080p!

It may have been crystal clear, but it wasn't in 1080p. It was either 1080i or 720p. 1080p over-the-air broadcasts still do not exist.

You're right, my mistake, it was 1080i.

Re: Day 1: IBM Watson Ties for first on Jeopardy

#75

Post by ubercore »

dimatura wrote: post #54
ubercore wrote: post #44

I'd like to see the next version take the avatar concept further. They went half way by making it physically buzz in. Go the rest of the way, and make it only receive input from the avatar. Add a computer vision and speech recognition component, and make it read the board and listen to Trebek like the human players. Then, make it mobile, and put the avatar in the actual Jeopardy! studio. EDIT: Clarified "human" playe…

According the designers they considered it, but finally decided to concentrate on the question answering. OCR for this situation could probably be done in fast and reliable way, so it wouldn't make much of a difference. Speech recognition would be harder and less reliable, but redundant most of the time (though it presumably could have helped for the 1920s question).

Understandable, and I agree that they definitely tackled the meat of the problem. I think there could be some interesting problems in the computer vision aspect of it, though. Seeing and interpreting the board is subtly different than a straight OCR problem.

Maybe I'm overstating the difference, though.

Re: Day 1: IBM Watson Ties for first on Jeopardy

#76

Post by nyellin »

ckwalsh wrote: post #65
nyellin wrote: post #13

Does anyone want to create a community site for the contest? I am looking for another Django developer who can participate in an all-night code sprint. The site would: 1. Show an online score board. 2. Track Watson's performance (like shkb is doing on Google Docs) 3. Aggregate pictures, videos, and written analysis of the event

I don't have the time to build a site at the moment (curse you schoolwork!), but if you'd like to help maintain/update the docs I'd love the help.

Thank you for the offer. I did some more research and I found a few good resources for normal Jeopardy games. I have less of an itch to scratch right now, so I think I will just hack on my main project tonight.

Re: Day 1: IBM Watson Ties for first on Jeopardy

#77

Post by foresterh »

shkb wrote: post #11
cryptoz wrote: post #10

I couldn't have wished for a more exciting opening to this game. Watson just shot right off into the lead like there was no tomorrow, jetting ahead by thousands! I was blown away. Then he slowly started to crumble a bit and his answers were faulty more and more. But his strategy is amazing! He hops from category to category, while most humans pick a category and stick with it. I wonder if IBM has him do that purely b…

I suspect the hopping categories was to try and buy time to calculate what sort of questions would be asked in each. But it's first move of getting into the lead (and the daily double) I would say is a combination of good strategy and PSYOP.

It looked like he went to the 4th row to find the daily double, then just went through all the cheapest questions, as he was most likely to answer those. He finished all of the $200s, then went to the $400s, etc. I don't think it was to trip up his opponents, I think it was to have a better chance at answering.

Re: Day 1: IBM Watson Ties for first on Jeopardy

#79

Post by bitwize »

hallman76 wrote: post #60
AgentConundrum wrote: post #12

Earlier quoted context omitted.

He hops from category to category, while most humans pick a category and stick with it. I noticed the same thing, but I really hoped it was unintentional. It doesn't feel like it's a fair fight otherwise, since it means he's exploiting a weakness intrinsic to his competitors. The whole point of this competition is to show that a computer can compete and win at a human level. If it weren't, then why force him to physi…

It doesn't feel like it's a fair fight otherwise, since it means he's exploiting a weakness intrinsic to his competitors Late in the first round you can see Jennings change his strategy a bit -- he starts ringing in before he knows the answer. Jennings is exploiting the fact that Watson won't ring in until it has confidence in an answer.

That's awesome. He can make that strategy pay off because he's Ken freaking Jennings. Usually the money penalty for buzzing in but answering incorrectly (or not at all) would dissuade a contestant from buzzing in before actually knowing the answer.

Re: Day 1: IBM Watson Ties for first on Jeopardy

#80

Post by tocomment »

ebiester wrote: post #68
spicyj wrote: post #42

Earlier quoted context omitted.

Not very many people have a state-of-the-art supercomputer in their basement, so it wouldn't be that useful.

IBM's using two racks of computers and a huge SAN. Maybe it couldn't compete with watson, but a decent setup could be cobbled together for 5,000. It wouldn't be as fast , or as comprehensive, but it could be done. The harder part is the tens of thousands of man hours with very smart people.

I think a little project called Linux got tens of thousands of man hours with very smart people ... Just saying.
Post reply on HN