HN: I'll run a marathon to build whatever you decide on tweet.in • Hacker News Board

Re: HN: I'll run a marathon to build whatever you decide on tweet.in

#21

Post by hotpockets » Thu, Jul 02, 2009, 5:16 AM UTC

chaosprophet wrote: post #17

hotpockets wrote: post #16

Earlier quoted context omitted.

There really aren't that many ways to say "I heard Jeff goldblum just died???". You could probably make a list of like 100 different ways and get good accuracy. In fact, question marks alone are a good indication the person didn't do any private verification.

If you were going to create 100 different rules for that one sentence, imagine the costs of scaling up. It simply is not viable. The only way to make this possible is to maybe train the system for a corpus consisting of about a million tweets. But the problem with that would be that those million tweets would first have to be individually tagged manually by humans.

But how many ways can a rumor be stated? Say the rumor is someone got married: "did so-and-so just get married???". If you make a 100 sentence structures to look for could you reuse them for multiple rumors?

Re: HN: I'll run a marathon to build whatever you decide on tweet.in

#22

Post by chaosprophet » Thu, Jul 02, 2009, 1:08 PM UTC

hotpockets wrote: post #21

chaosprophet wrote: post #17

Earlier quoted context omitted.

If you were going to create 100 different rules for that one sentence, imagine the costs of scaling up. It simply is not viable. The only way to make this possible is to maybe train the system for a corpus consisting of about a million tweets. But the problem with that would be that those million tweets would first have to be individually tagged manually by humans.

But how many ways can a rumor be stated? Say the rumor is someone got married: "did so-and-so just get married???". If you make a 100 sentence structures to look for could you reuse them for multiple rumors?

The problem is not how many different ways a rumor can be written, it's how many rumors can be written. Esp. considering the fact that you have websites which automate the process of rumor-manufacturing...

Re: HN: I'll run a marathon to build whatever you decide on tweet.in

#23

Post by chaosprophet » Thu, Jul 02, 2009, 1:24 PM UTC

plaggypig wrote: post #13

chaosprophet wrote: post #2

You could build a twitter search app, which searches and returns real time news. It should be capable of categorizing tweets about an event as headline(MJ's dead), more info at(latimes.com), comments. In short it should search twitter for real news and display it in a traditional blog style. Perhaps, it could even display excerpts from the info source site. The comments display should be real time so that you can in…

Interesting idea and I can think of a novel interface for this. I upvoted you, and that's where by influence will end.. for now :)

Well if you're interested in fleshing it out, I might be able to provide some help with the natural language parsing.

Re: HN: I'll run a marathon to build whatever you decide on tweet.in

#24

Post by hotpockets » Fri, Jul 03, 2009, 8:27 PM UTC

chaosprophet wrote: post #22

hotpockets wrote: post #21

Earlier quoted context omitted.

But how many ways can a rumor be stated? Say the rumor is someone got married: "did so-and-so just get married???". If you make a 100 sentence structures to look for could you reuse them for multiple rumors?

The problem is not how many different ways a rumor can be written, it's how many rumors can be written. Esp. considering the fact that you have websites which automate the process of rumor-manufacturing...

Thanks for your comments, I agree that something automated is more efficient and robust. However, I was also thinking doing something manual might be a quick way to build and launch a site, even if it has poor accuracy. If you want to discuss it further, send me an email. Its in my profile.