Live data from Hacker News

Twitter's garbage problem

jmillerinc.com

41–50 of 62 posts

Re: Twitter's garbage problem

#42

Earlier quoted context omitted.

I disagree. Some of the tweets I subjectively label as garbage might be legit. Hey, maybe some guy's mom is on twitter and wants to know what he had for lunch. It's not a people problem, it's a drawback of the platform that all of this different subject matter has to be broadcast in the same stream.

It's a people problem, but the problem is you. If you don't care about what he's having for lunch, then I'm not so sure what's so hard about ignoring his tweet. It's only going to be on your screen for a split second as you're scrolling through other stuff. Presumably anyone you follow would post more things that are worthwhile than not; otherwise, you should just find new people to follow. One of my friends tweets q…

I read all my twitter on my iphone, and if I could filter on the server, it could cut my daily reading from 20 minutes to 10 if this problem was solved, so to me that's a big deal.

Re: Twitter's garbage problem

#44
post #26

I can't believe nobody has mentioned naive Bayesian text classification yet. It sounds like it could work wonders for Twitter. I'm much more likely to be interested in tweets with words like "hylomorphism" than tweets with words like "omglol", and a text classification algorithm could learn that if you trained it up some. It doesn't have to be perfect; it just has to improve the signal-to-noise ratio significantly.

I've looked into this a bit, albeit more in a spam-filtering context; tweets have very little text for naive Bayes to latch onto. 140 characters would be 20-30 words, tops. That is so few words that it is hard to move the prior very much, unless there are blockbuster words that almost always indicate a bad tweet; as the article suggested, "breakfast", "beer", etc.

Here's a start. It's a simple ruby app that runs on heroku and proxies your twitter account, dropping uninteresting things. Extremely raw code.

http://github.com/fizx/sometweets

http://sometweets.heroku.com/

Re: Twitter's garbage problem

#47
post #26

Earlier quoted context omitted.

I've looked into this a bit, albeit more in a spam-filtering context; tweets have very little text for naive Bayes to latch onto. 140 characters would be 20-30 words, tops. That is so few words that it is hard to move the prior very much, unless there are blockbuster words that almost always indicate a bad tweet; as the article suggested, "breakfast", "beer", etc.

Filter by account, then, instead of by post? With a sample body of, say, the ten or twenty of an account's most recent posts.

The article's whole premise was that tweet quality does not correlate well within an account; e.g., some marvelous twitter streams include breakfast tweets.

Re: Twitter's garbage problem

#48
post #5

I can't tell which of the examples are supposed to be the valuable ones...

Oh, come on:

"A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty."

"I just ousted Owen S. as the mayor of Samovar Tea Lounge on @foursquare! "

Clearly, the second tweet can possibly be informative to some people.

Re: Twitter's garbage problem

#49

I have the same problem with Hacker News. I would love to be able to filter it based on my preferences.

If you caught the thread on whether search should be available on HN, and pg's response, I feel pretty safe saying that ain't going to happen, ever.

Thanks. I'm okay with that. Hacker News seems more like a labor of love than a monetizable product, so I don't expect pg to focus on it.

Re: Twitter's garbage problem

#50
The author misses a trick (or I missed a mention of it). Filtering out by the client used by the third party helps a lot. You can immediately filter out tweets coming automatically or semi-automatically from systems like Gowalla, Foursquare, blip.fm, Sharefeed, last.fm, auto news posting services, or even just through the Twitter API. Anyone else who posts crap in a manual, deliberate way should just be unfollowed.
Post reply on HN