Live data from Hacker News

Hey Twitter: Give us our Tweets

zachholman.com

51–60 of 77 posts

Re: Hey Twitter: Give us our Tweets

#52
post #30
post #14

The library of congress has them, if we could get a copy we could index it all somewhere to re-construct a copy of people's early timelines http://blogs.loc.gov/loc/2010/04/how-tweet-it-is-library-acq... I was thinking recently about getting all the tweets from 2006, it should be about 2 million tweets, so should be possible to fetch by sequentially walking through the ids.

This is great. It's going to be a goldmine for historians in a few years. Twitter is becoming a finger on the pulse of the world.

Twitter is going to be huge!

Re: Hey Twitter: Give us our Tweets

#53
I'm going to plug Anil Dash and Gina Trapani's open source PHP project ThinkUp (http://thinkupapp.com/). It started off as a pet project by Gina Trapani, but now it's used by the White House to try to engage the American public more.

ThinkUp can archive your tweets to your server for any reason you want to. Not only does it archive tweets, but it also allows you to view some nice analytics such as a map of where replies to a tweet came from, a chart of your followers over time, who your most active friends, etc.

But wait, there's more! If you check it out right now, we'll throw Facebook integration, for absolutely free! That's right. For absolutely nothing, you can archive your Facebook statuses, and preserve all those precious memories.

You can get all this for $0.00. That's right, $0.00. Just go to http://github.com/ginatrapani/thinkup/ and download the code today. You can create plugins for it, modify it, and do anything to it the GPL allows you to do.

Re: Hey Twitter: Give us our Tweets

#54
post #6

Well, I think you just found a way for Twitter to make some money. Remove the theoretical access and Web limit for accounts or increase limit to impossibly high number. It is much akin to NYT charging for access to old articles. They may have to store more tweets in the current database or charge a retrieval fee to obtain it from the archive. They could give it away for free too...

Charge for mapreduce jobs that can run over the whole corpus.

Re: Hey Twitter: Give us our Tweets

#55
Um. Hand on a sec... on the bottom of the same page you link to:

  Twitter still maintains a database of all 
  the tweets sent by a user. However, to 
  ensure performance of the site, this 
  artificial limit is temporarily in place
So maybe you can't do it now - but its not lost (yet).

Re: Hey Twitter: Give us our Tweets

#56
post #35

Earlier quoted context omitted.

While everything you said is correct, at the same time, there's no sense in losing access to data that's recorded and is sitting right there for the taking, if only there were a more convenient way to extract it. It would be very wise of Twitter to offer a bulk export function - not because they have to, but because it's to Twitter's benefit to not have me thinking twice about whether I should post to Twitter or comm…

From Twitter's perspective, you're absolutely right. From a user's perspective, the bulk reliance on singular companies to manage our data is an issue that I'm afraid is going to become dangerous. Look at the 2.5 hour Facebook outage from a couple of days ago. If Facebook were to be hit by an airplane or asteroid, a seriously significant portion of 500,000,000 peoples' lives would be lost, if only just in photos. If…

I have thousands of emails between close friends in Gmail that I would like to preserve, but no convenient system exists for archiving them away from Google.

How about IMAP?

Re: Hey Twitter: Give us our Tweets

#57
post #19
post #17

Earlier quoted context omitted.

$120/year is not worth it to me. $10/year would be. Just my feedback. :-)

You can run Tweetnest yourself http://pongsocket.com/tweetnest/ so that's free, here's mine: http://tweets.rythie.com/ for example. There are other solutions around if you look.

I agree with Tweetnest. It's an easy and free way to have a backup of all your tweets.

It's nice that it groups tweets by mont and day, so you can pinpoint that one tweet you tweeted that day. Got mine running here: http://jarqu.es/t/

Re: Hey Twitter: Give us our Tweets

#58
Check out http://pongsocket.com/tweetnest/ .. it's open source, free, PHP (easy to install), yada yada. It's a simple PHP Twitter backup system that also presents your tweets in a reasonably attractive archive on your own site. Example: http://peterc.org/twitter/

(Yes, there's also http://thinkupapp.com/ - but it's more complex and has more features/supported services. I found TweetNest crazy easy to get started with though.)

Thankfully I took a CSV dump of my tweets 2006-2008 but I'm missing quite a few in the gap :-(

Re: Hey Twitter: Give us our Tweets

#59
post #40

i like it that tweets are ephemeral. its good that way. why the need to archive every-fucking-thing? imagine how shit the world would be if everyone could browse everything they ever said? rather than the things memorable enough to remember. or memorable enough to someone else that they'd remember? imagine how many useful people would become lawyers? WHAT ARE YOU THINKING!!!!!!!!!!!!???????????????

Historians & archaeologists value having an unfiltered look at what people were doing & thinking, because it's more honest than, well, looking at memorable events in retrospect with 20/20 hindsight, filtered through biases that only happened later.

Re: Hey Twitter: Give us our Tweets

#60
post #35

Earlier quoted context omitted.

While everything you said is correct, at the same time, there's no sense in losing access to data that's recorded and is sitting right there for the taking, if only there were a more convenient way to extract it. It would be very wise of Twitter to offer a bulk export function - not because they have to, but because it's to Twitter's benefit to not have me thinking twice about whether I should post to Twitter or comm…

From Twitter's perspective, you're absolutely right. From a user's perspective, the bulk reliance on singular companies to manage our data is an issue that I'm afraid is going to become dangerous. Look at the 2.5 hour Facebook outage from a couple of days ago. If Facebook were to be hit by an airplane or asteroid, a seriously significant portion of 500,000,000 peoples' lives would be lost, if only just in photos. If…

That reminds me of Jason Scott's comments about trusting "the cloud" with your data and not keeping it locally: http://ascii.textfiles.com/archives/1717
Post reply on HN