Hey Twitter: Give us our Tweets • Hacker News Board

Re: Hey Twitter: Give us our Tweets

#61

Post by qthrul » Sat, Sep 25, 2010, 1:52 AM UTC

donohoe wrote: post #55

Um. Hand on a sec... on the bottom of the same page you link to: Twitter still maintains a database of all the tweets sent by a user. However, to ensure performance of the site, this artificial limit is temporarily in place So maybe you can't do it now - but its not lost (yet).

Hmmm. Long ago I sent examples of the "NULL" issues for Twitter status URL's that I had bookmarked (possibly before favorites were available) as replies to the Twitter team -- not sure where any of that went but while a database is likely to exist, a guaranteed and representative content store may not be as likely.

Re: Hey Twitter: Give us our Tweets

#62

Post by ck2 » Sat, Sep 25, 2010, 4:24 AM UTC

It's just a technological failure (because it requires too much cpu power to dig through a non-indexed non-relational db) and they don't want to show their shame of poor design.

Can you imagine any other service surviving where you cannot lookup older data?

BTW Google has a social search that indexes twitter and facebook but it's not complete by any means.

Re: Hey Twitter: Give us our Tweets

#63

Post by moron4hire » Sat, Sep 25, 2010, 6:37 AM UTC

pronoiac wrote: post #56

moron4hire wrote: post #41

Earlier quoted context omitted.

From Twitter's perspective, you're absolutely right. From a user's perspective, the bulk reliance on singular companies to manage our data is an issue that I'm afraid is going to become dangerous. Look at the 2.5 hour Facebook outage from a couple of days ago. If Facebook were to be hit by an airplane or asteroid, a seriously significant portion of 500,000,000 peoples' lives would be lost, if only just in photos. If…

I have thousands of emails between close friends in Gmail that I would like to preserve, but no convenient system exists for archiving them away from Google. How about IMAP?

Tried it, volume is too large. Every client I've tried crashes before it can get everything, and there is still the issue of convenient storage format. I'll probably have to write my own archiver.

Re: Hey Twitter: Give us our Tweets

#64

Post by moron4hire » Sat, Sep 25, 2010, 6:44 AM UTC

_delirium wrote: post #60

moron4hire wrote: post #41

Earlier quoted context omitted.

From Twitter's perspective, you're absolutely right. From a user's perspective, the bulk reliance on singular companies to manage our data is an issue that I'm afraid is going to become dangerous. Look at the 2.5 hour Facebook outage from a couple of days ago. If Facebook were to be hit by an airplane or asteroid, a seriously significant portion of 500,000,000 peoples' lives would be lost, if only just in photos. If…

That reminds me of Jason Scott's comments about trusting "the cloud" with your data and not keeping it locally: http://ascii.textfiles.com/archives/1717

That's exactly right

Re: Hey Twitter: Give us our Tweets

#65

Post by confuzatron » Sat, Sep 25, 2010, 8:33 AM UTC

It's funny to me to see people prizeing their tweets so highly.

I reviewed mine a while back and realised they were inconsequential drivel, so I deleted them along with the account. Now I have a read-only account for reading the tweets of some folk who tend to be funny to read. No techies trying to demonstrate value, no not-that-interesting acquaintances, no naive political rants from relatives - much better.

Re: Hey Twitter: Give us our Tweets

#66

Post by Raphael » Sat, Sep 25, 2010, 9:29 AM UTC

senko wrote: post #12

arst wrote: post #11

Archival of 'cloud' data is an issue I've been thinking a lot about lately. People are putting so much of their lives on the internet these days without generally giving too much thought to permanence and availability. Twitter's only been around a few years and people are already running into retention issues. The tweet from the article will still be valuable 30 years from now - will twitter even be around then? Will…

It's not local, and not in an open data format, and not free, but you might like http://www.backupify.com/ , they have a nice list of services that they can back up for you (haven't used it, so wouldn't know how good they are in practice).

Backupify works, but the formatting is odd. Your Twitter stream is packaged as a PDF book.

Re: Hey Twitter: Give us our Tweets

#67

Post by jaymon » Sat, Sep 25, 2010, 10:42 AM UTC

moron4hire wrote: post #63

pronoiac wrote: post #56

Earlier quoted context omitted.

I have thousands of emails between close friends in Gmail that I would like to preserve, but no convenient system exists for archiving them away from Google. How about IMAP?

Tried it, volume is too large. Every client I've tried crashes before it can get everything, and there is still the issue of convenient storage format. I'll probably have to write my own archiver.

I wrote this a few years ago to archive my Gmail emails to text files (and the attachments to whatever they are).

I haven't ran it in awhile but when I did it worked and was able to download my emails from the very beginning without crashing. Anyway, you might find it useful...

http://github.com/Jaymon/Popbak

Re: Hey Twitter: Give us our Tweets

#68

Post by svag » Sat, Sep 25, 2010, 11:00 AM UTC

moron4hire wrote: post #63

pronoiac wrote: post #56

Earlier quoted context omitted.

I have thousands of emails between close friends in Gmail that I would like to preserve, but no convenient system exists for archiving them away from Google. How about IMAP?

Tried it, volume is too large. Every client I've tried crashes before it can get everything, and there is still the issue of convenient storage format. I'll probably have to write my own archiver.

I have been using Zimbra Desktop as an email client, http://www.zimbra.com/products/desktop.html, in my Mac in several GMail accounts with no problem. There are versions for Window and Linux.

Re: Hey Twitter: Give us our Tweets

#69

Post by coryhaik » Sat, Sep 25, 2010, 1:02 PM UTC

as a journalist -- I loved this post. there is a lot we could mine and then synthesize. really need better ways to try to tell more _meaningful_ stories and cover all this 'now, now, now' content.

@coryhaik washington post

Re: Hey Twitter: Give us our Tweets

#70

Post by fjpoblam » Sat, Sep 25, 2010, 1:23 PM UTC

I'm glad Twitter doesn't keep a record of my tweets. Perhaps for what you want, they could make it an opt-in service, but for invasion of my privacy, I'd just as soon not. And for their service load (media needed to store your tweets and server bandwidth time needed to make them accessible to you) I can well see how it might be a payable service.