Free SQL dump with 200 million tweets from 13 million users • Hacker News Board

Re: Free SQL dump with 200 million tweets from 13 million users

#21

Post by calufa » Wed, Jun 08, 2011, 6:17 PM UTC

jason_slack wrote: post #19

calufa wrote: post #5

Earlier quoted context omitted.

I will blog about how I did it in a few days...

Where do you Blog so I can add to my RSS?

I dont have a blog, sorry. I will open one soon...

Feel free to follow me http://twitter.com/calufa.

Re: Free SQL dump with 200 million tweets from 13 million users

#22

Post by juiceandjuice » Wed, Jun 08, 2011, 6:19 PM UTC

Wow, I just downloaded that whole archive in a minute.

Re: Free SQL dump with 200 million tweets from 13 million users

#23

Post by calufa » Wed, Jun 08, 2011, 6:23 PM UTC

juiceandjuice wrote: post #22

Wow, I just downloaded that whole archive in a minute.

bz2 compression ;) --- 1147480:1 compression ratio

Re: Free SQL dump with 200 million tweets from 13 million users

#24

Post by jdvolz » Wed, Jun 08, 2011, 6:32 PM UTC

Calufa, next time you're in Vegas, send me a message and we'll get a beer. Thank you. You just made something I'm doing vastly more awesome.

Re: Free SQL dump with 200 million tweets from 13 million users

#25

Post by chrisjsmith » Wed, Jun 08, 2011, 6:34 PM UTC

All that is meaningless chatter between people and information about bathroom habits. Perhaps if we pooled that distributed effort into something constructive, the world would be a better place.

Re: Free SQL dump with 200 million tweets from 13 million users

#26

Post by [deleted] » Wed, Jun 08, 2011, 6:34 PM UTC

jdvolz wrote: post #24

Calufa, next time you're in Vegas, send me a message and we'll get a beer. Thank you. You just made something I'm doing vastly more awesome.

[deleted]

Re: Free SQL dump with 200 million tweets from 13 million users

#27

Post by calufa » Wed, Jun 08, 2011, 8:00 PM UTC

A EMAIL FROM TWITTER KILLED THE DATASET --- :S

Re: Free SQL dump with 200 million tweets from 13 million users

#28

Post by user24 » Wed, Jun 08, 2011, 8:13 PM UTC

calufa wrote: post #27

A EMAIL FROM TWITTER KILLED THE DATASET --- :S

Can you give more detail? The link is still up... What did they say?

edit reply via twitter: "they asked me to remove the dump due TOS" (http://twitter.com/#!/calufa/status/78556903772393474)

which I guess is what I expected.

But are scrapers subject to TOS?

Re: Free SQL dump with 200 million tweets from 13 million users

#29

Post by kodeshpa » Wed, Jun 08, 2011, 11:55 PM UTC

jason_slack wrote: post #19

calufa wrote: post #5

Earlier quoted context omitted.

I will blog about how I did it in a few days...

Where do you Blog so I can add to my RSS?

if you are interested in crawling FB, check this out http://www.zubha-labs.com/oauth-trick-for-facebook-desktop-a...

Re: Free SQL dump with 200 million tweets from 13 million users

#30

Post by laprise » Thu, Jun 09, 2011, 2:44 AM UTC

Neat ! here some tips for creating a kick ass graph visualization: http://www.martinlaprise.info/2010/02/15/visualize-your-own-...