Live data from Hacker News

Twitter to Release All Tweets to Scientists

scientificamerican.com

1–10 of 55 posts

Re: Twitter to Release All Tweets to Scientists

#4
post #2

This is exciting to me; does anyone know how Twitter will go about this? Will there be a public dataset available for download? A research contract through the recently-acquired GNIP? Or just firehose access for future streams?

Doesn't Twitter make a fair bit of money from selling access to various slices of their data? I'd be surprised if they released it all to the general public. I imagine scientists would have to be under some sort of NDA.

Re: Twitter to Release All Tweets to Scientists

#5
Twitter had initiated granting of datasets some time back (now closed)[0] on the merits of a short proposal. The number of groups who eventually got access to the data were very few[1]. I hope in the future they increase the number of grants.

[0] https://blog.twitter.com/2014/introducing-twitter-data-grant...

[1] https://blog.twitter.com/2014/twitter-datagrants-selections

Re: Twitter to Release All Tweets to Scientists

#6

Twitter had initiated granting of datasets some time back (now closed)[0] on the merits of a short proposal. The number of groups who eventually got access to the data were very few[1]. I hope in the future they increase the number of grants. [0] https://blog.twitter.com/2014/introducing-twitter-data-grant... [1] https://blog.twitter.com/2014/twitter-datagrants-selections

Yes, I am pretty sure this article is just rehashing the Twitter grants (I believe there were only 6 to 8 rewards), rather than announcing full open data to any researchers (thereby making the title misleading).

Re: Twitter to Release All Tweets to Scientists

#7
I've heard that before.

* Library of Congress: http://blogs.loc.gov/loc/2013/01/update-on-the-twitter-archi...

* Twitter Data grants: https://blog.twitter.com/2014/introducing-twitter-data-grant...

I'll admit, I haven't applied for access through either one, but neither have I seen any papers cite access through those venues—and I read quite a few NLP + Twitter papers.

Re: Twitter to Release All Tweets to Scientists

#10

This would likely make a great natural language data set for compression algorithms.

True, but doesn't Twitter already provide an API for access to a fraction of the firehose? Surely that would be enough data. If Twitter doesn't have a good API, Reddit allows full access to all comments through their API (although Reddit has orders of magnitude less data).
Post reply on HN