It is all available to the public to begin with anyway. I don't see the dilemma here.
Not all tweets are public.
Surely they aren't coughing up private tweets?
Twitter to Release All Tweets to Scientists
41–50 of 55 posts
Re: Twitter to Release All Tweets to Scientists
#42I've heard that before. * Library of Congress: http://blogs.loc.gov/loc/2013/01/update-on-the-twitter-archi... * Twitter Data grants: https://blog.twitter.com/2014/introducing-twitter-data-grant... I'll admit, I haven't applied for access through either one, but neither have I seen any papers cite access through those venues—and I read quite a few NLP + Twitter papers.
This article is just talking about Twitter Data Grants for which 6 universities were decided as winners [0]. You won't see papers through these grants as yet because well, the winners were announced about 40 days back!
[0] https://blog.twitter.com/2014/twitter-datagrants-selections
Re: Twitter to Release All Tweets to Scientists
#43Earlier quoted context omitted.
Jeez, 400 GB of text per day? How the hell? EDIT: That's 11.5k tweets/sec. How do you get eleven thousand people to tweet every second?
Maybe it includes images.
it does not include images. A tweet object does include a bunch of metadata though: https://dev.twitter.com/docs/platform-objects/tweets
Re: Twitter to Release All Tweets to Scientists
#44I can't wait to see someone legitimately design a better sewerage system by using twitter's geolocation.
Their geo-data is utter crap. The vast majority of it is based on 'profile location', which means that there are almost a million people tweeting from the exact center of Atlanta. It's a crowded spot, must be a Starbucks there or something.
Just find those mass locations and remove them from the data set.
Re: Twitter to Release All Tweets to Scientists
#45Re: Twitter to Release All Tweets to Scientists
#46There is no magic discovery about the nature of man hidden away in that data. Nothing you're average stand up comedian hasn't already written a bit about.
That was a good one. On a more serious tone, there is one area of research that Twitter data is very valuable for: how information and disinformation is created and spread during major news events: wars, catastrophes, uprisings, school shootings etc. My gut feeling is that today a lot of quality journalism happens outside of the traditional journalistic organisations. The downside is that also a lot of wild speculati…
And momentum stock trades. I've always felt a adept programmer who knew how to data mine and worked for Twitter
could make a fortune in the stock market?
Re: Twitter to Release All Tweets to Scientists
#47One thing I've wondered. Is it possible to follow "everyone" on Twitter? If-not, what type of cap does Twitter enforce on the number of accounts you're allowed to follow? I realize it'd be difficult to know which new accounts to add as people join, but how far could you push a roll-your-own stream of the Twitter firehose?
Hypothetically: I'm sure you could do it algorithmically; if your program sees a retweet from someone who is not on your following list, you then follow them. You might miss a few, but you would get most everyone.
Re: Twitter to Release All Tweets to Scientists
#48This is exciting to me; does anyone know how Twitter will go about this? Will there be a public dataset available for download? A research contract through the recently-acquired GNIP? Or just firehose access for future streams?
Considering there's at least 400GB of data generated per day, I don't think it'll be readily available for the public as a download.
hell lot of data! First they have to find some alternative to zip these much of data!
Re: Twitter to Release All Tweets to Scientists
#49Earlier quoted context omitted.
Jeez, 400 GB of text per day? How the hell? EDIT: That's 11.5k tweets/sec. How do you get eleven thousand people to tweet every second?
Have you ever heard about the 'Human Brain Project' ? Well, they produce 100 GB of data _per second_ Welcome to the real big data world.
Woah, 100 GB per second???!!
What exactly do they do?
Edit: Never mind, this is what they are doing
"We would like to develop some kind of ‘google' brain where we can zoom in and out, see it from different perspectives and understand how brain structure and function is related."
Re: Twitter to Release All Tweets to Scientists
#50Earlier quoted context omitted.
Their geo-data is utter crap. The vast majority of it is based on 'profile location', which means that there are almost a million people tweeting from the exact center of Atlanta. It's a crowded spot, must be a Starbucks there or something.
Just find those mass locations and remove them from the data set.
You end up with about 0.01% of tweets having locations after that. It's basically just iphone users.