I've also been collecting twitter data for a bit. I developed a heatmapping application that runs on the GPU to produce time-animated heatmaps in real-time for any user-generated query over a Solr database of hundreds of millions of geotagged tweets. You can see a rough demo at http://youtu.be/4_v2EZGiA7w . Hopefully I'll release it as a web app when I get time this summer.
Looking At The World Through Twitter Data
11–20 of 27 posts
Re: Looking At The World Through Twitter Data
#12We've been working on a similar project since last year @ http://smogfarm.com/ Feel free to get in touch!
Re: Looking At The World Through Twitter Data
#13http://topsy.com/ yeah. we do that.
Re: Looking At The World Through Twitter Data
#14Is any of this data available? It would be great to have access to it.
Re: Looking At The World Through Twitter Data
#15A friend and I started playing around with twitter data back in early 2010. We currently have something close to over 587 million tweets collected (We stopped collecting earlier this year). We only pulled English tweets and those that described what someone was feeling (Im, I am, I feel, I am feeling, etc. along with the negatives I don't feel, I do not feel, etc). We were able to see some interesting events happen d…
> There is a strange dip around noon that we are unsure of how to account for as our servers did not report any failures.
Maybe people are away from their computer at lunch.
What do the blue and green line colors indicate? It would also be interesting to track emoticons. :)
Re: Looking At The World Through Twitter Data
#16Hi, if you like that kind of stuff, I might give you an intro with Peter Gloor, who is author of swarmcreativity.net and at the MIT Center for Collective Intelligence. Tag #Twitter, Stock Prediction, Mood etc. You might meet on campus :)
Re: Looking At The World Through Twitter Data
#17A friend and I started playing around with twitter data back in early 2010. We currently have something close to over 587 million tweets collected (We stopped collecting earlier this year). We only pulled English tweets and those that described what someone was feeling (Im, I am, I feel, I am feeling, etc. along with the negatives I don't feel, I do not feel, etc). We were able to see some interesting events happen d…
> There is a strange dip around noon that we are unsure of how to account for as our servers did not report any failures. Maybe people are away from their computer at lunch. What do the blue and green line colors indicate? It would also be interesting to track emoticons. :)
The green line represents the number of tweets that were marked as being anxious and the blue lines represents tweets marked as calm.
You can see that after the tsunami hit there was a general spike in the overall traffic, but a much larger spike for tweets where the user described being anxious.
We also analyzed the tweets for emotions flagging each to either be "happy" or "sad". Don't have the data able to be displayed in any consumable format at the moment though.
These are some logs for the day (totals)
Calm Anxious Happy Sad
2011-03-08 2034 8730 77032 94119
2011-03-09 1349 5129 47708 59406
2011-03-10 1614 6020 51623 72214
2011-03-11 4126 20427 87763 126688
2011-03-12 3251 13009 104434 136389
Re: Looking At The World Through Twitter Data
#18Hi, if you like that kind of stuff, I might give you an intro with Peter Gloor, who is author of swarmcreativity.net and at the MIT Center for Collective Intelligence. Tag #Twitter, Stock Prediction, Mood etc. You might meet on campus :)
We know Peter, actually. He's a great guy and we've been talking to him about this. Thanks though! :)
Re: Looking At The World Through Twitter Data
#19If you're interested in how information diffuses through social networks like Twitter, take a look at Truthy (one of my projects): http://truthy.indiana.edu Truthy is a system to analyze and visualize the diffusion of information on Twitter. The Truthy system evaluates thousands of tweets an hour to identify new and emerging bursts of activity around memes of various flavors. The data and statistics provided by Truth…
sounds like a cool thing to tackle. We'll definitely look at it!
Re: Looking At The World Through Twitter Data
#20But, this is because non-English tweets that we have discarded are much more frequent during the night in our time zone, and they often don’t contain the word ‘a’ as often as English tweets do. This doesn't make sense; are they only discarding the non-English tweets during certains times?
We just mean that there's more tweeting going on in non-English speaking countries when it's night-time here.