Live data from Hacker News

Data Science of the Facebook World

blog.stephenwolfram.com

31–40 of 86 posts

Re: Data Science of the Facebook World

#31

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

Did your 2012 Strange Loop talk touch on the technology you used? Do you know if the video of the talk is still going to be released? https://thestrangeloop.com/news/strange-loop-2012-video-sche...

No, I invented most of the technology over the last month.

The naive Bayes and corpus wrangling was done by my colleague Etienne Bernard: http://www.wolframscience.com/summerschool/2012/alumni/berna...

Re: Data Science of the Facebook World

#32
post #25

Very nice looking graphs, but running "Wolfram Alpha Personal Analytics for Facebook" for my own profile comes with a rather nerve-wracking warning: Wolfram Connection would like to access your public profile, friend list, email address, custom friends lists, News Feed, relationships, birthday, status updates, checkins, education history, hometown, current city, photos, religious and political views, videos, likes an…

How do you think they are going to get all the data to analyze your facebook account?

I didn't realize going into it that the report would be that detailed. However, having opted in, I'm definitely impressed by what you get out of it.

Re: Data Science of the Facebook World

#33
post #17

How much of the friends with zero friends is simply because that information is blocked? If my friends "donated" their data, I would show as having 0 friends if I've blocked that information to apps.

Actually, NONE of the people in our dataset had zero friends. The x-axis starts at 1, not 0. The point is that resampling to remove the friendship paradox shows that there are many more people with single-digit friends than we expected.

Re: Data Science of the Facebook World

#34
post #2

How is it possible that Facebook, which owns the data, does not give tools like these but others tap this using their data?

The attitude of companies like Facebook, Google, Twitter is: if the product isn't addictive or useful to Billions of people, its not worth doing. Hence there are vastly more resources dedicated to assimilating eg photos and games into their ecosystem, than into something computationally innovative. This is IMHO a huge mistake, since they could instead be introducing simple forms of programming that takes you on a con…

I'm sure Facebook's Data Science team does a lot of interesting things internally. They do in fact have some interesting papers [0] and [1], though obviously with more of an 'academic' feel than the blog post.

[0]: http://arxiv.org/abs/1111.4503

[1]: http://arxiv.org/pdf/1201.4145 (edit)

Edit: they also have this FB page which has a steady stream of interesting stuff: https://www.facebook.com/data

Re: Data Science of the Facebook World

#35
post #17

How much of the friends with zero friends is simply because that information is blocked? If my friends "donated" their data, I would show as having 0 friends if I've blocked that information to apps.

All, since they're friends with the person who donated the data.

Re: Data Science of the Facebook World

#36

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

Are you planning to share some of that data in a reusable format (e.g. csv), not only as end-user plots (nice, BTW)?

It would be great to play with it a bit.

Re: Data Science of the Facebook World

#37

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

Is there a description available of the algorithm used to generate the 'cluster diagrams'? Really nice work.

Thanks!

Some of the plots use this: http://reference.wolfram.com/mathematica/ref/CommunityGraphP...

The underlying community detection uses: http://reference.wolfram.com/mathematica/ref/FindGraphCommun...

If you look under "Method", there are a bunch of different methods to use that I'm told correspond to various landmark papers in the field. If you know about community detection, you'll recognize which methods correspond to which papers, but if you don't, why do you care? At least, that's our philosophy for documentation, but I'm not sure I entirely agree with that philosophy.

Re: Data Science of the Facebook World

#38
I found this interesting. What I would love to have seen, however, is a probe into the dynamics. You did a nice abstraction over time as you measured property X as age was varied. I would have loved to have seen the manner in which topics and ideas spread over your network.

For instance: If an event occurred in New York, say, how long would it have taken to spread to San Francisco? If there were no progression, topic times would center around the same time. This would indicate that people were getting their information from national, not local sources (e.g. the evening news), then talking about it on facebook. On the other hand, if a local topic was spread on facebook alone, we should see some sort of progression.

It's possible that this progression could take more interesting forms besides geolocation, but that might require a more extensive network. A simple experiment would work like this: A few thousand people who are not friends but have a similar interest (say an interest in Elizabeth Warren) post independently a video of her. This particular esoteric interest is unlikely to be valued a priori by their friends, but perhaps they are compelled to repost the information. What's the threshold of "esotericness" such that it won't "go viral?" Is there a way to predict virality as a function of how popular it is to begin with? Is there no actual progression across the network, but rather a small bump in topic expression, until it is picked up by larger media sources at which point the entire network is inundated with people reposting Elizabeth Warren recaps from HuffPo et al?

The reason this is interesting is that it sheds insight into the role of social networks: are we fundamentally disposed toward central sources like the NYTimes, or is facebook a fundamental sharing mechanism? That is, do I post on facebook just to have my views expressed, validated, and challenged, so that they might change the world over a few years? Or do I post on facebook to have my views propagate across the world much more quickly?

Finally, a question: How did you estimate the power law? I know how difficult it is to do this (e.g. not linear regression on a log-log scale). Did you compare the power law fit to other, similar distributions, like lognormal? Preferential attachment is indeed a beautiful theoretical result, because it implies the existence of power law degree distributions. Unfortunately, many networks are not as well represented by power laws as by alternative distributions, which casts doubt on the preferential attachment hypothesis as is. (Also, many sampling methods give rise to fictive power laws). That said, a fat tail can still be interesting.

In any case, this is a beautiful piece of work.

Re: Data Science of the Facebook World

#39
post #36

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

Are you planning to share some of that data in a reusable format (e.g. csv), not only as end-user plots (nice, BTW)? It would be great to play with it a bit.

I'll have to check. Certainly there can be no harm in releasing some of the more aggregated data (i.e. that was behind the plots). Perhaps tweet at me so I don't forget -- @taliesinb

Re: Data Science of the Facebook World

#40

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

When it comes to "questions I would ask [the data]": which interests are likely shared by friends? (Can you make a correlation table, or a graph out of it?)

And - is correlations of one's interest vs friends interests the same as correlation of one's interests with itself.

Post reply on HN