Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science of the Facebook World

blog.stephenwolfram.com

Post reply on HN
Print view View original

41–50 of 86 posts

Re: Data Science of the Facebook World

#41

Post by taliesinb »

stared wrote: post #40
taliesinb wrote: post #13

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

When it comes to "questions I would ask [the data]": which interests are likely shared by friends? (Can you make a correlation table, or a graph out of it?) And - is correlations of one's interest vs friends interests the same as correlation of one's interests with itself.

That's a good idea! Though if I remember correctly, interests aren't canonicalized, so it might be pretty messy. And I'm not sure if people fill them in non-ironically any more.

Re: Data Science of the Facebook World

#42

Post by ececconi »

taliesinb wrote: post #13

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

This is some really awesome work. How much time do you spend analyzing social network information?

Do you think analyzing information about social networks will bring out real world insights about people and relationships?

Re: Data Science of the Facebook World

#43

Post by taliesinb »

kevinalexbrown wrote: post #38

I found this interesting. What I would love to have seen, however, is a probe into the dynamics. You did a nice abstraction over time as you measured property X as age was varied. I would have loved to have seen the manner in which topics and ideas spread over your network. For instance: If an event occurred in New York, say, how long would it have taken to spread to San Francisco? If there were no progression, topic…

Interesting points.

1. Dynamics

You're right, that would be very interesting. The most obvious way we could have done this is by looking at the spread of our app itself as people started to use it. Unfortunately, we only started recording anonymized stats for the second release, so we've somewhat missed the boat there.

To do it with links and general "memes" would be technically much harder, because we'd have to periodically rescrape walls of all the donors to see time evolution. It was somewhat out of scope of the blog post, given all the more basic stuff we could do instead.

I'd be surprised if Facebook didn't already do an analysis of this when they "cracked down" on app virality a while back.

Bit.ly's Hilary Mason might have looked at this question too, and I'm sure it has been done to death with Twitter, though the demograph info is much sparser there.

2. This not being a scientific paper, we estimated it by drawing on the log-log CDF. Barring the noise that "deparadoxing" the friend's friend count distribution induces on the low end of the distribution, it was very linear over two decades. We didn't think the exact number was all that interesting, so we didn't spend any more effort than that. Facebook's anatomy paper probably has a very accurate number.

I'd heard about the fictive power law stuff. What makes me even more skeptical is that FB friends are probably a poor proxy for 'true' friends. You'd be better off looking at number of friends as defined by some cross-commenting threshold.

3. Thanks! It was a lot of fun!

Re: Data Science of the Facebook World

#44

Post by taliesinb »

ececconi wrote: post #42
taliesinb wrote: post #13

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

This is some really awesome work. How much time do you spend analyzing social network information? Do you think analyzing information about social networks will bring out real world insights about people and relationships?

Quite a bit -- I've been analyzing my own data for years now.

I did some network analysis stuff like this for Twitter long before it was built into Mathematica (rant: someone at Twitter needs to make a 'graph query' API call so that it doesn't take 3 hours to get a single graph of your own network).

I would link to the relevant posts at taliesinb.net, but Posterous is down 7 days ahead of schedule.

I think it can bring out real world insights. You just have to be very cautious and not leap to conclusions because they seem to tell an interesting story. Although it is somewhat disturbing how "gendered" the wall post topic distributions are.

Re: Data Science of the Facebook World

#46

Post by stared »

taliesinb wrote: post #39
stared wrote: post #36

Earlier quoted context omitted.

Are you planning to share some of that data in a reusable format (e.g. csv), not only as end-user plots (nice, BTW)? It would be great to play with it a bit.

I'll have to check. Certainly there can be no harm in releasing some of the more aggregated data (i.e. that was behind the plots). Perhaps tweet at me so I don't forget -- @taliesinb

Great! Tweeted.

Re: Data Science of the Facebook World

#47

Post by taliesinb »

austinl wrote: post #45

I've been doing Facebook network visualization for a while now with Gephi. Here are some of the graphs I came up with: http://visualizingpolitics.wordpress.com/2012/05/02/facebook...

Nice! Do you know if Gephi can do something similar to the summarization that we did using cluster diagrams? The whole "ball of hair" problem doesn't have any other real solution, I don't think (well, unless you use edge clustering, but that doesn't help in-group connections).

Re: Data Science of the Facebook World

#48

Post by susi22 »

taliesinb wrote: post #13

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

The only visualization I didn't like was the chord diagram: http://blog.stephenwolfram.com/data/uploads/2013/04/chordplo...

Did you try this visualization?: http://bl.ocks.org/mbostock/4062006

Re: Data Science of the Facebook World

#49

Post by susi22 »

jonpeda wrote: post #3

People donate their data to support Wolfram's closed-source, paid-license, for-profit program?

Even though WA is closed source and for-profit, I few them (and the company) as a kind of fellow scientist and not the evil big corporation (like oracle, microsoft etc.).

It all depends who ask for my data.

Re: Data Science of the Facebook World

#50

Post by taliesinb »

susi22 wrote: post #48
taliesinb wrote: post #13

I did the analysis and worked with Stephen on the science side of it. If anyone would like to ask questions about what we did, I'd be happy to answer them. There's still lots more interesting stuff to do, but it was enough for a blog post. Suggest away if you think we missed something obvious!

The only visualization I didn't like was the chord diagram: http://blog.stephenwolfram.com/data/uploads/2013/04/chordplo... Did you try this visualization?: http://bl.ocks.org/mbostock/4062006

Yes, I did.

We'll have to agree to disagree. I think the visualization you linked to is much harder to read, because the visual weight accorded to each edge is a non-linear (and somewhat arbitrary) function of the 'true' weight. It also doesn't scale well with number of vertices.

Whereas with the chord diagram, your eye is naturally drawn to the big arrows, and you can easily follow them. It's also bidirectional in a more straightforward way.

Post reply on HN