Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Network visualization of 50k blogs and links

graph.henryn.ca

Post reply on HN
Print view View original

41–50 of 58 posts

Re: Network visualization of 50k blogs and links

#41

Post by dredmorbius »

TuringNYC wrote: post #37
ng-henry wrote: post #2

I scraped my favorite blogs and made a graph from the domains that each blog links to. You can see clusters forming of websites that talk about similar topics, like crypto, rationality, Canada, India, and even postgres! The visualization was made entirely in webgl with some neat optimizations to render that many lines and circles.

> I scraped my favorite blogs and made a graph from the domains that each blog links to. Nice analysis! However, I'm guessing these arent your fav blogs as there are tens of thousands of entries! How did you decide which blogs to index, did you use some central registry of blogs?

https://news.ycombinator.com/item?id=40137135>

Re: Network visualization of 50k blogs and links

#42

Post by ng-henry »

3abiton wrote: post #35
PaulHoule wrote: post #14

People still upvote hairball graphs every time. Fortunately there is a cure: https://cambridge-intelligence.com/how-to-fix-hairballs/

Is graph data processing considered visualization style? It is changing the data, how can this be considered "visualization"?

We aren't changing the data, just changing how the graph is displayed.

Re: Network visualization of 50k blogs and links

#45

Post by jmmv »

Reminds me of the intern project I worked on at Google back in 2008.

My mentor at the time had a traceroute dataset of the Internet and wanted to render it on top of Google Maps. I implemented a MapReduce algorithm that geolocated the data points and then produced Google Maps tiles at various zoom levels to show how the Internet was connected. It was pretty cool to visualize how the data flowed throughout the world and to be able to "dig deeper" by zooming into the mess of connections. Very similar to what this project does!

The project didn't go anywhere but it was a cool fun experiment and a great learning opportunity for me (S2 geometry is... well, weird, but touching MapReduce and Bigtable were invaluable exercises for my later tenure at the company). Those were very different times. I don't think you would be able to pursue such a "useless" project as an intern at Google these days.

Re: Network visualization of 50k blogs and links

#46

Post by omeid2 »

3abiton wrote: post #35
PaulHoule wrote: post #14

People still upvote hairball graphs every time. Fortunately there is a cure: https://cambridge-intelligence.com/how-to-fix-hairballs/

Is graph data processing considered visualization style? It is changing the data, how can this be considered "visualization"?

The relationship of data is part of the data.

Your visualisation tool may require it in a specific format, but it still about properties of your data.

Re: Network visualization of 50k blogs and links

#47

Post by jll29 »

This is a neat idea - however, I think the graphical view of the blog graph trades "coolness" for "utility".

Have you thought of a front end that is basically just text/plain HTML (in normal size) + navigation links to explore the blogs in one frame, and the currently chosen blog in another frame? That way, you could look at the blogs while travelling your crawl graph, a kind of "blog explorer".

Re: Network visualization of 50k blogs and links

#49

Post by ploum »

etimberg wrote: post #43
jszymborski wrote: post #20

My blog is on here, but as a lonely, lonely node. I link to stuff I promise!

Same. Honestly a bit surprised, in a good way, thst my site made it. I didn’t think anyone read it

Mine is listed as "personal growth" and I find it quite funny (that’s probably the last category I would think for my blog posts)
Post reply on HN