Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Network visualization of 50k blogs and links

graph.henryn.ca

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Network visualization of 50k blogs and links

#31

Post by ng-henry »

jseliger wrote: post #27
ng-henry wrote: post #2

I scraped my favorite blogs and made a graph from the domains that each blog links to. You can see clusters forming of websites that talk about similar topics, like crypto, rationality, Canada, India, and even postgres! The visualization was made entirely in webgl with some neat optimizations to render that many lines and circles.

This is very cool but also not accurate, at least for jakeseliger.com. Henryn.ca lists 0 links from jakeseliger.com to nytimes.com, reason.com, and numerous others that simple search demonstrates are linked to, for example: https://jakeseliger.com/?s=nytimes.com&submit=Search I put up many links posts, so I probably link to an abnormally large number of sites.

Yep this is only for stuff that we've crawled, so we can't detect all of your links. Because we have limited crawling resources, we rate-limit the crawling by domain so we don't get stuck in spider traps.

The current visualization only shows the current state of the crawl, so it won't know about all of the posts.

Re: Network visualization of 50k blogs and links

#33

Post by ng-henry »

rcarmo wrote: post #30

Hmmm. My site is listed, but I have _way_ more inbound and outbound links than shown. And I have my own internal links visualization, which might be a bit over the top (GPU recommended): https://taoofmac.com/static/graph

See this comment I posted in another thread:

Yep this is only for stuff that we've crawled, so we can't detect all of your links. Because we have limited crawling resources, we rate-limit the crawling by domain so we don't get stuck in spider traps. The current visualization only shows the current state of the crawl, so it won't know about all of the posts.

Re: Network visualization of 50k blogs and links

#34

Post by nickjj »

ng-henry wrote: post #2

I scraped my favorite blogs and made a graph from the domains that each blog links to. You can see clusters forming of websites that talk about similar topics, like crypto, rationality, Canada, India, and even postgres! The visualization was made entirely in webgl with some neat optimizations to render that many lines and circles.

Thanks a lot for including my site in your list. It was fun to see where it appeared on the map. It was pretty close to RealPython and GitHub.

Re: Network visualization of 50k blogs and links

#35

Post by 3abiton »

PaulHoule wrote: post #14

People still upvote hairball graphs every time. Fortunately there is a cure: https://cambridge-intelligence.com/how-to-fix-hairballs/

Is graph data processing considered visualization style? It is changing the data, how can this be considered "visualization"?

Re: Network visualization of 50k blogs and links

#37

Post by TuringNYC »

ng-henry wrote: post #2

I scraped my favorite blogs and made a graph from the domains that each blog links to. You can see clusters forming of websites that talk about similar topics, like crypto, rationality, Canada, India, and even postgres! The visualization was made entirely in webgl with some neat optimizations to render that many lines and circles.

> I scraped my favorite blogs and made a graph from the domains that each blog links to.

Nice analysis! However, I'm guessing these arent your fav blogs as there are tens of thousands of entries! How did you decide which blogs to index, did you use some central registry of blogs?

Post reply on HN