Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is the best software to visualize a graph with a billion nodes?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 117 posts

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#51

Post by Xcelerate »

I might be prematurely classifying your question as an instance of the XY problem, but I worked at a company that tried to create something similar — a graph visualization system that could handle 100B nodes as part of our core product and... well... I would caution you not to do so if your purpose is something along those lines.

There's almost never a use case where a customer wants to see a gigantic graph. Or researchers. Or family members for that matter. People's brains just don't seem to mesh with giant graphs. Tiny graphs, sure. Sub-graphs that display relevant information, sure. The whole thing? Nah. Unless it's for an art project, in which case giant graphs can be pretty cool looking.

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#52

Post by surrTurr »

Cytoscape JS[1] with canvas rendering. Probably won't be able to do a billion nodes, but the last time I compared graph rendering libraries it was the best one in terms of performance/customizability. If you need even more performance, there's VivaGraphJS[2], which uses webgl to render.

If you want other resources, I also have a GitHub list of Graph-related libraries (visualizations etc.) on GitHub[3].

[1]: https://js.cytoscape.org/ [2]: https://github.com/anvaka/VivaGraphJS [3]: https://github.com/stars/AlexW00/lists/graph-stuff

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#53

Post by tgv »

aeonik wrote: post #50
infecto wrote: post #44

Earlier quoted context omitted.

This is the way imo. Nobody is consuming 100b nodes in a chart.

It really depends on what the nodes represent, right? A 1080p monitor has: 1,920 × 1,080 = 2,073,600 pixels Each pixel can display 32-bit color, which equates to: 2^32 = 4,294,967,296 colors So, while each pixel can display one of 4.3 billion colors, the monitor can display combinations of those colors across its 2,073,600 pixels. The total number of possible color combinations on the screen is astronomical. The actu…

That way of looking at it doesn't make sense. When visualizing a graph, you want to see the connections between the nodes; coloring each node individually almost never makes sense; and the eye cannot distinguish 4B colors.

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#54

Post by learn_more »

insomniacity wrote: post #8

Tacking on a related question - what software should one use to interactively create/update/see a small graph? Thinking specifically about a graph of knowledge, so will be an iterative process. Just looking for anything more than a text editor really!

https://schematix.com

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#55

Post by hombre_fatal »

aeonik wrote: post #50
infecto wrote: post #44

Earlier quoted context omitted.

This is the way imo. Nobody is consuming 100b nodes in a chart.

It really depends on what the nodes represent, right? A 1080p monitor has: 1,920 × 1,080 = 2,073,600 pixels Each pixel can display 32-bit color, which equates to: 2^32 = 4,294,967,296 colors So, while each pixel can display one of 4.3 billion colors, the monitor can display combinations of those colors across its 2,073,600 pixels. The total number of possible color combinations on the screen is astronomical. The actu…

You end up bucketing those colors into differences the human eye can see, so you end up with a much smaller domain.

You do something similar with 100B data points since you're not literally looking at the relation between individual nodes when all 100B are on screen at once.

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#56

Post by abeppu »

aeonik wrote: post #50
infecto wrote: post #44

Earlier quoted context omitted.

This is the way imo. Nobody is consuming 100b nodes in a chart.

It really depends on what the nodes represent, right? A 1080p monitor has: 1,920 × 1,080 = 2,073,600 pixels Each pixel can display 32-bit color, which equates to: 2^32 = 4,294,967,296 colors So, while each pixel can display one of 4.3 billion colors, the monitor can display combinations of those colors across its 2,073,600 pixels. The total number of possible color combinations on the screen is astronomical. The actu…

Aside from tgv's correct point that this is implicitly a recipe for something that isn't useful as a visualization, I think even if we were able to distinguish 4B colors and make sense of each pixel -> color assignment ... the math isn't on your side. You responded to a statement about nobody consuming a graph of 100B nodes. Suppose we don't have any concept of edge weight, and an edge is either present or not, but edges are directed, then you have 100B^2 (i, j) pairs for potential edges, each of which is either present or not (i.e. 10^22 edges, each of which is a bit).

4,294,967,296^2,073,600 is very large but 2^(10^22) is much much larger

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#57

Post by quietbritishjim »

viraptor wrote: post #16

It really feels like an under defined task. Do you actually need to see those nodes? At that scale, you never want to render 100B of them. Instead you would need some kind of density aggregation when zoomed out and moving to LoD style k-d tree partitioning when zoomed in. That's almost the area of rendering engines like Unreal's Nanite. You can create your own renderer for data like this, but game engines are likely…

You're right, but I think that may be what the OP is actually asking for. They talk of "zooming out" but I don't think they mean so they can literally zoom out and see all 100b nodes individually on their screen at once but instead mean that some high level / clustered view is shown to give an overview.

That being the case, I think you're suggesting that this high level summarisation happens as a separate preprocessing step (which I agree with FWIW) whereas I think they're imagining it happening dynamically as part of rendering.

Re: Ask HN: What is the best software to visualize a graph with a billion nodes?

#58

Post by slightwinder »

viraptor wrote: post #16

It really feels like an under defined task. Do you actually need to see those nodes? At that scale, you never want to render 100B of them. Instead you would need some kind of density aggregation when zoomed out and moving to LoD style k-d tree partitioning when zoomed in. That's almost the area of rendering engines like Unreal's Nanite. You can create your own renderer for data like this, but game engines are likely…

> Do you actually need to see those nodes?

Even 8k-screens have not enough pixel to show that many nodes at the same time. So some visual optimization has to happen anyway.

Post reply on HN