Live data from Hacker News

PyGraphistry – A library to extract, transform, and visually explore big graphs

github.com

11–20 of 30 posts

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#11
post #10

Does anyone know a good tool to visually explore large ontology-type graphs?

You may want to look into Linkurious: https://linkurio.us/ We provide a graph visualization interface that can connect to Neo4j, DataStax (DSE Graph), Titan or Allegrograph.

Disclaimer: I'm a co-founder of the company

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#12
Personally from using it, it goes well beyond just visualization of large graphs. Its not readily useful to see so many nodes, but what they've done is made it easier for people to parse through large graphs without a lot of hand holding. I would point out the work of others in this field, like Marc Khoury (which @MurrayHill1980 has already kindly mentioned), but this points to making large graphs usable for a variety of folks in my opinion.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#13
post #2

Fun to see a link here! Always happy to describe what's happening underneath w/ connecting GPUs in the browser to GPUs in the datacenter. Likewise, the connection between event data & graph analytics is powerful as data scales, so happy to dig into that too. Not shown there, we're piloting a 'visual playbook' investigation layer to help teams who investigate through a lot of event data. This has been especially relev…

Is it possible to visualize graphs entirely on the client side (without sending any data to your backend)? We have some very large graphs that we'd like to explore, but unfortunately it's not possible to send the data to the cloud, hence a local solution would be great. I have investigated Gephi but unfortunately the performance is quite disappointing for very large graphs.

Not sure how large very large is or the performance you need but you might want to check out the fairly recently added graph support in datashader (examples: https://anaconda.org/jbednar/edge_bundling/notebook)

Here's an example of the output which I then put into tiles & rendered out: https://proseandcode.co.uk/beta_gtr_viz/

I think there's 600k edges in this, all rendered.

I did some of the work on the edge bundling, the split out code is here: https://gitlab.com/ianjcalvert/edgehammer

All of this is more work than just loading it into a current program though, but it might be a useful component of what you need, hopefully a large useful part.

Get in touch if I can help out.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#14
post #2

Fun to see a link here! Always happy to describe what's happening underneath w/ connecting GPUs in the browser to GPUs in the datacenter. Likewise, the connection between event data & graph analytics is powerful as data scales, so happy to dig into that too. Not shown there, we're piloting a 'visual playbook' investigation layer to help teams who investigate through a lot of event data. This has been especially relev…

Is it possible to visualize graphs entirely on the client side (without sending any data to your backend)? We have some very large graphs that we'd like to explore, but unfortunately it's not possible to send the data to the cloud, hence a local solution would be great. I have investigated Gephi but unfortunately the performance is quite disappointing for very large graphs.

[deleted]

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#15
post #10

Does anyone know a good tool to visually explore large ontology-type graphs?

You may want to look into Linkurious: https://linkurio.us/ We provide a graph visualization interface that can connect to Neo4j, DataStax (DSE Graph), Titan or Allegrograph. Disclaimer: I'm a co-founder of the company

https://linkurio.us/try/ should go straight to a demo page, not some signup popup

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#16
post #2

Fun to see a link here! Always happy to describe what's happening underneath w/ connecting GPUs in the browser to GPUs in the datacenter. Likewise, the connection between event data & graph analytics is powerful as data scales, so happy to dig into that too. Not shown there, we're piloting a 'visual playbook' investigation layer to help teams who investigate through a lot of event data. This has been especially relev…

Is it possible to visualize graphs entirely on the client side (without sending any data to your backend)? We have some very large graphs that we'd like to explore, but unfortunately it's not possible to send the data to the cloud, hence a local solution would be great. I have investigated Gephi but unfortunately the performance is quite disappointing for very large graphs.

Indeed, most of our users run on-premise / air-gapped. I recommend trying our cloud API to make sure it's in the right ballpark, and then we can get you setup!

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#17
post #13

Earlier quoted context omitted.

Is it possible to visualize graphs entirely on the client side (without sending any data to your backend)? We have some very large graphs that we'd like to explore, but unfortunately it's not possible to send the data to the cloud, hence a local solution would be great. I have investigated Gephi but unfortunately the performance is quite disappointing for very large graphs.

Not sure how large very large is or the performance you need but you might want to check out the fairly recently added graph support in datashader (examples: https://anaconda.org/jbednar/edge_bundling/notebook ) Here's an example of the output which I then put into tiles & rendered out: https://proseandcode.co.uk/beta_gtr_viz/ I think there's 600k edges in this, all rendered. I did some of the work on the edge bundli…

I'm a fan of the continuum team - we have been working together on the GPU GoAI project. We'll load 2M+ edges, and edge bundling helps make that visually useful.

Graphistry is a bit different where the result is a full interactive visual analytics session, not a zoomable png. So you get visual filtering, histogramming, search, etc. Our goal is to get from a lot of data to the answer, including whatever data pivots/cleaning/etc., as fast as possible, and that includes helping analysts skip a lot of the visualization/data coding and instead do direct visual interactions.

Worth stating: Edge bundling is beautiful! It was an early algorithm we implemented. A bit differently, we did it interactively, so you could do things like adjust sliders real-time to get the right physics settings. However, we found you'd want to hover over individual nodes/edges to see what's in them, and "zoomable image" style makes that hard. I've been wanting to bring it back now that we're getting close to supporting dynamic grouping interactions, so cool to see you call it out.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#18
post #4
post #2

Fun to see a link here! Always happy to describe what's happening underneath w/ connecting GPUs in the browser to GPUs in the datacenter. Likewise, the connection between event data & graph analytics is powerful as data scales, so happy to dig into that too. Not shown there, we're piloting a 'visual playbook' investigation layer to help teams who investigate through a lot of event data. This has been especially relev…

Is it possible to use a local GPU from the machine that has the data? Kind of like Gephi?

Yep! And we're working towards multi-GPU boxes!

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#19

Earlier quoted context omitted.

You may want to look into Linkurious: https://linkurio.us/ We provide a graph visualization interface that can connect to Neo4j, DataStax (DSE Graph), Titan or Allegrograph. Disclaimer: I'm a co-founder of the company

https://linkurio.us/try/ should go straight to a demo page, not some signup popup

It goes to a signup popup.

I'm interested to see if anyone's offering a large-scale graph visualization that's not the "pulsating rainbow hairball".

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#20

Earlier quoted context omitted.

Is it possible to visualize graphs entirely on the client side (without sending any data to your backend)? We have some very large graphs that we'd like to explore, but unfortunately it's not possible to send the data to the cloud, hence a local solution would be great. I have investigated Gephi but unfortunately the performance is quite disappointing for very large graphs.

Graphistry is impressive, but not free, and needs their server and an API key. You probably already know about graphviz sfdp (not interactive). There's been interesting recent work on distributed maxent optimization https://arxiv.org/pdf/1506.04383.pdf and sparse stress models https://arxiv.org/abs/1608.08909 (see its references as well, for example, Khoury's MARS is still available at https://github.com/marckhoury/m…

Those projects are pretty exciting. We accelerated FA2 and are now adding acceleration to the surrounding analytics interactions. And yep, to support an eco-system here, we've been iterating on a sustainable & scalable model (... "enterprise").

To that end, we have a variety of non-profits & scientists using our cloud tier. We want to streamline that program in the coming months: If that's you today, we'd love to help, so feel free to reach out for early access. For example, we've been excited by data science and cybersecurity schools, journalists, and scientists.

Post reply on HN