Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyGraphistry – A library to extract, transform, and visually explore big graphs

github.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#21

Post by ThePhysicist »

IanCal wrote: post #13
ThePhysicist wrote: post #5

Earlier quoted context omitted.

Is it possible to visualize graphs entirely on the client side (without sending any data to your backend)? We have some very large graphs that we'd like to explore, but unfortunately it's not possible to send the data to the cloud, hence a local solution would be great. I have investigated Gephi but unfortunately the performance is quite disappointing for very large graphs.

Not sure how large very large is or the performance you need but you might want to check out the fairly recently added graph support in datashader (examples: https://anaconda.org/jbednar/edge_bundling/notebook ) Here's an example of the output which I then put into tiles & rendered out: https://proseandcode.co.uk/beta_gtr_viz/ I think there's 600k edges in this, all rendered. I did some of the work on the edge bundli…

That looks impressive, will definitely check it out!

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#22

Post by lmeyerov »

bearjaws wrote: post #8
m3nu wrote: post #7

Another plot.ly pushing their freemium product via Github? No pricing on the homepage yet. Will come later, after enough people integrated it in their projects.

Anyone looking for an opensource version of plot.ly, take a look at http://www.metabase.com/

We're more complementary to Plotly & D3 than competitive. As an analogy for our developer API layers, we're closer to Google Maps than say D3 geo. Data tool teams shouldn't have to sink so much time on building out all the little interactions, just some configuring & skinning, and then back to data engineering & analytics. The result is we're getting embedded side-by-side. E.g., drop us into your Splunk/ELK dashboard that already does the basic charts. Or pilot our visual investigator, and get a direct visual exploration interface through a variety of connectors.

In terms of paid vs. unpaid, we're a pretty transparent team. We're iterating on a sustainable pricing model for enterprise investigation teams, advanced analysts, and developers building internal investigation tools. I don't believe in charging source developers, research scientists, non-profits, etc., and we'll be making our ongoing outreach work with those kinds of folks into a more formal program.

We have been and continue to be serious about community-minded open source contributions. Our team has contributed research that helped shape the modern web and leads open source projects that power a lot of tools used by the HN community today. Just this week, we released some work as part of the GoAI / Apache Arrow project, and that is part of our ongoing efforts to bring real GPU compute to the web world.

Maybe a good time to write -- we're hiring!

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#23

Post by lmeyerov »

rspeer wrote: post #19
btbuildem wrote: post #15

Earlier quoted context omitted.

https://linkurio.us/try/ should go straight to a demo page, not some signup popup

It goes to a signup popup. I'm interested to see if anyone's offering a large-scale graph visualization that's not the "pulsating rainbow hairball".

Our full investigation platform supports connectors like those, and to non-graph sources (Spark, APIs, ...). We found teams need more than just a graph db frontend, especially as we enable them to put in more data into a session than previous-gen technologies. Ex: we created visual playbooks for running repeat investigations and binding the result to an interactive visual analytics session.

We're actively piloting our investigation platform with interesting teams, so please reach out: info@gr...com.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#24

Post by mentatseb »

rspeer wrote: post #19
btbuildem wrote: post #15

Earlier quoted context omitted.

https://linkurio.us/try/ should go straight to a demo page, not some signup popup

It goes to a signup popup. I'm interested to see if anyone's offering a large-scale graph visualization that's not the "pulsating rainbow hairball".

Hi, I'm Gephi+Linkurious co-founder. I've found visualizing large graphs pretty useless beyond the "I see meatballs!" effect and my opinion, after a decade in the field, is that it's the wrong problem for data analytics.

Much more interesting information is discovered during the process of dynamically building a visualization that is focused on user questions. I see with Linkurious that investigators usually need to visualize less than 1000 edges of a 1M+ edges graph to get answers.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#25

Post by lmeyerov »

I realized this is a great time to say -- we're hiring!

If you're into data visualization / UI engineering, fullstack node for data/security, or enterprise security sales, would love to chat. Our team is mostly in the bay area. If you've worked remotely before, that works great too.

We're especially growing in the security market around incident response + hunt. Our engineering work is around establishing more scalable best practices for investigation teams, building out our fullstack app, and we're in the middle of our next GPU visual analytics initiatives (accelerating interactive visual analytics another 100X!). So a lot of good stuff happening.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#26

Post by lmeyerov »

mentatseb wrote: post #24
rspeer wrote: post #19

Earlier quoted context omitted.

It goes to a signup popup. I'm interested to see if anyone's offering a large-scale graph visualization that's not the "pulsating rainbow hairball".

Hi, I'm Gephi+Linkurious co-founder. I've found visualizing large graphs pretty useless beyond the "I see meatballs!" effect and my opinion, after a decade in the field, is that it's the wrong problem for data analytics. Much more interesting information is discovered during the process of dynamically building a visualization that is focused on user questions. I see with Linkurious that investigators usually need to…

The ultimate answer is generally a small graph: Graphistry is a tool that helps you get there. Why that's hard is most Splunk, Spark, etc. queries will return a bunch of events, and each event has a bunch of metadata. A tool should help, not fall over.

I think you're referring to scenarios closer to why we created the visual playbook concept and our embedding APIs. Small visualizations are often a good starting point in investigative scenarios. Even better.. no visualization, just full automation. We find this thinking comes up when the investigative flow is more established and curated. With visual playbooks, teams can record & automate multistep flows, run them whenever an incident happens, take action, and share & document the results. If part of the incident involves a bunch of events, or the analysts wants to dig in, our stack won't fall over. Instead, it provides a full visual analytics session with multiple cross-linked data views.

And we're fans of Gephi. We GPU accelerated the core algorithm -- we may be coming from a different perspective and user base.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#27

Post by lmeyerov »

stared wrote: post #6

Is it possible to run it locally? (i.e. without an API key)

Yep! Feel free to give the cloud version a go, and if it looks useful, we can help get you going on-premise. That would include our investigation platform as well.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#28

Post by mentatseb »

lmeyerov wrote: post #26
mentatseb wrote: post #24

Earlier quoted context omitted.

Hi, I'm Gephi+Linkurious co-founder. I've found visualizing large graphs pretty useless beyond the "I see meatballs!" effect and my opinion, after a decade in the field, is that it's the wrong problem for data analytics. Much more interesting information is discovered during the process of dynamically building a visualization that is focused on user questions. I see with Linkurious that investigators usually need to…

The ultimate answer is generally a small graph: Graphistry is a tool that helps you get there. Why that's hard is most Splunk, Spark, etc. queries will return a bunch of events, and each event has a bunch of metadata. A tool should help, not fall over. I think you're referring to scenarios closer to why we created the visual playbook concept and our embedding APIs. Small visualizations are often a good starting point…

Yup, it's important that people understand the role of visualization in the complete data chain.

Re: PyGraphistry – A library to extract, transform, and visually explore big graphs

#30

Post by rotten »

mentatseb wrote: post #28
lmeyerov wrote: post #26

Earlier quoted context omitted.

The ultimate answer is generally a small graph: Graphistry is a tool that helps you get there. Why that's hard is most Splunk, Spark, etc. queries will return a bunch of events, and each event has a bunch of metadata. A tool should help, not fall over. I think you're referring to scenarios closer to why we created the visual playbook concept and our embedding APIs. Small visualizations are often a good starting point…

Yup, it's important that people understand the role of visualization in the complete data chain.

I'm not sure I understand. Is there a resource that explains the role of visualizations in data flows in the context explained here?
Post reply on HN