I'm curious what makes it for embedding large graphs - afaict the core is a spring model implemented as O(n^2) in nodes and single-threaded? Wasn't sure of the _size param for the nested loop, maybe I'm misreading. Was hoping there'd be something fun to solve the asymptotic scaling limit via fast multipole, NN's, etc :)
Show HN: Python library for embedding large graphs (Written in Rust)
11–20 of 26 posts
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#12I like how this is a Rust project without any Python, that publishes a package to PyPI.
It uses Maturin (https://github.com/PyO3/maturin) for this, which I've never heard of but sounds really useful.
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#13I like how this is a Rust project without any Python, that publishes a package to PyPI. It uses Maturin ( https://github.com/PyO3/maturin ) for this, which I've never heard of but sounds really useful.
This is really nice, I have been looking into learning Rust or Cython to speed up some critical paths. Does anyone know if there’s any overhead to using rust vs cython for extending python?
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#14I like how this is a Rust project without any Python, that publishes a package to PyPI. It uses Maturin ( https://github.com/PyO3/maturin ) for this, which I've never heard of but sounds really useful.
In my experience, the whole `PyO3`-project is a rare gem in language integrations since the developers appear to have an intricate understanding of the ecosystems on both side of the integration.
The python parts are pythonic and the rust parts are "rusty". A joy to work with.
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#15Can someone point me to some resources on why you would want to embed graphs?
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#16Can someone point me to some resources on why you would want to embed graphs?
[deleted]
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#17I like how this is a Rust project without any Python, that publishes a package to PyPI. It uses Maturin ( https://github.com/PyO3/maturin ) for this, which I've never heard of but sounds really useful.
In my experience, the whole `PyO3`-project is a rare gem in language integrations since the developers appear to have an intricate understanding of the ecosystems on both side of the integration. The python parts are pythonic and the rust parts are "rusty". A joy to work with.
It probably does help that both Python and Rust treat FFI as first-class in terms of importance. That said, yeah, PyO3 is a pretty awesome suite to work with.
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#18[deleted]
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#19A lib generating 2D artifacts might benefit from showcasing these artifacts using image technology for the curious minds.
Thanks for the feedback. The library only calculates the positions and has no graphical output. As an example: I created this to build this embedding of Mastodon instances. https://h4kor.github.io/fediverse-explorer/
I did also want to see a picture, presumably the point of the lib is to make a nice layout of nodes even though that then has to be rendered with another tool?
Re: Show HN: Python library for embedding large graphs (Written in Rust)
#20Can someone point me to some resources on why you would want to embed graphs?
graph embedding is super useful for both viz + decision procedures
- Viz: Embedding gives x/y coords, and if they rendered the edges, a traditional graph view. A cool thing about recent shift to embedding approaches to the graph drawing problem is optimizing for objective functions that 1980's style spring layouts don't -- think the same things a neural network would optimize for. The code here appears more useful for small/medium graphs (ex: some ec2 tenant diagram) and I didn't see the neural network stuff, but with work, you can scale up several of the pipeline steps to handle 100X+ bigger ones (ex: we work with a lot of fraud or cyber event logs), and in headless cases, ~billion scale. It's a cool new subfield, google "graph drawing neural network".
- Decisions: Node, edge, and subgraph/graph embeddings are all super useful. I'm giving a talk at Friday's Infosec Jupyterthon (https://infosecjupyterthon.com/introduction.html) on the link prediction case for ~identity protection & resource ~monitoring (account takeover, insider threat, rogue devices, data leakage, ...) by mining log data, and as another example, link prediction is basically recsys, which is how any site with a shopping cart makes more money. Node classification, graph motif mining, etc are different but the same. Search for one of the many introductions to graph neural networks for a technical perspective, and I co-authored this survey at the beginning of the year to give a market perspective: https://gradientflow.com/what-is-graph-intelligence/
All this comes up a bunch in cyber/fraud/retail/supplychain -- we're certainly busy there. For anyone into that, we're hiring someone to own a bunch of backend/infra (k8s/gpu cloud/enterprise), 1-2 cleared folks in DC, and in Q1, (graph) data scientists. Simple question but one we're really into :)