Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NetworkX 3.0 - create, manipulate, and study complex networks in Python

networkx.org

Post reply on HN
Print view View original

41–50 of 58 posts

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#41

Post by mcyc »

veducha wrote: post #29
mcyc wrote: post #21

NetworkX is my go-to library for helping people get started with open source (like mentoring new devs that want git practice, practice working in a large codebase, etc.). I feel it is laid out in an easy to understand way and all the main parts do one clear thing that can be understood even with only a basic knowledge of graph theory. It also hits the sweet spot of being an obviously useful and widely used package, b…

Do you have any recommendations on how to get started? I have been using open source for around 10 years and always wanted to contribute. I am a confident python scripter and have used NetworkX in the past for various academic projects, however I have no idea how to get started as a beginner dev and contribute to open issues and bugs. Heck, I don't even know how large codebases are structured! Any hints or recommenda…

If projects have a "good first issue" tag, they can sometimes be a good place to start. The problem is they are mostly "add tests" or "expand docs", which are important but not very exciting.

Imo the best thing is to have a mentor who hopefully can find a reasonable thing to improve/add/fix and can help calm your nerves (submitting a pull request can be kinda scary haha). With NetworkX, in the past there were a lot of minor inefficiencies (things like materializing a list comprehension when you don't need to, etc) that are 1) easy to understand 2) don't require in depth knowledge of graph theory 3) are generally applicable to all python code and 4) easy to measure the results. They aren't the flashiest contributions, but the people I've done it with (6 or 7 at this point) have all seemed to enjoy it.

To be clear, I'm _not_ affiliated with NetworkX, but if you are interested in trying it out, my contact info is in my profile. Feel free to reach out.

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#42

Post by samsquire »

samsquire wrote: post #24

I use NetworkX rather unusually to parse Graphviz dot specification files in my devops pipeline tool, mazzle. They allow me to define infrastructure build instructions as a dot file that looks similar to this. It's similar to an advanced makefile. This segment of the infrastructure specification sets a dependency on the bastion box, the vault AMI (Amazon Machine Image) and web server provisioned by terraform on the s…

I should add, I also use the resulting dependency graph to parallelise the build.

So if there's two packer builds that do not depend on eachother, they can run in parallel since they are slow.

https://github.com/samsquire/devops-schedule https://github.com/samsquire/parallel-workers

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#43

Post by dallathee »

veducha wrote: post #29
mcyc wrote: post #21

NetworkX is my go-to library for helping people get started with open source (like mentoring new devs that want git practice, practice working in a large codebase, etc.). I feel it is laid out in an easy to understand way and all the main parts do one clear thing that can be understood even with only a basic knowledge of graph theory. It also hits the sweet spot of being an obviously useful and widely used package, b…

Do you have any recommendations on how to get started? I have been using open source for around 10 years and always wanted to contribute. I am a confident python scripter and have used NetworkX in the past for various academic projects, however I have no idea how to get started as a beginner dev and contribute to open issues and bugs. Heck, I don't even know how large codebases are structured! Any hints or recommenda…

A good place to start specifically for NetworkX would be to go through the new contributor documentation: https://networkx.org/documentation/latest/developer/new_cont...

We also have some structured projects https://networkx.org/documentation/latest/developer/projects... but they are usually for programs like GSoC/Outreachy.

Feel free to start a discussion https://github.com/networkx/networkx/discussions if you are looking for something specific :)

[I am one of the NetworkX devs]

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#44

Post by dallathee »

tekkk wrote: post #34

Is it now finally faster than GraphLab? Couple years ago when I last used them, there were some operations like PageRank that was magnitudes faster with GraphLab. Hope they have caught up now. GraphLab has been deprecated for a long while now. Curious did or will they ever make it open-source.

> Is it now finally faster than GraphLab?

NetworkX(pure python with dict as the data structure) itself will never be faster than any C++ based library (I think GraphLab doesn't even exist now?) but with the new experimental backend plugins, you should be able to use NetworkX API and dispatch the computation to more efficient backends like GraphBLAS, and hopefully CuGraph in the future (if you have GPUs). A bit more info: https://twitter.com/networkx_team/status/1612478129649459202

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#45

Post by dallathee »

college_physics wrote: post #13

wow, great to see that networkx keeps evolving. the graphblas backend is particularly intriguing. does networkx expose the graphblas API at the python level or is it working behind the scenes for computation?

GraphBLAS is wrapped by https://github.com/python-graphblas/python-graphblas/ and the algorithms are available at https://github.com/python-graphblas/graphblas-algorithms. NetworkX only dispatches the computation for a subset of algorithms to graphblas-algorithms right now.

If you want the graphblas API in python, https://github.com/python-graphblas/python-graphblas/ is the right place :)

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#46

Post by VectorLock »

samsquire wrote: post #24

I use NetworkX rather unusually to parse Graphviz dot specification files in my devops pipeline tool, mazzle. They allow me to define infrastructure build instructions as a dot file that looks similar to this. It's similar to an advanced makefile. This segment of the infrastructure specification sets a dependency on the bastion box, the vault AMI (Amazon Machine Image) and web server provisioned by terraform on the s…

Nice I like the concept of turning around Graphviz dot files and turning them from an intermediate representation of visualization into the actual source of truth for the system. Cool idea.

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#47

Post by math_dandy »

Xcelerate wrote: post #28

I just started using NetworkX a few days ago and was trying to figure out a way to find all paths in a DAG from a source node to a destination node, with the requirement that no path is a superpath of any other path. I ended up writing my own approach that iterates through all pairs of paths but have been bothered since then that I couldn’t find a function in NetworkX to do this; I’m sure it probably exists but I don…

Will nx.all_simple_paths do the job? (A simple path is a path with no repeated nodes.)

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#48

Post by GraphLover9000 »

taubek wrote: post #40
GraphLover9000 wrote: post #39

Earlier quoted context omitted.

In general! Graph drawing and interactivity is an interesting problem in my opinion.

Maybe this will be interesting to you. An open source graph visualization library called Orb.[1] There is a series of blog post that talk about the development and reasoning behind this library. [2] [1] https://github.com/memgraph/orb [2] https://memgraph.com/blog?topics=Orb#list

Thank you! I'll take a look.

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#49

Post by jim22k »

I presented the new dispatching feature at PyData Global back in December. The first 30 minutes of the follow video: https://youtu.be/pd-Sbm8lHTc?t=46

It shows how to use the feature, some benchmarks, and how other graph libraries can become plugins to the NetworkX dispatching mechanism.

Re: NetworkX 3.0 - create, manipulate, and study complex networks in Python

#50

Post by anigbrowl »

GraphLover9000 wrote: post #35
anigbrowl wrote: post #11

By the way if you want to explore network science but don't really know where to start, consider Gephi (currently being refactored, and just updated a few days ago) or Cytoscape (if you're more drawn to bioinformatics). Both make it easy to load/generate standard datasets, import tabular data, and have a good selection of plugins. It's easy to kick stuff out to either from NetworkX using /gefx or graphml, and you'll…

Do you have any advice/ideas about ways to learn about graph layout algorithms themselves, including dynamic/real-time algorithms (which allow for user interaction)? I have been skimming through various papers and the first book on this page [0], in particular the chapter on force directed algorithms, because they seem to be the earliest and most general graph drawing methods. [0] http://graphdrawing.org/books.html

It's kind of a mysterious art, and I too mostly rely on scientific papers. It's a surprisingly small field and you need to get into the habit of chasing down citations in papers, because many important ideas got laid out long before the computer power existed to realize them at scale. Sometimes a 20-30 year old paper of only a few pages has the actual algorithm, and it's so well known in the field that it no longer stands out in more recent papers.

Here's a few useful references:

Good overview on big graphs: https://towardsdatascience.com/large-graph-visualization-too...

A gallery of large graphs - horrid user interface, but you can click through and find an absolute wealth of resources. Curated by Yifan Hu, who developed one of the popular layout algorithms: http://yifanhu.net/GALLERY/GRAPHS/

Graphviz is a very well-documented library with a lot of the 'classic' layouts.

Astronomy, physics, and bio people have a lot of useful visualization tools and techniques for huge datasets, but you will have to go looking for them - not because they don't like to share, but because they mostly write to each other so you won't just land on stuff by browsing Github. Absolute must-have literature review: https://arxiv.org/abs/2110.01866

A lot of large graph visualization techniques are about using simple graph visualization techniques but first combing out the hairballs through the application of dimensionality reduction, motif extraction, backbone identification and so on. This is an important paper whose techniques have yet to be fully explored: https://jgaa.info/accepted/2015/NocajOrtmannBrandes2015.19.2...

For a combination of theoretical and practical reasons, most visualization zeroes in on rendering smallish graphs in 2 dimensions. Large graphs are either so densely connected as to be be intractable (the brain being the ultimate hairball) or so sparse as be like digital planetariums - gorgeous, impressive, and looking much the same in every direction.

I could go on at length but as you can maybe guess I'm a consumer of other people's research rather than an expert in implementing the fundamentals. Also I don't have any academic background whatsoever so I apologize for the haphazard infodump. I've been studying/applying stuff from this field for ~15 years but it's too out there for most people. Feel free to email though.

Post reply on HN