Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm Lukewarm on Graph Neural Networks

singlelunch.com

Post reply on HN
Print view View original

21–30 of 50 posts

Re: Why I'm Lukewarm on Graph Neural Networks

#21

Post by vladTheInhaler »

lostmsu wrote: post #5

By no means I am an expert in deep learning, but lately I've been considering graph NNs snake oil of neural networks. There are no impressive results on any common tasks and that picture is supported by a recent paper claiming transformers to "contain a graph network inside".

A bunch of fully connected layers also "contains a CNN inside", but empirically CNNs lead to better performance on image classification tasks. When you can cut away an enormous, and mostly irrelevant part of the parameter space, that has a lot of value.

Re: Why I'm Lukewarm on Graph Neural Networks

#22

Post by VHRanger »

dumb1224 wrote: post #10
VHRanger wrote: post #8

Earlier quoted context omitted.

So are graph embedding methods, which the post discusses

The post can't be accessed with a Internal Server Error. Or is it just my ISP? I often confuse myself with other representation learning methods such as graph kernels https://en.wikipedia.org/wiki/Graph_kernel Are there any relations between them at all?

No, my blog went down from hitting the front page of HN

Re: Why I'm Lukewarm on Graph Neural Networks

#23

Post by VHRanger »

sischoel wrote: post #13

I don't know much about graph neural networks although it is a topic that I want to study in the next few months. But what bothered me in your article is what you wrote about graph data structures. NetworkX is indeed very slow, this is due to two facts: - NetworkX is a pure Python implementation and does not relay on some methods written in a faster language like C. - They use dictionaries to represent the graphs, wh…

Right, the data structure depends on the tradeoff being made.

Do you add or remove nodes as often as you traverse the graph? Then a dictionary/network-of-pointers make sense.

But generally, if you're doing that a lot, a database also makes sense which is why I don't like the tradeoff NetworkX made at all.

If you don't add nodes often at all, but traverse from edge-to-edge a lot, a CSR representation makes sense.

Adjacency array representations have other tradeoffs vis-a-vis CSR representaitons and are kind of an in-between solution.

Re: Why I'm Lukewarm on Graph Neural Networks

#24

Post by dallathee »

VHRanger wrote: post #20
mrtranscendence wrote: post #19

NetworkX isn't a bad library, nor is it only suitable for "babies". Yeesh. Actual people work on this stuff, you know, and they may have different goals, requirements, etc than you. This whole post reeks of someone who's so in love with being a maverick speaking uncomfortable truths that they've lost sight of this human element. If you're offended by (say) research papers that fail to break enough new ground to satis…

I never said NX is bad, but it is for "baby" graphs. NX can't scale past a few hundred thousand nodes. I appreciate NX's place in the ecosystem, but its implementation leaves a large gap to be filled for an intermediate library that is a Pandas analogue for graphs.

There is igraph, graph-tool, snap and even a CUDA enabled one (cuGraph, they try to follow networkx API).

Re: Why I'm Lukewarm on Graph Neural Networks

#25

Post by pred_ »

VHRanger wrote: post #20
mrtranscendence wrote: post #19

NetworkX isn't a bad library, nor is it only suitable for "babies". Yeesh. Actual people work on this stuff, you know, and they may have different goals, requirements, etc than you. This whole post reeks of someone who's so in love with being a maverick speaking uncomfortable truths that they've lost sight of this human element. If you're offended by (say) research papers that fail to break enough new ground to satis…

I never said NX is bad, but it is for "baby" graphs. NX can't scale past a few hundred thousand nodes. I appreciate NX's place in the ecosystem, but its implementation leaves a large gap to be filled for an intermediate library that is a Pandas analogue for graphs.

From the post:

> NetworkX is a bad library.

Re: Why I'm Lukewarm on Graph Neural Networks

#26

Post by beaupletga »

I need to handle (for work) a graph with 40 million nodes and more than 130 million edges. As expected, networkx couldn't handle more than a million nodes so I had to search for python libs which might handle that much data.

This is why I've been using your lib (https://github.com/VHRanger/nodevectors) for at least 2 weeks now as well as these 2 other libs: https://github.com/louisabraham/fastnode2vec and https://github.com/sknetwork-team/scikit-network. What do they have in common? They handle sparse graphs (using CSR representations).

Having a graph with several million nodes isn't just some edge case, social graph for instance grow way faster than anyone could expect. So I do totally agree that there is a gap between popular graph libs which handle very small graphs and real life libs which need to handle way bigger graphs.

Btw, thanks for the work you've done with 'nodevectors'.

PS: I'm not criticizing either networkx which is very handy and quite good when prototyping a solution.

Re: Why I'm Lukewarm on Graph Neural Networks

#27

Post by dimatura »

mrtranscendence wrote: post #19

NetworkX isn't a bad library, nor is it only suitable for "babies". Yeesh. Actual people work on this stuff, you know, and they may have different goals, requirements, etc than you. This whole post reeks of someone who's so in love with being a maverick speaking uncomfortable truths that they've lost sight of this human element. If you're offended by (say) research papers that fail to break enough new ground to satis…

I have to say I agree. NX is totally fine for the applications it was built for. I appreciate the simple API and the flexibility of using pure Python. There's a reason it's so popular.

I'm not an expert in graph neural networks, so can't really say much about the novelty of Node2Vec. But I do think it's often misguided to judge scientific work as trivial or incremental in retrospect. Specially in a relatively young field like deep learning, where four years (Node2Vec is from 2016) is a long time.

Re: Why I'm Lukewarm on Graph Neural Networks

#28

Post by jedbrown »

sischoel wrote: post #13

I don't know much about graph neural networks although it is a topic that I want to study in the next few months. But what bothered me in your article is what you wrote about graph data structures. NetworkX is indeed very slow, this is due to two facts: - NetworkX is a pure Python implementation and does not relay on some methods written in a faster language like C. - They use dictionaries to represent the graphs, wh…

The index type for CSR need only index nodes. It's the offsets (one per row) that need to index edges.

The blog's portrayal of NVMe vs distributed memory is a bit off:

> With modern NVMe drives random seeks aren’t slow anymore, much faster than distributed network calls like you do when scaling the linked list-based graph.

Distributed memory machines has 1 microsecond off-node latency and 15-20 microsecond reductions (with thousands of nodes). These latencies have been pretty flat for the past 20 years in supercomputing, and have been widely available at cloud providers for the past few years. NVMe is not as fast as it's made out to be. https://twitter.com/SwiftOnSecurity/status/99079794835421184...

The other issue is that NVMe is more expensive than DRAM or HBM when you care about bandwidth (and graph/network analysis is almost always memory bandwidth limited, even on HBM). Suppose you need to do 1000 traversals of a 1 TB data structure. This takes about 10 seconds with 100 HBM devices ($1 at on-demand pricing, assuming amortized startup), but days of saturating NVMe bandwidth.

Re: Why I'm Lukewarm on Graph Neural Networks

#29

Post by sischoel »

jedbrown wrote: post #28
sischoel wrote: post #13

I don't know much about graph neural networks although it is a topic that I want to study in the next few months. But what bothered me in your article is what you wrote about graph data structures. NetworkX is indeed very slow, this is due to two facts: - NetworkX is a pure Python implementation and does not relay on some methods written in a faster language like C. - They use dictionaries to represent the graphs, wh…

The index type for CSR need only index nodes. It's the offsets (one per row) that need to index edges. The blog's portrayal of NVMe vs distributed memory is a bit off: > With modern NVMe drives random seeks aren’t slow anymore, much faster than distributed network calls like you do when scaling the linked list-based graph. Distributed memory machines has 1 microsecond off-node latency and 15-20 microsecond reductions…

Yes, I had something mixed up here - you are completely right about CSR matrices, they should not use more memory than an adjacency list.
Post reply on HN