Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm Lukewarm on Graph Neural Networks

singlelunch.com

Post reply on HN
Print view View original

31–40 of 50 posts

Re: Why I'm Lukewarm on Graph Neural Networks

#31

Post by jim22k »

enriquto wrote: post #17
sischoel wrote: post #13

I don't know much about graph neural networks although it is a topic that I want to study in the next few months. But what bothered me in your article is what you wrote about graph data structures. NetworkX is indeed very slow, this is due to two facts: - NetworkX is a pure Python implementation and does not relay on some methods written in a faster language like C. - They use dictionaries to represent the graphs, wh…

Indeed, the best support for large graphs in Python is found inside scipy.sparse

Anyone who is serious about graph analysis in Python using linear algebra should check out GraphBLAS (graphblas.org). The inclusion of semirings makes things like SSSP and BFS extremely elegant on sparse adjacency matrices.

The main implementation is SuiteSparse::GraphBLAS, a C library which has two Python bindings (search for grblas or pygraphblas). Disclosure: I'm the author of grblas. Both are available on conda-forge for easy installation.

If you want to try it out without installing, here is a binder link: https://mybinder.org/v2/gh/metagraph-dev/grblas/HEAD?filepat...

Re: Why I'm Lukewarm on Graph Neural Networks

#32

Post by sanxiyn »

FL33TW00D wrote: post #30

Interesting post, I’m working on a whole graph classification problem at the moment. If GNNs aren’t the future, then where else should I be looking?

You should look at label propagation. Combining Label Propagation and Simple Models Out-performs Graph Neural Networks: https://arxiv.org/abs/2010.13993

Re: Why I'm Lukewarm on Graph Neural Networks

#35

Post by sooheon »

sanxiyn wrote: post #32
FL33TW00D wrote: post #30

Interesting post, I’m working on a whole graph classification problem at the moment. If GNNs aren’t the future, then where else should I be looking?

You should look at label propagation. Combining Label Propagation and Simple Models Out-performs Graph Neural Networks: https://arxiv.org/abs/2010.13993

Linked paper is focused on node classification, parent is asking for whole graph prediction.

Re: Why I'm Lukewarm on Graph Neural Networks

#36

Post by sooheon »

leecarraher wrote: post #11

I suspect we will not see a revolution in deep learning from GNN without a corresponding hardware(FPGA?) or optimization method(HSIC? HTMs?) advance. GNNs trained by backprop are making many of the same mistakes that LSTMs did: solve one important problem(exploding/vanishing gradients), but introduce a bunch of hyperparameters that bring along their own set of problems. GRUs are successful in my opinion, because they…

That hardware advance may be closer than expected: https://www.graphcore.ai

Re: Why I'm Lukewarm on Graph Neural Networks

#37

Post by Der_Einzige »

lostmsu wrote: post #5

By no means I am an expert in deep learning, but lately I've been considering graph NNs snake oil of neural networks. There are no impressive results on any common tasks and that picture is supported by a recent paper claiming transformers to "contain a graph network inside".

So wrong oh my God!

This beauty got an outstanding paper award at COLING 2020.

GNNs are not applied to "common tasks" because they are primarily suited for environments where graph structure is present. In this case, predicting how the Indian congress will vote.

https://www.aclweb.org/anthology/2020.coling-main.426/

Re: Why I'm Lukewarm on Graph Neural Networks

#39

Post by lmeyerov »

enriquto wrote: post #17
sischoel wrote: post #13

I don't know much about graph neural networks although it is a topic that I want to study in the next few months. But what bothered me in your article is what you wrote about graph data structures. NetworkX is indeed very slow, this is due to two facts: - NetworkX is a pure Python implementation and does not relay on some methods written in a faster language like C. - They use dictionaries to represent the graphs, wh…

Indeed, the best support for large graphs in Python is found inside scipy.sparse

I've found sparse good for many-columns (10K+). However, for graphistry's graph workloads, we do most scipy-size graphs at much higher speeds via cudf + cugraph. Similar apis, more speed.

Ex: Preprocess many cols with scikit sparse (genomes, many-featured fraud, ...) -> cudf (= gpu dataframes) for most of workload -> infer relationships with gpu UMAP then gpu k-nn -> graphistry for visual analysis.

We're starting to experiment with bigger-than-memory (TB etc) scale via the gpu-enabled dask libs. I definitely recommend trying. If interested in adding visual analytics here, these'll be hitting our free + self-hosted layers soon, and drop a contact method if you'd like early access. Exciting times!

Re: Why I'm Lukewarm on Graph Neural Networks

#40

Post by eachro »

Has the author looked at how GNNs are used in say predicting molecular properties[1] or molecule generation[2]? GNNs are the state of the art on those tasks and those problems are certainly much harder than the Cora/Citeseer/etc semisupervised node label prediction tasks.

[1] https://arxiv.org/abs/1704.01212

[2] http://proceedings.mlr.press/v80/jin18a/jin18a.pdf

Post reply on HN