Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Topological Deep Learning: A Survey on Topological Neural Networks

arxiv.org

Post reply on HN
Print view View original

11–16 of 16 posts

Re: Topological Deep Learning: A Survey on Topological Neural Networks

#11

Post by mplemay »

chaxor wrote: post #8
dpflan wrote: post #2

I am not well versed in topological deep learning; but can topological concepts be learned by a graph neural network -- like can there a be topological layer(s) that learns the topology of the graphs, like how it's easy to visualize how a CNN architecture can see different hierarchical patterns at deep depths, seemingly learning the greater complexities of a image representation and understanding?

Yeah this is being done by several groups across the field Check out the work by Gunnar Carlsson and Bastien Reick. If those are enjoyable also check out Micheal Bronstein and Petar Velolickovic, although not directly related to your question.

Do you mind linking the homepage, google scholar, etc. of "Petar Velolickovic"? The only reference to the name on google search appears to be this post.

Re: Topological Deep Learning: A Survey on Topological Neural Networks

#12

Post by dpflan »

lmeyerov wrote: post #6
zwaps wrote: post #5

Earlier quoted context omitted.

I agree that it's exciting to move toward a mathematical approach to characterize what can, and what can not be learned by a DNN! Maybe someone else has the knowledge to give you some more precise answers. As far as I understand we need to differentiate between higher order relations that can be learned by message parsing and those that cannot. Some higher order relationships can be represented in a graph (of pairwis…

Agreed, the definitions here of what a GNN is (=> cannot do) seem pedantically narrow & extreme, and then throws the baby out w the bath water See last ~2 years of work by Michael Bronstein's teams (who moonlights as Twitter's ex? GNN r&d lead) that address 2+ of the points here It's true they are not the end-all, but our issues in practice aren't the above, but more like how to best combine temporal deep learning te…

Interesting. Could one graph be the topological representation, let's say like SCC? So there are heterogenous graphs with meta-graphs connected to them...?

Re: Topological Deep Learning: A Survey on Topological Neural Networks

#13

Post by dpflan »

mplemay wrote: post #11
chaxor wrote: post #8

Earlier quoted context omitted.

Yeah this is being done by several groups across the field Check out the work by Gunnar Carlsson and Bastien Reick. If those are enjoyable also check out Micheal Bronstein and Petar Velolickovic, although not directly related to your question.

Do you mind linking the homepage, google scholar, etc. of "Petar Velolickovic"? The only reference to the name on google search appears to be this post.

- https://geometricdeeplearning.com/ -- group of researchers presenting the paradigm

- https://petar-v.com/ - Petar's homepage

- https://scholar.google.co.uk/citations?user=kcTK_FAAAAAJ&hl=... - Petar's Google Scholar

- https://www.linkedin.com/in/petarvelickovic/ - Petar's LI

Re: Topological Deep Learning: A Survey on Topological Neural Networks

#14

Post by dpflan »

dpflan wrote: post #13
mplemay wrote: post #11

Earlier quoted context omitted.

Do you mind linking the homepage, google scholar, etc. of "Petar Velolickovic"? The only reference to the name on google search appears to be this post.

- https://geometricdeeplearning.com/ -- group of researchers presenting the paradigm - https://petar-v.com/ - Petar's homepage - https://scholar.google.co.uk/citations?user=kcTK_FAAAAAJ&hl=... - Petar's Google Scholar - https://www.linkedin.com/in/petarvelickovic/ - Petar's LI

GAT is particularly interesting at the moment: https://scholar.google.co.uk/citations?view_op=view_citation...

Re: Topological Deep Learning: A Survey on Topological Neural Networks

#15

Post by lmcinnes »

chaxor wrote: post #9
liorben-david wrote: post #7

The paper seems to be using homology as the topological feature here. I've done some work in Topological Data Analysis before and it feels like the hidden issue is that computing homology is generally very inefficient(Since it usually amounts to reducing an nxn matrix). It definitely feels like graphs/topology should be helpful tools to work with data(Since graph-like structures are good representations of the real w…

I think most people in the field recognize this efficiency issue and focus on very small NNs. As of yet I have seen very little promising work on improving efficiency, other than work outside of NNs, e.g. Eirene for PH (which is actually a mess under the hood - someone needs to clean up the code base last time I checked - 9 deep nested for loops, really?). There are several options for topological NN layers out there…

For suitable specialized cases thins can be quite efficient. For persistent H_0 of VR-complexes in low-dimensional space there is an O(N log(N)) algorithm for N data points; that's decently fast. If you want H_1 I believe (but cannot prove) that there exists an O(N^2 log(N)) algorithm. Beyond H_1 things get painful. Since most PH software is written for general cases they don't tend to avail themselves of these special case shortcuts. Given that H_0 and H_1 of VR-complexes of low dimensional data covers a vast amount of the use cases I think specialized code for this would be worthwhile.

Re: Topological Deep Learning: A Survey on Topological Neural Networks

#16

Post by lmeyerov »

dpflan wrote: post #12
lmeyerov wrote: post #6

Earlier quoted context omitted.

Agreed, the definitions here of what a GNN is (=> cannot do) seem pedantically narrow & extreme, and then throws the baby out w the bath water See last ~2 years of work by Michael Bronstein's teams (who moonlights as Twitter's ex? GNN r&d lead) that address 2+ of the points here It's true they are not the end-all, but our issues in practice aren't the above, but more like how to best combine temporal deep learning te…

Interesting. Could one graph be the topological representation, let's say like SCC? So there are heterogenous graphs with meta-graphs connected to them...?

There are a lot of cool experiments happening like this, both principled and from intuition

Ex:Label prop following non-local jumps ("metapaths") based on community detection (network-of-network, ...) or other shapes. Same thing for what constitutes a feature.

Michael Bronstein's papers (and talks!) are often in this vein, in the mainstream, leading, and quite accessible, so would recommend starting there..

Post reply on HN