Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

arxiv.org

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#21

Post by llm_trw »

jqgatsby wrote: post #12
llm_trw wrote: post #7

Earlier quoted context omitted.

I've had some success using hyperbolic embeddings for bert like models. It's not something that the companies I've worked for advertised or wrote papers about.

Hyperbolic embeddings have been an interest of mine ever since the Max Nickel paper. Would love to connect directly to discuss this topic if you're open. here's my email: https://photos.app.goo.gl/1khCwXBsVBuEP6xF7

Not much to discuss really, I just monkey patched a different metric function, then results for our use case became substantially better after training a model from scratch on the same data compared to the previous euclidean model trained from scratch.

I'm currently working on massive multi agent orchestration so don't have my head in that side of things currently.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#22

Post by funnygiraffe »

mistrial9 wrote: post #17

some people on this thread are asking about jobs. The bigger picture here is that previously intractable problems are going to be solved with a new combination of math, data and compute.. there are lots of commercial cases that will change dramatically. How can individual people or small groups benefit from serious problem solving, economically?

Why are we so sure that a lot of "previously intractable problems" are/will be solved with this family of methods? (and I mean real-life/real-world problems, not toy problems constructed specifically to show the proposed methods in the best light possible in the research papers) Of course others above have pointed out drug or protein design as a potential area, but there still seems uncertainty as to the practical impact on the real world. Other than that, I don't see areas of impact for these approaches so far.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#23

Post by johmathe »

funnygiraffe wrote: post #22
mistrial9 wrote: post #17

some people on this thread are asking about jobs. The bigger picture here is that previously intractable problems are going to be solved with a new combination of math, data and compute.. there are lots of commercial cases that will change dramatically. How can individual people or small groups benefit from serious problem solving, economically?

Why are we so sure that a lot of "previously intractable problems" are/will be solved with this family of methods? (and I mean real-life/real-world problems, not toy problems constructed specifically to show the proposed methods in the best light possible in the research papers) Of course others above have pointed out drug or protein design as a potential area, but there still seems uncertainty as to the practical im…

One of the authors here. Thanks for your comment! While a lot of this research is theoretical and does not have immediate use cases, we have tried to summarize some of them in the last paragraph of the paper (VII. Applications of Non-Euclidean Geometry). See page 26. We present some in Chemistry and Drug Development, Structural Biology and Protein Engineering, Computer Vision, Biomedical Imaging, Recommender Systems and Social Networks and Physics.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#24

Post by data_maan »

dpflan wrote: post #2

The paper's references have some good ones for getting more acquainted with these subjects; this one being a nice dense one to start with: - Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges: https://geometricdeeplearning.com/

Don't read that. That's one of the most horrible papers I know, a hodgepodge of mathematical well-known concepts thrown together with some vague ideas how they connect. The mathematical parts are explained better in any undergraduate book. I'm not sure why the author felt the need to expound on what a manifold is when that has been done better in hundreds of other texts - literally.

And it lacks a definite conclusion: They don't prove anything, don't make any particular experiment, but just loosely talk about how these ideas might be relevant to machine learning.

I'm surprised that such highly cited researcher have produced such a paper. I would be embarrassed to be on it - and I'm embarrassed on behalf of the ML community that they are citing it.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#25

Post by nborwankar »

llm_trw wrote: post #7
funnygiraffe wrote: post #3

Is geometric, topological, and algebraic ML/data analysis actually used in the industry? It is certainly beautiful math. However, during grad school I met a few pure math PhD students who were saying that after finishing their PhD they will just go into industry to do topological data analysis (this was about 10 years ago and ML wasn't yet as hyped up). However, I have never heard of anybody actually having success o…

I've had some success using hyperbolic embeddings for bert like models. It's not something that the companies I've worked for advertised or wrote papers about.

Can you share what kinds of problems were conducive to hyperbolic embeddings in your experience. Also, separately, are you saying companies are using these in practice but don’t talk about them because of the advantage they give? Or am I reading too much into your last sentence.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#26

Post by dpflan »

data_maan wrote: post #24
dpflan wrote: post #2

The paper's references have some good ones for getting more acquainted with these subjects; this one being a nice dense one to start with: - Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges: https://geometricdeeplearning.com/

Don't read that. That's one of the most horrible papers I know, a hodgepodge of mathematical well-known concepts thrown together with some vague ideas how they connect. The mathematical parts are explained better in any undergraduate book. I'm not sure why the author felt the need to expound on what a manifold is when that has been done better in hundreds of other texts - literally. And it lacks a definite conclusion…

Do you have better references you could share?

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#27

Post by Pseudomanifold »

Great selection of works, but I am missing a lot of references from topology in ML, with the article only assuming a very cursory perspective in terms of 'topology captures connectivity and/or continuity.'

Some works from my colleagues and me go a little bit deeper (no pun intended), for instance:

- Neural Persistence Dynamics: https://arxiv.org/abs/2405.15732

- Simplicial Representation Learning with Neural $k$-Forms: https://openreview.net/forum?id=Djw0XhjHZb

- A general review on topology in machine learning: https://www.frontiersin.org/journals/artificial-intelligence...

There are more things in topology and machine learning, Horatio, than are dreamt of in your article ;-)

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#28

Post by johmathe »

Pseudomanifold wrote: post #27

Great selection of works, but I am missing a lot of references from topology in ML, with the article only assuming a very cursory perspective in terms of 'topology captures connectivity and/or continuity.' Some works from my colleagues and me go a little bit deeper (no pun intended), for instance: - Neural Persistence Dynamics: https://arxiv.org/abs/2405.15732 - Simplicial Representation Learning with Neural $k$-Form…

Thanks for the great pointers!

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#29

Post by llm_trw »

nborwankar wrote: post #25
llm_trw wrote: post #7

Earlier quoted context omitted.

I've had some success using hyperbolic embeddings for bert like models. It's not something that the companies I've worked for advertised or wrote papers about.

Can you share what kinds of problems were conducive to hyperbolic embeddings in your experience. Also, separately, are you saying companies are using these in practice but don’t talk about them because of the advantage they give? Or am I reading too much into your last sentence.

They are better at separating clusters and keep the fact that distances under the correct metric also provide semantic information. The issue is that training is longer and you need at least 32, and ideally 64 bit floats during training and inference.

And possibly.

The company I did the work for kept it very quiet. Bert like models are small enough that you can train them a a work station today so there is a lot less prestige in them than 5 years ago, which is why for profit companies don't write papers on them any more.

Post reply on HN