Guide to Machine Learning with Geometric, Topological, and Algebraic Structures • Hacker News Board

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#11

Post by dpflan » Mon, Jul 15, 2024, 6:51 PM UTC

heyitsguay wrote: post #6

dpflan wrote: post #4

Earlier quoted context omitted.

I believe a use-case(s) receiving attention is drug design, protein design, chemical design, etc. Here is a summer school by the London Geometry and Machine Learning group where research topics are shared and discussed. - https://www.logml.ai/ Here is another group, a weekly reading group on graphs and geometry: https://portal.valencelabs.com/logg

As someone who did an applied math PhD before drifting towards ML, it's worth pointing out that these applied math groups typically talk about applications, but the real question is whether they are actually used for the stated application in practice due to outperforming methods that use less pretty math. Typically (in every case i have seen) the answer is "no", and the mathematicians don't even really care about so…

Indeed, the ivory tower has nice chats and ideas and is a cool place to hang out, but does application actually occur.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#12

Post by jqgatsby » Mon, Jul 15, 2024, 7:01 PM UTC

llm_trw wrote: post #7

funnygiraffe wrote: post #3

Is geometric, topological, and algebraic ML/data analysis actually used in the industry? It is certainly beautiful math. However, during grad school I met a few pure math PhD students who were saying that after finishing their PhD they will just go into industry to do topological data analysis (this was about 10 years ago and ML wasn't yet as hyped up). However, I have never heard of anybody actually having success o…

I've had some success using hyperbolic embeddings for bert like models. It's not something that the companies I've worked for advertised or wrote papers about.

Hyperbolic embeddings have been an interest of mine ever since the Max Nickel paper. Would love to connect directly to discuss this topic if you're open. here's my email: https://photos.app.goo.gl/1khCwXBsVBuEP6xF7

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#13

Post by uoaei » Mon, Jul 15, 2024, 7:02 PM UTC

itissid wrote: post #8

One common theme I see in the paper(e.g. in protein folding) is: "Identify what properties are important (geometry, algebra, topo) and which one is an useful prior and then "use" the guide to select an initial struct. This is probably harder than it sounds(unlike bayesian priors which are more forgiving for one to select, but quite like them in that they both require special assumptions)." I wonder: could one use it…

All machine learning is just embedding of various forms. If you have a way to translate disparate types of data into a common space, in ways that preserve inductive bias and information content, you can then combine them for downstream tasks.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#14

Post by mjhay » Mon, Jul 15, 2024, 7:05 PM UTC

I am 100% convinced that these kind of approaches will be what delivers ML research from the current resource-hungry and ungeneralizable status quo. Low-dimensional Euclidean geometry is special. Higher-dimensional Euclidean spaces are less special. Most real-life data is high-dimensional, not at all smooth, and possessing a structure you cannot call Euclidean with a straight face. Look at what works with tabular data (which is probably most of what practitioners work with in the wild). It's gradient boosted trees, not neural networks.

There is a fundamental mismatch between the data we usually work with and the spaces we shove it into. Tools from algebraic topology and geometry are old hat in physics. If anything, they should be even more useful in ML.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#15

Post by Davidzheng » Mon, Jul 15, 2024, 7:47 PM UTC

mjhay wrote: post #14

I am 100% convinced that these kind of approaches will be what delivers ML research from the current resource-hungry and ungeneralizable status quo. Low-dimensional Euclidean geometry is special. Higher-dimensional Euclidean spaces are less special. Most real-life data is high-dimensional, not at all smooth, and possessing a structure you cannot call Euclidean with a straight face. Look at what works with tabular dat…

I heavily disagree with the statement "tools of algebraic topology is old hat in physics"

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#16

Post by [deleted] » Mon, Jul 15, 2024, 8:25 PM UTC

funnygiraffe wrote: post #3

Is geometric, topological, and algebraic ML/data analysis actually used in the industry? It is certainly beautiful math. However, during grad school I met a few pure math PhD students who were saying that after finishing their PhD they will just go into industry to do topological data analysis (this was about 10 years ago and ML wasn't yet as hyped up). However, I have never heard of anybody actually having success o…

[deleted]

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#17

Post by mistrial9 » Mon, Jul 15, 2024, 9:07 PM UTC

some people on this thread are asking about jobs. The bigger picture here is that previously intractable problems are going to be solved with a new combination of math, data and compute.. there are lots of commercial cases that will change dramatically. How can individual people or small groups benefit from serious problem solving, economically?

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#18

Post by BenoitP » Mon, Jul 15, 2024, 9:13 PM UTC

Davidzheng wrote: post #15

mjhay wrote: post #14

I am 100% convinced that these kind of approaches will be what delivers ML research from the current resource-hungry and ungeneralizable status quo. Low-dimensional Euclidean geometry is special. Higher-dimensional Euclidean spaces are less special. Most real-life data is high-dimensional, not at all smooth, and possessing a structure you cannot call Euclidean with a straight face. Look at what works with tabular dat…

I heavily disagree with the statement "tools of algebraic topology is old hat in physics"

Well, I consider Lorentz' work to be old hat. I can't find an older example though.

https://en.m.wikipedia.org/wiki/Lorentz_group

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#19

Post by OutOfHere » Mon, Jul 15, 2024, 9:16 PM UTC

mjhay wrote: post #14

I am 100% convinced that these kind of approaches will be what delivers ML research from the current resource-hungry and ungeneralizable status quo. Low-dimensional Euclidean geometry is special. Higher-dimensional Euclidean spaces are less special. Most real-life data is high-dimensional, not at all smooth, and possessing a structure you cannot call Euclidean with a straight face. Look at what works with tabular dat…

Comment is heavily exaggerated in every way.

Re: Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

#20

Post by OutOfHere » Mon, Jul 15, 2024, 9:22 PM UTC

Note that the GPU hardware is setup for Euclidean matrix operations. Even if you had a deep structure learner, it won't necessarily help you if you have to go back to emulating it on Euclidean hardware.