Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Category Theory ∩ Machine Learning

github.com

Post reply on HN
Print view View original

21–30 of 73 posts

Re: Category Theory ∩ Machine Learning

#21

Post by mydogcanpurr »

donnowhy wrote: post #16

category theory is 'native 2-dimensional' math. i.e. category theory explains everything in terms of graphs, where a graph is made from two different sorts of 'entities', nodes and vertices i.e. categories and morphisms this being math, I wonder to which extent can category theory be re-expressed in terms of sets. perhaps a better question is if category theory can be re-expressed (or founded on) functions? lastly, I…

> for some reason I don't think just sets are sufficient

The reason you're looking for is that the category of sets is not a set.

Re: Category Theory ∩ Machine Learning

#22

Post by LudwigNagasena »

jurynulifcation wrote: post #18
red_trumpet wrote: post #11

Earlier quoted context omitted.

Your paper didn't pass my smell test at all, tbh. For example the formula you write about "product" and "coproduct" in section 3 is literally identical (as "=" is symmetric). In section 4.2 you write "the product is the standard tensor product" with a formula that doesn't at all involve the map m: A \otimes A \to A. The formula you write is the induced product on A \otimes A, assuming that you already have a product…

adamnemecek has posted too many comments and is in cooldown phase, but he's asked me to post this comment: "It's the programmers equal sign. I think that the surrounding text provides a decent explanation what the deal is. You are right, there's a missing sentence fragment, "standard tensor product that satisfies the property...". Read the Diaconis paper. " --- This isn't a sock puppet and I hope this isn't against s…

> It's the programmers equal sign.

That doesn’t seem to make any sense.

Re: Category Theory ∩ Machine Learning

#23

Post by red_trumpet »

hgsgm wrote: post #13
gexaha wrote: post #6

Earlier quoted context omitted.

could you advertise your research a bit less often, please? i see your post like literally almost every other day here

Or at least explain it in more accessible way. Every time Adam posts about the paper, it gets confused comments and no engagement on the content, because it's pretty deep graduate level pure math, which is occasionally seen but rare on HN.

As a maths PhD student that has seen Hopf algebras before (though I'm no expert, and the context was different), I'm not convinced Adam understands things about Hopf algebras.

Re: Category Theory ∩ Machine Learning

#24

Post by [deleted] »

adamnemecek wrote: post #3

I have recently written a paper on understanding machine learning via the lens of Hopf algebra https://arxiv.org/abs/2302.01834 . Hopf algebras (which are really just tensors with recurrence relations built in) subsume convnets, transformers and diffusion model and also provide a theoretically better autodiff that operates within single layers as opposed to across entire graphs. Furthermore, there is a correspondence…

[deleted]

Re: Category Theory ∩ Machine Learning

#25

Post by justatdotin »

donnowhy wrote: post #16

category theory is 'native 2-dimensional' math. i.e. category theory explains everything in terms of graphs, where a graph is made from two different sorts of 'entities', nodes and vertices i.e. categories and morphisms this being math, I wonder to which extent can category theory be re-expressed in terms of sets. perhaps a better question is if category theory can be re-expressed (or founded on) functions? lastly, I…

> I wonder to which extent can category theory be re-expressed in terms of sets...

yoneda

Re: Category Theory ∩ Machine Learning

#26

Post by umutisik »

It is tempting to believe that category theory will shed new light on and simplify machine learning, just like it did in algebraic geometry, algebraic topology and other mathematical things. This is wishful thinking. Folks who care about doing something useful should stay away from this content.

Re: Category Theory ∩ Machine Learning

#27

Post by bgavran »

rmdamiao wrote: post #2

Is this simply a consequence of exponential growth in CS publications driven by machine learning or is there something really going on here?

OP here.

The exponential growth in CS publication is much faster. This repository is simply a testament that CT is slowly ramping up.

It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.

Re: Category Theory ∩ Machine Learning

#28

Post by bgavran »

bgavran wrote: post #27
rmdamiao wrote: post #2

Is this simply a consequence of exponential growth in CS publications driven by machine learning or is there something really going on here?

OP here. The exponential growth in CS publication is much faster. This repository is simply a testament that CT is slowly ramping up. It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.

It's also meant to suggest where things are going (the kind of a chart I have in mind is this one https://twitter.com/bgavran3/status/1422206118688956420 ), though I understand this is something that deserves a much more substantial proof.

Re: Category Theory ∩ Machine Learning

#29

Post by riku_iki »

Yoric wrote: post #14
hgsgm wrote: post #9

Earlier quoted context omitted.

Why? How? The OP GitHub site doesn't promote any material that introduces the concepts at all. The "survey" paper at the top is nigh-impenetrable. I'm sure the category theorists are having fun modelling machine learning, but it doesn't show how machine learning benefits from the category theory.

Category Theory (just as all mathematical models for programming or subsets thereof) are building blocks for reasoning on what we build. Past applications of such mathematical models include: - programming languages with semantics that are better adapted to specific problems (e.g. Rust's ownership); - better compilers (see e.g. Haskell's supercompiler, which puts to shame `constexpr`-style features); - better static…

why those approaches never picked up outside of some academia projects?..

Re: Category Theory ∩ Machine Learning

#30

Post by bgavran »

hgsgm wrote: post #9
adamnemecek wrote: post #4

Earlier quoted context omitted.

The field needs better foundations. CT is pretty good.

Why? How? The OP GitHub site doesn't promote any material that introduces the concepts at all. The "survey" paper at the top is nigh-impenetrable. I'm sure the category theorists are having fun modelling machine learning, but it doesn't show how machine learning benefits from the category theory.

There's a well-articulated essay from Christopher Olah that captures the undercurrent, and perhaps even the motivation of many of these papers https://colah.github.io/posts/2015-09-NN-Types-FP/

You're completely right that the repo doesn't promote any introductory material. CT is notoriously difficult to get into, and this repository wasn't meant to be a pedagogical one, but rather a list of all the relevant papers.

Though, I'll see about remedying this. I have a different repository that curates a list of (what I consider to be, as a CS major) relatively approachable CT introductory materials https://github.com/bgavran/Category_Theory_Resources

and it might be a good idea to add a pointer to it.

Post reply on HN