Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

Thanks for the reply. I'm glad my comment is no longer flagged.

What do you mean that "this point isn't particularly controversial?" If you just mean that "X may be useful", then of course. But the particular X matters, and "could be useful" is much different than "is useful".

People who like category theory want it everywhere. I don't know your mathematical background, but spend any time in a math department, or even classes, and you'll find people ready to explain any topic in the language of CT.

The may be useful, but it has to be justified. It's clear in some mathematical contexts, but definitely not in ML (yet alone analysis).

ML has a problem in that no one knows what certain methods work. Just look at something like batch normalization: I can think of at least 3 different "explanations" on why it works.

ML people want explanations, and mathematicians need work. Category theorists therefore have work. But I don't think you should mistake this as being an explanation. You just get a nice get a "cleaner way" to present concepts.

> You have a parrot that can paint original pictures, compose original songs and essays, and translate math into both English and program code?

Kinda, kinda, yes, and yes.

I think there's far less originality than most people think. But it's not surprising when your job isn't leading you to look at thousands of pictures a day. I have yet to see a generative model that isn't pulling heavily towards the training data and you might be noticing the memorization rates are getting higher. But yes, a stochastic parrot doesn't mean memorization, it is about generalization and the stability around the p-norm ball around the training data.

Btw, what's wrong with a stochastic parrot? They are absolutely fucking useful. I use them every day. Hell, I even use things that are complete memorizations and all compression every day. What's with everyone equating powerful statistical systems with uselessness. Anyone saying that they aren't extremely useful is pulling wool over their eyes (but the same is true for anyone claiming baby AGI).

I'd also appreciate it if you discussed in good faith. The snarkiness is not appreciated.

> I'm glad my comment is no longer flagged.

FYI, I flagged you because the comment does not live up to the HN community standards[0]. A new account with just a comment to me made shortly after my comment was made just to say something sarcastic and does not contribute to the conversation. I decided to flag instead of commenting and continuing an unproductive exchange.

> People who like category theory want it everywhere.

This isn't surprising. It is an attempt at further generalization of mathematics. Albeit it can get annoying, it isn't wrong because cat theory is about looking from the high abstract level and making connections between differing branches of mathematics. If you don't see it everywhere you either don't have an understanding or have discovered something those people would really like to know. From personal experience, it can be a quite useful tool to describe things because of this.

> The may be useful, but it has to be justified.

The former begets the latter.

> Just look at something like batch normalization: I can think of at least 3 different "explanations" on why it works.

Are those the same thing? What are those?

> But I don't think you should mistake this as being an explanation. You just get a nice get a "cleaner way" to present concepts.

The latter is de facto the former.

And yes, math is just models. Or as Poincaré said, math is the study of relationship between numbers. One might also say "the map is not the territory" and you can find several math theorems making this point explicitly about math. You may even find one by reading my username with a little care. More than one if you take more care.

[0] https://news.ycombinator.com/newsguidelines.html

I'm not being snarky! I genuinely feel I'm the one being gaslighted, by people telling me I shouldn't be utterly blown away by answers like the earlier example, or the one I just received:

https://i.imgur.com/JSWLFOi.png

I regularly get downvoted and criticized for suggesting this tool to other students, in defiance of what I can clearly see happening with my own eyes. I see a tool that, if developed further, will answer much deeper questions, including original ones, just as accurately and effectively. One that appears capable of taking humanity to the next level so fast it will make the monolith in 2001 look like an abacus by comparison.

Meanwhile, you tell me, "Don't suggest this to other students, it might hallucinate." Other people say, "Shut this down at once (or nerf it beyond any possibile utility), it might hurt somebody's feelings." Another contingent warns, "Shut this down at once, it might start a nuclear war." Still other people say, "Shut this down at once, it violates copyright law." The objections just get dumber from there, yet gain traction by the day.

There's never been a time when standing in the way of something like this was right. Why should I think it's time to do so now? (And yes, I acknowledge that you're not personally 'standing in the way', but it really bugs me when people who claim they aren't 'standing in the way' of the technology tell other people not to use it.)

I have yet to see a generative model that isn't pulling heavily towards the training data

When's the last time you saw a human mind that didn't work that way? (Or, for that matter, a parrot's mind.) The real truth behind the stochastic-parrot metaphor is that parrots, stochastic or otherwise, are nothing all that special, and neither are we. We're just better at using tools than the birds are, that's all.

Or at least we were up until now. But muh COPYRITE!!!11! ...

> If you don't see it everywhere you either don't have an understanding or have discovered something those people would really like to know. From personal experience, it can be a quite useful tool to describe things because of this.

Get off your high horse. I've had my share of Mac Lane. If you can describe something in terms of CT, you can talk to mathematicians who care about CT. I don't see why this helps ML.

> The may be useful, but it has to be justified. "May be useful" does not beget "justified." CT may be useful in all areas if you ask a CT theorist. I fail to see how CT helps me build a car.

>The latter is de facto the former.

No it's not. You can take you favorite analysis topic and find a suitable category to view your topic from a CT perspective, but this won't tell you how to prove anything. If you did the CT correct you can now make some analogies, but it won't tell you anything specific.

> And yes, math is just models. Or as Poincaré said, math is the study of relationship between numbers. One might also say "the map is not the territory" and you can find several math theorems making this point explicitly about math.

How do you square "math is the study of relationship between numbers" with CT? You can diagram chase without seeing a single number. I have no idea what mathematical theorem you are referring to, but if you're extrapolating philosophical points from a mathematical theorem, you're doing it wrong

> You may even find one by reading my username with a little care. More than one if you take more care.

Ok I'll bite. You seem to be into Normalizing Flows. How does CT explain it being useful?