Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

arxiv.org

Post reply on HN
Print view View original

21–30 of 177 posts

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#21

Post by godelski »

light_hue_1 wrote: post #17
godelski wrote: post #13

Earlier quoted context omitted.

There's something I tell my students. You don't need math to make good models, but you do need to know math to know why your models are wrong. So yes, math is needed. If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone. You'll just use transformers everywhere because that's what everyone else does and you'll get confused between activation functions. You'll make mo…

Oh sure. I say the same to my students. But the particular spin on this book makes it look to non-experts that this is the math you need to do something useful with deep learning. And that's just not true. Certainly you need to understand what you're optimizing, how your optimizer works, what your objective function is doing, etc. But the vast majority of people don't need to know about theoretical approximation resu…

Yeah I would have a very hard time recommending this book too. It is absurdly math heavy. I'm not sure I've even seen another book this math dense before and I've read some pretty dense books targeting review. So I'm not even sure what audience this book is aimed for. Citations? And I fully agree that the title doesn't fit whoever that audience is.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#22

Post by godelski »

axpy906 wrote: post #16

This is in Tensorflow. Would rather see a numpy version or something along those lines so that students can better understand what each step looks like in code. I concur on the comments noting lack of explanation for the notation/lemmas/proof.

I second this. Numpy would be the way to go, so students can switch to JAX or PyTorch trivially. Or they could use a mix, starting with numpy, build the layer from scratch, then hand over the abstraction. Pyro would be really good for this too

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#23

Post by nerdponx »

godelski wrote: post #9
reqo wrote: post #4

Is it common to publish books directly to ArXiv, especially books that have just been released?

It's not too uncommon to see books available online from an official location. At least math and CS textbooks

Normally I just see it on the author's website.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#24

Post by _giorgio_ »

axpy906 wrote: post #16

This is in Tensorflow. Would rather see a numpy version or something along those lines so that students can better understand what each step looks like in code. I concur on the comments noting lack of explanation for the notation/lemmas/proof.

Tensorflow? LOL what is this, the year 2010?

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#25

Post by HybridCurve »

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know).

Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#26

Post by HighFreqAsuka »

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated to proving lemmas that provide no additional understanding and are only loosely related to deep learning. And a significant chunk of the code I see is just the plotting code, which I don't even understand why you'd include. I'm confident that very few people will ever read significant chunks of this.

I think the best textbooks are still Deep Learning by Goodfellow etal and the more modern Understanding Deep Learning (https://udlbook.github.io/udlbook/).

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#27

Post by joshuanapoli »

HybridCurve wrote: post #25

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

Mathematical notation usually has a problem with preferring single-letter names. We usually prefer to avoid highly abbreviated identifier names in software, because they make the program harder to read. But they’re common in Math, and I think that it makes for a lot of work jumping back and forth to remind oneself what each symbol means when trying to make sense of a statement.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#28

Post by blauditore »

HighFreqAsuka wrote: post #26

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

I think the mathematical background starts making sense once you get a good understanding of the topic, and then people make the wrong assumption that understanding the math will help learning the overall topic, but it that's usually pretty hard.

Rather than trying to form an ituition based on the theory, it's often easier to understand the technicalities after getting an intuition. This is generally true in exact sciences, especially mathematics. That's why examples are helpful.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#29

Post by joe_the_user »

godelski wrote: post #11
aquafox wrote: post #3

[flagged]

There's a lot to critique but this is a really weird one (page 49 if anyone is following). The whole thing is 5 sentences and all the space is because a diagram and code block. The 5 sentences should be the thing to complain about

I only skimmed but I get the impression that sort of thing is common in the text.

I think it's got the problem that deep learning "isn't really math" - in the sense that deep learning using indeed very elaborate computational structures that can be specified mathematically but it doesn't prove theorems about them - not theorems that characterize what's happening. The theorems are just hints about what might be happening.

The key deep learning knowledge is in papers that basically only show that X approach works best on Y (plus maybe some suggestive theorem) - for example Attention Is All You Need.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#30

Post by aabajian »

HybridCurve wrote: post #25

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

All three authors are PhDs or PhD-candidates in mathematics. The notation is extremely dense. I'm curious who their target audience of "students and scientists" are for this book.
Post reply on HN