Live data from Hacker News

Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

arxiv.org

41–50 of 177 posts

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#41

Earlier quoted context omitted.

[flagged]

Honestly, because the very first sentence of the preface is "This book aims to provide an introduction to the topic of deep learning algorithms." Really? LOL. If you're going to pitch 600 pages of dense mathematical notation as "introductory," you're going to have to expect some people to call BS. What's interesting/unfortunate is that their Python code samples really are easy to follow and pedagogically useful to a…

[flagged]

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#42
post #16

This is in Tensorflow. Would rather see a numpy version or something along those lines so that students can better understand what each step looks like in code. I concur on the comments noting lack of explanation for the notation/lemmas/proof.

Most of the examples I saw used Pytorch. (Which is still a step or two removed from the actual machinery, of course.)

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#43

Earlier quoted context omitted.

Honestly, because the very first sentence of the preface is "This book aims to provide an introduction to the topic of deep learning algorithms." Really? LOL. If you're going to pitch 600 pages of dense mathematical notation as "introductory," you're going to have to expect some people to call BS. What's interesting/unfortunate is that their Python code samples really are easy to follow and pedagogically useful to a…

[flagged]

[deleted]

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#44

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

It's not too difficult to understand, but this introduction isn't written with pedagogy in mind IMO

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#45

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

This book is not aimed at practitioners but I don’t think that means it deserves to be called „actively the worst one”. Even though the frontier of deep learning is very much empirical, there’s interesting work trying to understand why the techniques work, not only which ones do. I’m sorry but saying proofs are not a good method for gaining understanding is ridiculous. Of course it’s not great for everyone but a book…

Just read the section on ResNets (Section 1.5) and tell me if you think that's the best way to explain ResNets to literally anyone. Tell me if, from that description, you take away that the reason skip connections improve performance is that they improve gradient flow in very deep networks.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#46
This makes me wonder. Is deep learning as a field an empirical science purely because everyone is afraid of the math? It has the richness of modern day physics but for some reason most the practioners seem to want to keep thinking of it as the wild west

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#47

This makes me wonder. Is deep learning as a field an empirical science purely because everyone is afraid of the math? It has the richness of modern day physics but for some reason most the practioners seem to want to keep thinking of it as the wild west

No, there are many very mathematically inclined deep learning researchers. It's an empirical science because the mathematical tools we possess are not sufficient to describe the phenomena we observe and make predictions under one unified theory. Being an empirical science does not mean that the field is a "wild west". Deep learning models are subjectable to repeatable controlled experiments, from which you can improve your understanding of what will happen in most cases. Good practitioners know this.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#48

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

Obligatory hn comment on any math-related topic: "notation bad"

Please be more original.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#49

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

I’m just wrapping up a PhD in ML. The notation here is unnecessarily complex IMO. Notation can make things easier, or it can make things more difficult, depending on a number of factors.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#50

Earlier quoted context omitted.

This book is not aimed at practitioners but I don’t think that means it deserves to be called „actively the worst one”. Even though the frontier of deep learning is very much empirical, there’s interesting work trying to understand why the techniques work, not only which ones do. I’m sorry but saying proofs are not a good method for gaining understanding is ridiculous. Of course it’s not great for everyone but a book…

Just read the section on ResNets (Section 1.5) and tell me if you think that's the best way to explain ResNets to literally anyone. Tell me if, from that description, you take away that the reason skip connections improve performance is that they improve gradient flow in very deep networks.

the reason skip connections improve performance is that they improve gradient flow in very deep networks.

Can you prove this statement?

Post reply on HN