Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

arxiv.org

Post reply on HN
Print view View original

31–40 of 177 posts

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#31

Post by layer8 »

HybridCurve wrote: post #25

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

Mathematical notation is more concise, which may take some getting used to. One reason is that it is optimized for handwriting. Handwriting program code would be very tedious, so you can see why mathematical notation is the way it is.

Apart from that, there is no “the code” equivalent. Mathematical notation is for stating mathematical facts or propositions. That’s different from the purpose of the code you would write to implement deep-learning algorithms.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#32

Post by danielmarkbruce »

HighFreqAsuka wrote: post #26

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

UDL has some dense math notation in it.

Math isn't just about proofs. It's a way to communicate. There are several different ways to communicate how a neural net functions. One is with pictures. One is with some code. One is with words. One is with some quite dense math notation.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#33

Post by HighFreqAsuka »

danielmarkbruce wrote: post #32
HighFreqAsuka wrote: post #26

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

UDL has some dense math notation in it. Math isn't just about proofs. It's a way to communicate. There are several different ways to communicate how a neural net functions. One is with pictures. One is with some code. One is with words. One is with some quite dense math notation.

I agree with that, I think UDL uses the necessary amount of math to communicate the ideas correctly. That is obviously a good thing. What it does not do is pretend to be presenting a mathematical theory of deep learning. Basically UDL is exactly how I think current textbooks should be presented.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#34

Post by conformist »

HybridCurve wrote: post #25

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

Yes, it's easier for mathematicians, because a lot of background knowledge and intuition is encoded in mathematical conventions (eg "C(R)" for continuous functions on the reals etc...). Note that this is probably a book for mathematicians.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#35

Post by joe_the_user »

godelski wrote: post #13
dachworker wrote: post #2

Is anyone using any of this math? My guess is no. At best it provides "moral support" for deep learning researchers who want to feel reassured that what they are attempting to do is not impossible. Glad to be proven wrong, though.

There's something I tell my students. You don't need math to make good models, but you do need to know math to know why your models are wrong. So yes, math is needed. If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone. You'll just use transformers everywhere because that's what everyone else does and you'll get confused between activation functions. You'll make mo…

If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone.

There are very smart people who think we can get to AGI by scale alone - they call that the "the scaling hypothesis", in fact. I think they're wrong but I thought they knew a fair amount of math.

What math would you use to describe the limitations of deep learning? My impression is there aren't any exact theorems that describe either it's limits or it's behavior/possibilities, there are just suggestive theorems and constructions combined with heuristics.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#36

Post by thehappyfellow »

HighFreqAsuka wrote: post #26

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

This book is not aimed at practitioners but I don’t think that means it deserves to be called „actively the worst one”.

Even though the frontier of deep learning is very much empirical, there’s interesting work trying to understand why the techniques work, not only which ones do.

I’m sorry but saying proofs are not a good method for gaining understanding is ridiculous. Of course it’s not great for everyone but a book titled „Mathematical Introduction to x” is obviously for people with some mathematical training. For that kind of audience lemmas and their proof are natural way of building understanding.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#37

Post by [deleted] »

HighFreqAsuka wrote: post #26

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

[deleted]

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#38

Post by n3ur0n »

danielmarkbruce wrote: post #32
HighFreqAsuka wrote: post #26

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated t…

UDL has some dense math notation in it. Math isn't just about proofs. It's a way to communicate. There are several different ways to communicate how a neural net functions. One is with pictures. One is with some code. One is with words. One is with some quite dense math notation.

I would say UDL should be very accessible to any undergrad from a strong program.

I would not call the notation ‘dense’ rather it’s ‘abused’ notation. Once you have seen the abused notation enough times, it makes just makes sense. Aka “mathematical maturity” in the ML space.

My views on this have changed as a first year PhD in ML I got annoyed by the shorthand. Now as someone with a PhD, I get it — It’s just too cumbersome to write out what exactly you mean and you write like you’re writing for peers +\- a level.

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#39

Post by talentedcoin »

HybridCurve wrote: post #25

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

[flagged]

Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

#40

Post by CamperBob2 »

talentedcoin wrote: post #39
HybridCurve wrote: post #25

As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?

[flagged]

Honestly, because the very first sentence of the preface is "This book aims to provide an introduction to the topic of deep learning algorithms." Really? LOL. If you're going to pitch 600 pages of dense mathematical notation as "introductory," you're going to have to expect some people to call BS.

What's interesting/unfortunate is that their Python code samples really are easy to follow and pedagogically useful to a beginner. I think a lot of people will be turned off by the text unnecessarily.

It should have been promoted as a rigorous reference textbook, which is what it is, and not any sort of tutorial or primer.

Post reply on HN