Matrix Calculus for Deep Learning
explained.ai
Matrix Calculus for Deep Learning
1–10 of 49 posts
Re: Matrix Calculus for Deep Learning
#2I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”.
In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’ “You just did in 10 mins with 30 lines of code what I worked on for an entire semester”.
That’s been the trajectory of the tools and increasing abstraction in this space. It’s just getting easier and easier to build models that work (which is great), and it gets easier and easier to do so without knowing more than an extremely high level overview of the math behind it all.
So while this looks like a great resource - who’s it for?
For jobs/problems that need you to have a thorough understanding of the math and theory behind the networks this isn’t going to cut it.
For jobs/problems that need you to get something working math or not - this likely isn’t necessary to get started.
So it’s for people that have been getting into DL but also haven’t bothered or needed to look up the math concepts?
Re: Matrix Calculus for Deep Learning
#3I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
[deleted]
Re: Matrix Calculus for Deep Learning
#4I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
Wouldn't be a proper DL post without some good ol' gatekeeping.
Re: Matrix Calculus for Deep Learning
#5I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
Wouldn't be a proper DL post without some good ol' gatekeeping.
Honestly, I was trying to convey the opposite - the gates are wide open and it’s never been easier to drive through.
Re: Matrix Calculus for Deep Learning
#6I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
Does this not count as looking up the math concepts?
Re: Matrix Calculus for Deep Learning
#7I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
Well one of the authors (Jeremy Howard) strongly advocates for learning by writing. I think it's a good mission: there is always value in finding better ways to convey information. If you already know this material then it's easily ignored. Other people may be excited about something you already know, and that's fine. Unfortunately that makes it pop up in your news feed and sorry I have no solution for that =).
Edit: Just noticed the other author is the creator of ANTLR, which I recall using in school to write our own languages. Cool!
Re: Matrix Calculus for Deep Learning
#8I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
I'd guess it's for people in the second camp (practical) who are just trying to satisfy their intellectual curiosity or want an intro to the math behind it all.
I agree with your assessment that's it doesn't have much practical use on its own, nor is it an efficient means to any particular end.
Re: Matrix Calculus for Deep Learning
#9I am very impressed with the clarity of presentation here. I usually link to The Matrix Cookbook[1] when I need to cite a reference for matrix calculus theorems but I might reference this instead in the future. I particularly like the section on the vector chain rule (which is very clear) and the section on element-wise operations (which uses novel notation to present many results in a compact form.)
[1]: https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf
Re: Matrix Calculus for Deep Learning
#10I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’…
It matters, in certain contexts.
For example: a large number of clustering methods boil down to matrix factorization, with variations in the constraints. If you have both domain understanding and a general understanding of what kind of output these variations are likely to result in, you can often narrow down the list of methods you need to try.
Also, interviews.