The matrix calculus you need for deep learning (2018)
1–10 of 43 posts
Re: The matrix calculus you need for deep learning (2018)
#2[deleted]
Re: The matrix calculus you need for deep learning (2018)
#3> We assume no math knowledge beyond what you learned in calculus 1
Anyone got recommendations for self-studying and testing Calculus 1?
Re: The matrix calculus you need for deep learning (2018)
#4> We assume no math knowledge beyond what you learned in calculus 1 Anyone got recommendations for self-studying and testing Calculus 1?
1) go into knowing that the terminology is the hardest part to learn. The arithmetic is fairly easy, but the word salads can get daunting.
2) khan academy, coursera, MIT open courseware, whatever books your library has. Piece together multiple sources, b/c you don't have a professor to ask questions to, you'll need all the different sources and how each source explains the same idea differently for everything to 'click'.
Re: The matrix calculus you need for deep learning (2018)
#5> We assume no math knowledge beyond what you learned in calculus 1 Anyone got recommendations for self-studying and testing Calculus 1?
1) go into knowing that the terminology is the hardest part to learn. The arithmetic is fairly easy, but the word salads can get daunting. 2) khan academy, coursera, MIT open courseware, whatever books your library has. Piece together multiple sources, b/c you don't have a professor to ask questions to, you'll need all the different sources and how each source explains the same idea differently for everything to 'cli…
I refreshed on linear algebra and multi variable calculus through Khan Academy before I took a theory-based machine learning class. For me, it was a really good experience and I can’t overstate my appreciation for Sal Khan’s teaching style.
Re: The matrix calculus you need for deep learning (2018)
#6is there an online course for maths? like undergrad maths?
Re: The matrix calculus you need for deep learning (2018)
#7> We assume no math knowledge beyond what you learned in calculus 1 Anyone got recommendations for self-studying and testing Calculus 1?
To begin, try to get a really high level explanation for differential and integral calculus. Look at videos, illustrations, etc.
Then, once you have a mental image of what you will be doing, get into the math from the ground up.
Re: The matrix calculus you need for deep learning (2018)
#8If you know single variable calculus and basic linear algebra, then you can often muddle through by just remembering that the derivative of,
(1) f: R^n -> R is a vector
(2) f: R -> R^n is a vector
(3) f: R^m -> R^n is an n x m matrix (the Jacobian)
(*) f(x)=x^tAx is f'(x)=(A + A^t)x (this is an example of (1))
(**) and that the derivative (gradient) of (1) gives you (3) with m=n, and in this case the derivative of (3) will be symmetric (the Hessian).
then just do your best to apply the single variable rules of differentiation (product rule, chain rule, etc.), and then mess around a bit with the result until all the dimensions match up and such that your result matches the appropriate case (1)-(3).
For any more complicated functions you encounter such as the determinant, you can just look up its derivative when needed.
Re: The matrix calculus you need for deep learning (2018)
#9> We assume no math knowledge beyond what you learned in calculus 1 Anyone got recommendations for self-studying and testing Calculus 1?
To begin, try to get a really high level explanation for differential and integral calculus. Look at videos, illustrations, etc. Then, once you have a mental image of what you will be doing, get into the math from the ground up.
When I was like 14 years old one one my math teachers explained to us what calculus was using zero equations. He made everything look really easy.
At college, it was the complete opposite. My professor started with limits and convergence, didn't even bother to explain why we were studying the subject.
Kudos for people who actually explain things.
Re: The matrix calculus you need for deep learning (2018)
#10> We assume no math knowledge beyond what you learned in calculus 1 Anyone got recommendations for self-studying and testing Calculus 1?
I found outlier.org to be fantastic.