Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Matrix Calculus for Deep Learning

parrt.cs.usfca.edu

Post reply on HN
Print view View original

41–50 of 84 posts

Re: Matrix Calculus for Deep Learning

#41

Post by jph00 »

smith-kyle wrote: post #29
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

Hi Jeremy, thanks for creating so many great learning resources. Myself and a fellow ML enthusiast are starting a deep learning meetup here in Phoenix next month. Do you have any advice for creating a welcoming environment for people to learn/teach? Thanks!

Probably better asking that at http://forums.fast.ai , since a lot of participants there have set up local meetups, and some of them have gotten pretty big. I haven't created a meetup myself - my focus has been on the online course and community, frankly.

Re: Matrix Calculus for Deep Learning

#42

Post by jph00 »

smrtinsert wrote: post #40

Thanks for this. Was taking Andrew Ng's course but the way he glosses over the calculus and then expects the student to understand the implications at end of lecture was a turn off so I dropped it. I hated the feeling I wasn't learning, just memorizing solutions.

You might prefer the approach at http://course.fast.ai - all the concepts are taught with code, instead of math, and understanding is developed by running experiments.

Re: Matrix Calculus for Deep Learning

#43

Post by [deleted] »

smrtinsert wrote: post #40

Thanks for this. Was taking Andrew Ng's course but the way he glosses over the calculus and then expects the student to understand the implications at end of lecture was a turn off so I dropped it. I hated the feeling I wasn't learning, just memorizing solutions.

[deleted]

Re: Matrix Calculus for Deep Learning

#44

Post by parrt »

edhu2017 wrote: post #39
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

In the article, you say the gradient of 2x + y^8 is [1, 7y^8]. Shouldn't it be [2, 7y^8]?

Oops. yeah. thanks

Re: Matrix Calculus for Deep Learning

#45

Post by improbable22 »

calebh wrote: post #33

Fortunately there is a website now capable of doing matrix calculus! http://www.matrixcalculus.org Mathematica doesn't seem to be able to do matrix calculus, which surprised me quite a bit.

Mathematica can absolutely do all of this! D[ matrix ,x] ... or things like this:

f[x_,y_] := x^2 + Sin[y]

vars={x,y};

Table[ D[f[x,y],var1, var2], {var1,vars}, {var2,vars}] // MatrixForm

Re: Matrix Calculus for Deep Learning

#46

Post by danbruc »

edhu2017 wrote: post #39
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

In the article, you say the gradient of 2x + y^8 is [1, 7y^8]. Shouldn't it be [2, 7y^8]?

Now you are just replacing one error with another, you mean [2, 8y^7]. ;)

Re: Matrix Calculus for Deep Learning

#48

Post by SoerenL »

improbable22 wrote: post #45
calebh wrote: post #33

Fortunately there is a website now capable of doing matrix calculus! http://www.matrixcalculus.org Mathematica doesn't seem to be able to do matrix calculus, which surprised me quite a bit.

Mathematica can absolutely do all of this! D[ matrix ,x] ... or things like this: f[x_,y_] := x^2 + Sin[y] vars={x,y}; Table[ D[f[x,y],var1, var2], {var1,vars}, {var2,vars}] // MatrixForm

But how do you compute the derivative of x'Ax in Mathematica (x being a vector and A being a matrix)? What you have pointed out is only scalar derivatives, if I am not mistaken here.

Re: Matrix Calculus for Deep Learning

#49

Post by vervez »

smrtinsert wrote: post #40

Thanks for this. Was taking Andrew Ng's course but the way he glosses over the calculus and then expects the student to understand the implications at end of lecture was a turn off so I dropped it. I hated the feeling I wasn't learning, just memorizing solutions.

I've taken 3/5 of his Deep Learning MOOC and thought the calculus, at least its implementation, was very well explained. Were you dissatisfied with the lack of depth or lack of explanation? Just curious.

Re: Matrix Calculus for Deep Learning

#50

Post by KSS42 »

lprd wrote: post #22

In school, I didn't make it much past basic calculus/algebra. As a self-taught programmer (my highest level of education is a high-school diploma), I seriously wish I could go back and put more effort into math. I love looking at these types of topics, but I have absolutely no clue what I'm looking at. If anyone can recommend any books, courses, or any other material that starts from high-school level math, and gradu…

I would start by watching the Linear Algebra and Calculus videos by 3Blue1Brown. This will give you an intuitive understanding of both.

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53...

Post reply on HN