Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Matrix Calculus for Deep Learning

parrt.cs.usfca.edu

Post reply on HN
Print view View original

11–20 of 84 posts

Re: Matrix Calculus for Deep Learning

#11

Post by jph00 »

jules wrote: post #5
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

What do you think about the index notation physicists use for tensor calculus?

I don't find it very accessible, myself - but I'm not a physicist, so the materials using or about that notation aren't aimed at me.

The only tensor notation I've been happy with is that used in J (http://www.jsoftware.com), which is simple, flexible, and concise.

There's also some nice-enough modern notation used in this excellent review: http://www.cs.cmu.edu/~christos/courses/826-resources/PAPERS...

Re: Matrix Calculus for Deep Learning

#12

Post by parrt »

jacobolus wrote: post #8
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

Typographic advice: the body text has very long lines in a desktop browser, which makes it a bit slow and tiring to read. I’d say the ideal is somewhere between 1/2 and 2/3 this length. I’d recommend keeping the same width on screen but bumping the font size up by 30%. As an extra minor nit, italicizing functions like sin , etc. is also somewhat unconventional in mathematical typesetting.

I agree that the font should be bigger. I need to learn more CSS in order to switch between font sizes per platform. The font of the text is easy but all of the images were generated from latex using a specific font size. I need to scale the in-line equation images as the font size bumps up.

Re: Matrix Calculus for Deep Learning

#13

Post by jtmcmc »

jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

This is great! my graduate advisor a long time ago made a really great matrix calculus study sheet for me that was absolutely invaluable in learning ML ( i mean really this is great for not just DL but all sorts of reasoning in ML)

Re: Matrix Calculus for Deep Learning

#14

Post by parrt »

jtmcmc wrote: post #13
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

This is great! my graduate advisor a long time ago made a really great matrix calculus study sheet for me that was absolutely invaluable in learning ML ( i mean really this is great for not just DL but all sorts of reasoning in ML)

We originally had that generic ML target in mind but figured a DL bent would make it a wee bit more interesting.

Re: Matrix Calculus for Deep Learning

#16

Post by ransom1538 »

marrowgari wrote: post #3

If you're looking at this with the intention of getting started in Deep Learning and feeling overwhelmed by the math then Andrew Ng offers a great course on Coursera that goes over all of the formulas needed to calculate the forward propagation, loss computation, backward propagation, and gradient descent. Highly recommend it for anyone interested in breaking into the field of machine learning.

It is also all free on youtube: https://www.youtube.com/watch?v=UzxYlbK2c7E

Re: Matrix Calculus for Deep Learning

#18

Post by wuliwong »

ransom1538 wrote: post #16
marrowgari wrote: post #3

If you're looking at this with the intention of getting started in Deep Learning and feeling overwhelmed by the math then Andrew Ng offers a great course on Coursera that goes over all of the formulas needed to calculate the forward propagation, loss computation, backward propagation, and gradient descent. Highly recommend it for anyone interested in breaking into the field of machine learning.

It is also all free on youtube: https://www.youtube.com/watch?v=UzxYlbK2c7E

They also have it all on Stanford's site with some other information and course materials.

https://see.stanford.edu/Course/CS229

Re: Matrix Calculus for Deep Learning

#19

Post by [deleted] »

jph00 wrote: post #11
jules wrote: post #5

Earlier quoted context omitted.

What do you think about the index notation physicists use for tensor calculus?

I don't find it very accessible, myself - but I'm not a physicist, so the materials using or about that notation aren't aimed at me. The only tensor notation I've been happy with is that used in J ( http://www.jsoftware.com ), which is simple, flexible, and concise. There's also some nice-enough modern notation used in this excellent review: http://www.cs.cmu.edu/~christos/courses/826-resources/PAPERS...

[deleted]

Re: Matrix Calculus for Deep Learning

#20

Post by posterboy »

jacobolus wrote: post #8
jph00 wrote: post #4

Jeremy here. Here to answer any questions or comments that you have. But more importantly - I need to mention that Terence Parr did nearly all the work on this. He shared my passion for making something that anyone could read on any device to such an extent that he ended up creating a new tool for generating fast, mobile-friendly math-heavy texts: https://github.com/parrt/bookish . (We tried Katex, Mathjax, and prett…

Typographic advice: the body text has very long lines in a desktop browser, which makes it a bit slow and tiring to read. I’d say the ideal is somewhere between 1/2 and 2/3 this length. I’d recommend keeping the same width on screen but bumping the font size up by 30%. As an extra minor nit, italicizing functions like sin , etc. is also somewhat unconventional in mathematical typesetting.

it's a variable name for a polymorphic function, taylor series, euler formular, etc. depending on context. This is a significant difference to singular types, but being a variable name to an abstract concept is in principle no different to typesetting x. This goes neatly with "everything is an objectreference" and might be more of a programmer's perspective.
Post reply on HN