Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Matrix Calculus You Need for Deep Learning

explained.ai

Post reply on HN
Print view View original

71–79 of 79 posts

Re: The Matrix Calculus You Need for Deep Learning

#71

Post by RobertDeNiro »

tachyonbeam wrote: post #61
RobertDeNiro wrote: post #57

Earlier quoted context omitted.

You don't need much CS knowledge for deep learning, in fact pure math/stats is more useful. I'd say CS knowledge is only useful if you do RL stuff and even then you don't really need it that much.

That really depends on what you're doing. I've seen people coming from a pure math background really struggle. If you've never programmed before, then using something like TensowFlow isn't exactly going to feel natural. Familiarity with Python and the Linux ecosystem would definitely be quite helpful. Yes, I'm sure you could run ML models on a Windows machine, but at some point, you might want to perform experiments…

True. I was assuming programing experience but no data structure and algorithms. I can attest that a lot of people in my ML class couldn't program and they sucked at the assignments we had.

Re: The Matrix Calculus You Need for Deep Learning

#72

Post by RobertDeNiro »

hypersoar wrote: post #60
RobertDeNiro wrote: post #57

Earlier quoted context omitted.

You don't need much CS knowledge for deep learning, in fact pure math/stats is more useful. I'd say CS knowledge is only useful if you do RL stuff and even then you don't really need it that much.

Tell that to my potential employers :|

Depends what jobs you're applying to. But I've seen it first hand traditional HR doesn't know how to filter for ML resumes.

Re: The Matrix Calculus You Need for Deep Learning

#73

Post by Eridrus »

mindviews wrote: post #63
hypersoar wrote: post #37

Does anyone know of good resources for studying machine learning or data science given a strong mathematical background? I'm transitioning careers from pure math research into industry. I know very little about machine learning, but I know the crap out of linear algebra and real analysis (and other, less relevant fields). It'd be great to have some sources that leverage that without assuming much prior CS knowledge.

For Machine Learning, I found "Learning From Data" http://amlbook.com/ to be a very strong, foundational book. It does the best job of any reference I've seen of answering the question: "Is learning possible and feasible?" Provides and excellent mathematical foundation for understanding when and why ML systems should succeed and fail. Associated MOOC from Caltech at https://work.caltech.edu/telecourse.html

I'm a big fan of the MOOC, even though it didn't explicitly cover it, it basically explained what people mean when they say "we don't know why neural nets work", which is that our previous generalization bounds would imply that neural nets would not generalize well, but in practice they do.

Re: The Matrix Calculus You Need for Deep Learning

#74

Post by Myrmornis »

pvg wrote: post #35
Myrmornis wrote: post #29

Earlier quoted context omitted.

I understand what you’re trying to say, but Wikipedia is a fantastic resource for mathematics. “Pretty awful” is not a correct choice of words. But yes, much of it is written at beyond-undergrad-math level. And undergrad math is already advanced! And no I’m not someone with a math PhD talking down! I’m struggling through teaching myself undergrad math.

The only way "pretty awful" is incorrect is that it is too polite and reserved. Reams upon reams of pages are written completely at odds with Wikipedia's own style guidelines and common-sense expectations of what one might find in an encyclopedia. Unlike some famously dense mathematical texts, wikipedia maths pages don't even come with any of the benefits of brevity or focus. It's like a giant joke competition of who…

We’re saying much the same thing, it’s just that I find your and GP’s use of the absolute “pretty awful” to be hyperbolic and something of a loss of perspective. Remember what we have here: a free, actively maintained, accurate, comprehensive and advanced corpus of expository writing on mathematics. Adjectives that are missing there are “intuitition-rich”, and “helpful for undergraduates”. I do understand if you are disappointed with it. As noted, I (undergrad level) don’t approach it with an expectation that it will be my favorite reading on a topic.

Re: The Matrix Calculus You Need for Deep Learning

#75

Post by poster123 »

srean wrote: post #70
hypersoar wrote: post #37

Does anyone know of good resources for studying machine learning or data science given a strong mathematical background? I'm transitioning careers from pure math research into industry. I know very little about machine learning, but I know the crap out of linear algebra and real analysis (and other, less relevant fields). It'd be great to have some sources that leverage that without assuming much prior CS knowledge.

Given your math bent expecially analysis, I would recommend you start with Luc Devroye, follow it up with Vapnik. These are a bit dated but still very relevant. After these two go with Alex Smola and Bernard Scholkopf. The books that have been suggested to you so far aren't bad but with high fluff to meat ratio, not meant for a reader like you. If you have function analysis in your bag you will feel right at home wit…

Thanks for the recommendations. Devroye wrote several books, and I assume the one being mentioned is "A Probabilistic Theory of Pattern Recognition".

Re: The Matrix Calculus You Need for Deep Learning

#76

Post by poster123 »

srean wrote: post #70
hypersoar wrote: post #37

Does anyone know of good resources for studying machine learning or data science given a strong mathematical background? I'm transitioning careers from pure math research into industry. I know very little about machine learning, but I know the crap out of linear algebra and real analysis (and other, less relevant fields). It'd be great to have some sources that leverage that without assuming much prior CS knowledge.

Given your math bent expecially analysis, I would recommend you start with Luc Devroye, follow it up with Vapnik. These are a bit dated but still very relevant. After these two go with Alex Smola and Bernard Scholkopf. The books that have been suggested to you so far aren't bad but with high fluff to meat ratio, not meant for a reader like you. If you have function analysis in your bag you will feel right at home wit…

The book "A Probabilistic Theory of Pattern Recognition" by Devroye, Györfi and Lugosi is available from the site of Gyorfi at http://www.szit.bme.hu/~gyorfi/pbook.pdf .

Re: The Matrix Calculus You Need for Deep Learning

#77

Post by srean »

poster123 wrote: post #75
srean wrote: post #70

Earlier quoted context omitted.

Given your math bent expecially analysis, I would recommend you start with Luc Devroye, follow it up with Vapnik. These are a bit dated but still very relevant. After these two go with Alex Smola and Bernard Scholkopf. The books that have been suggested to you so far aren't bad but with high fluff to meat ratio, not meant for a reader like you. If you have function analysis in your bag you will feel right at home wit…

Thanks for the recommendations. Devroye wrote several books, and I assume the one being mentioned is "A Probabilistic Theory of Pattern Recognition".

Yes that's the one I had in mind. His other books are also very good, but are more specialized.

You may also enjoy graycat's comments on HN. He know his math but is contrarian about machine learning but its good to have that view point.

Re: The Matrix Calculus You Need for Deep Learning

#78

Post by glitch »

soVeryTired wrote: post #41
hypersoar wrote: post #37

Does anyone know of good resources for studying machine learning or data science given a strong mathematical background? I'm transitioning careers from pure math research into industry. I know very little about machine learning, but I know the crap out of linear algebra and real analysis (and other, less relevant fields). It'd be great to have some sources that leverage that without assuming much prior CS knowledge.

For non-deep learning, read David Barber's book: http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/090310.pdf Some sections may be less relevant, depending on what you want to do, but Section III is a very good introduction to machine learning methods. Do the exercises as you're reading. Theory is one thing, but in ML my rule of thumb is that you don't really understand a model until you've coded it up. A collection o…

Note: 090310.pdf (9 March 2010) is an older version of the textbook. The latest revision of the textbook PDF (091117.pdf, 9 November 2017) is available at:

→ http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/091117.pdf

Aside:

⸰ Textbook homepage: http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...

⸰ Online version homepage (should always contain link to latest revision): http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...

⸰ Directory sorted by date: http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/?C=M;O=D

Re: The Matrix Calculus You Need for Deep Learning

#79

Post by pvg »

Myrmornis wrote: post #74
pvg wrote: post #35

Earlier quoted context omitted.

The only way "pretty awful" is incorrect is that it is too polite and reserved. Reams upon reams of pages are written completely at odds with Wikipedia's own style guidelines and common-sense expectations of what one might find in an encyclopedia. Unlike some famously dense mathematical texts, wikipedia maths pages don't even come with any of the benefits of brevity or focus. It's like a giant joke competition of who…

We’re saying much the same thing, it’s just that I find your and GP’s use of the absolute “pretty awful” to be hyperbolic and something of a loss of perspective. Remember what we have here: a free, actively maintained, accurate, comprehensive and advanced corpus of expository writing on mathematics. Adjectives that are missing there are “intuitition-rich”, and “helpful for undergraduates”. I do understand if you are…

We’re saying much the same thing

No.

something of a loss of perspective.

You'd have to provide some alternative perspective or argument that goes beyond 'pretty awful sounds kinda mean'.

Remember what we have here: a free, actively maintained, accurate, comprehensive and advanced corpus of expository writing on mathematics

We already have a few of those. As I mentioned, mathworld is far better at this and it's been around longer than Wikipedia.

Post reply on HN