Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#11

Post by gtani »

For going all in, https://ocw.mit.edu/courses/mathematics/18-657-mathematics-o...

But a good number of people that are doing work haven't taken real analysis, or it's been awhile and so you should be current on multivariable and vector calculus. Calculus of variations shows up from time to time.

For math reviews, look at the following (there's others if you want more refs, ping me):

http://www.deeplearningbook.org/

https://metacademy.org/roadmaps/

http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning...

Re: Ask HN: What maths are critical to pursuing ML/AI?

#13

Post by ivansavz »

Probability theory and linear algebra are pretty much the core. Learning LA will help you become comfortable with multi-dimensional quantities, vector spaces, and give you some powerful computational techniques, e.g., SVG==PCA.

Here is a short tutorial on linear algebra: https://minireference.com/static/tutorials/linear_algebra_in... and a preview of the full book: https://minireference.com/static/excerpts/noBSguide2LA_previ...

Re: Ask HN: What maths are critical to pursuing ML/AI?

#15

Post by blt »

If you want to understand SVMs deeply, a course in convex optimization. In general, proving maximum likelihood estimation for a lot of classic machine learning models involves using the method of Lagrange multipliers. But not deep neural networks :)

Re: Ask HN: What maths are critical to pursuing ML/AI?

#16

Post by Myrmornis »

It depends whether you want to work more as an engineer / data analyst, or more as a "ML researcher". For the latter, then, yes, as everyone says below, you need to be totally comfortable with multivariable calculus, linear algebra, probability and statistics, numerical optimization etc. But many jobs are more practical in nature, in which the main case essential skill is, being able to run a bunch of different models with different parameter values and collect and interpret the results, efficiently and reproducibly, and be able to talk about them and make recommendations for the way forwards. In those jobs you're not actually going to need to be able to derive updates for backpropagation, even though it's certainly satisfying to understand it.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#18

Post by cf »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

I disagree about the graph theory as well. Unless you are doing things with learning on networks you won't need it.

I think a solid background in linear algebra, multivariate calculus, and convex optimization will take you really far.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#20

Post by tptacek »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

I'm not super interested in ML but I am very interested in applied mathematics in computer science. I've got a fair bit of linear algebra due to cryptography, but have had virtually no need of any form of calculus (unless I'm relying on it without knowing it) in my career.

So beyond just saying that you'd need grounding in multivariable calculus to do serious ML work, I would be super interested in hearing more about why that is and what kinds of problems crop up in ML that demand it.

Post reply on HN