Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#41

Post by jules »

tptacek wrote: post #20
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

I'm not super interested in ML but I am very interested in applied mathematics in computer science. I've got a fair bit of linear algebra due to cryptography, but have had virtually no need of any form of calculus (unless I'm relying on it without knowing it) in my career. So beyond just saying that you'd need grounding in multivariable calculus to do serious ML work, I would be super interested in hearing more about…

Most of ML is fitting models to data. To fit a model you minimise some error measure as a function of its real valued parameters, e.g. the weights of the connections in a neural network. The algorithms to do the minimisation are based on gradient descent, which depends on derivatives, i.e. differential calculus.

If you're doing Bayesian inference you're going to need integral calculus because Bayes' law gives the posterior distribution as an integral.

For ML you just need Calculus 1 and 2. The curl/div and Stokes is Calculus 3 which a physics thing. You don't need that for ML.

You may need the basics of functional analysis in certain areas of ML, which is arguably Calculus 4.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#42

Post by leecarraher »

It will depend on the level you plan to engage in the ML/AI space. If you just want a job in ML/AI , you are in luck. Due to the growing assortment of available, mostly to fully automated, solutions like Datarobot, H2O, sckit-learn, keras(w/ tensorflow) the only math you will absolutely 'need' is probably just Statistics. Regardless of what's going on behind the scenes with whatever automatically tuned and selected algorithm your chosen solutions uses, you will still need some stats in the end to show the brass that 'your' model works. the upside is that then you can spend time, learning feature extraction, data engineering, and the aforementioned toolkits, in particular what models they make available.

If you want to develop new techniques and algorithms, the the skies the limit, you'll of course want Stats too though.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#44

Post by rcarrigan87 »

leecarraher wrote: post #42

It will depend on the level you plan to engage in the ML/AI space. If you just want a job in ML/AI , you are in luck. Due to the growing assortment of available, mostly to fully automated, solutions like Datarobot, H2O, sckit-learn, keras(w/ tensorflow) the only math you will absolutely 'need' is probably just Statistics. Regardless of what's going on behind the scenes with whatever automatically tuned and selected a…

Can you recommend a Stats course that would be most relevant for people trying to be more practitioners (not researchers)?

Re: Ask HN: What maths are critical to pursuing ML/AI?

#45

Post by septimus111 »

Not knowing anything about you, I'll assume that

- you are starting with the equivalent of a high school level of maths

- you want to take a ML course or read an ML book without feeling totally lost

As some commenters have said, Calculus, Probability and Linear Algebra will be very helpful.

Some people like to recommend the "best" or "most important" books which you "should" read, but there is a strong chance these will end up sitting on a bookshelf, barely touched. So I will recommend some books which are perhaps more accessible.

- Calculus by Gilbert Strang

- Linear Algebra by Gilbert Strang

For Probability: I don't have any favourites, sorry.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#46

Post by septimus111 »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

These are all brilliant books, but I feel like anyone who is ready for them wouldn't need to be asking this question.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#47

Post by septimus111 »

irchans wrote: post #22

Basic probability is very helpful: Expectation, Standard deviation, P(A and B) = P(A)*P(B) is A and B are independent, P(A or B) = P(A)+P(B) if A and B are mutually exclusive. Also, knowing algebra is very helpful. In a way, you don't really need to know much more because there is a lot of good software out there. If you want to learn more math, learn Linear Regression, Logistic Regression, p-values, probability dens…

This is a great list of the main concepts to know.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#48

Post by jmh530 »

digitalzombie wrote: post #26
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

> "Bayesian Data Analysis" by Andrew Gelman is another great read. If you want to read that book you need real analysis more specifically measure theory (unless that subject is in probability theory for you). You cannot get into the last few chapters without it. Dirichlet Process are described using measures. I don't believe you need multivar calc or info theory. Info theory stuff are used but not as often. I believe…

My recollection is that you need neither real analysis nor measure theory to appreciate it, but it's been a while since I read it. You might get more out of it if you have studied those.

I disagree on multivar calc. Statistics often makes use of matrix derivatives. I have found it helpful to know.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#49

Post by anst »

leecarraher wrote: post #42

It will depend on the level you plan to engage in the ML/AI space. If you just want a job in ML/AI , you are in luck. Due to the growing assortment of available, mostly to fully automated, solutions like Datarobot, H2O, sckit-learn, keras(w/ tensorflow) the only math you will absolutely 'need' is probably just Statistics. Regardless of what's going on behind the scenes with whatever automatically tuned and selected a…

What "maths" is keras? Or scikit-learn? For what it's worth, to understand scikit-learn doc/tutorial I'd say you'll need Probability, Linear Algebra, Multivariate Calculus and, yeah, Stats. Not necessarily at a PhD level but still. And more you understand maths farther you can get in AL/ML.
Post reply on HN