Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#101

Post by mtzet »

Honestly, I don't think having to learn some stuff before starting anything is nessecary, especially for learning a field as wide as ML/AI. It's much better to start out trying to learning something you're interested in, and then trying to fill in the gaps. This will also help you understand and motivate the underlying theory you're reading.

So for example, start with some source in ML/AI you'd like to read. If you get stuck, ask somewhere (possibly an online forum like this) what field you're having trouble with and how to get started there.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#102

Post by bjourne »

It depends on what "pursuing ML/AI" means. I've written a recommendation engine with barely understanding linear algebra and a spam filter without knowing Bayes theorem. A programmer can work on ML systems without having a solid foundation in higher maths. However, if you want to develop your own solutions then you surely need the math.

I would recommend reading Toby Segaran's Programming Collective Intelligence: http://shop.oreilly.com/product/9780596529321.do

Re: Ask HN: What maths are critical to pursuing ML/AI?

#103

Post by rocqua »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

For calculus, I'd skip the more physics like finding of integrals and derivatives. What matters is understanding the concepts of integrals and derivatives, and knowing properties like the chain rule. It pays much less to know that the integral of 1/x is ln(x) (or the other way round).

The linear algebra and probability theory are most important imho. I'd also distinguish between probability theory and statistics. Both are important, but they are distinct disciplines.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#104

Post by jochenleidner »

1. You can get a long way with high school calculus and probability theory.

2. Regarding books I second the late David McKay's "Information Theory, Inference and Learning Algorithms" and the second edition of "Elements of Statistical Learning" by Tibshirani et al. (there's also a more accessible version of a subset of the material targeting MBA students called James et al., An Introduction to Statistical Learning). Duda/Hart/Stork's Pattern Classification (2nd ed.) is also great. The self-published volume by Abu-Mostafa/Magdon-Ismail/Lin, Learning from Data: A Short Course is impressive, short and useful for self-study.

3. Wikipedia is surprisingly good at providing help, and so is Stack Exchange, which has a statistics sub-forum, and of course there are many online MOOC courses on statistics/probability and more specialized ones on machine learning.

4. After that you will want to consult conference papers and online tutorials on particular models (k-means, Ward/HAC, HMM, SVM, perceptron, MLP, linear and logistic regression, kNN, multinomial naive Bayes, ...).

Re: Ask HN: What maths are critical to pursuing ML/AI?

#105

Post by bluetwo »

srean wrote: post #85
bluetwo wrote: post #50

Not a mention so far about game theory or Nash equilibrium. I'm no expert but does anyone think these apply?

It very much does. Boosting, a one of the best off the shelf ensemble classifier is derived from a game theoretic formulation. Besides that there is this huge body of literature about prediction under non-probabilistic sequence of test cases. This line of work is primarily held up by game theoretic arguments and that of online convex optimization.

I wasn't familiar with Boosting. Now off the read some articles.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#106

Post by rocqua »

tptacek wrote: post #20
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

I'm not super interested in ML but I am very interested in applied mathematics in computer science. I've got a fair bit of linear algebra due to cryptography, but have had virtually no need of any form of calculus (unless I'm relying on it without knowing it) in my career. So beyond just saying that you'd need grounding in multivariable calculus to do serious ML work, I would be super interested in hearing more about…

Much of ML is optimization. This is linked to calculus by derivatives. There is the simple part that at a minimum or maximum the derivative is 0. However, more relevance comes from gradient descent. This depends very heavily on calculating derivatives, and its one of the most universal fast optimization methods.

Beyond that, for iterative methods, convergence is a matter of limits. This again is calculus. Formulating iteration as repeatedly applying a function, we converge to a fixed point of that function if and only if the derivative at that fixed point lies between -1 and 1. Again derivatives come in.

Finally, for error estimation, taylor-expansions are often useful. Again, the topic here is calculus. Notably, all I can think of regards limits and derivatives, not integrals. That might just be due to my hatred of integrals though.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#108

Post by wadams19 »

Totally depends on where you want to land on the engineering-AI-products to pure-AI-research spectrum.

So, what do you mean by "pursuing"?

But even still, I would caution against trying to upload a bunch of new math concepts into your brain without first understanding the ML/AI context.

I would say go through both of Andrew Ng's ML and DL courses on Coursera.

Then, pick a domain/ problem that you're interested in.

Then, read papers about how ML/AI is applied in that domain.

Then, try to reproduce a paper that you understand and are interested in.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#109

Post by rhaps0dy »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

What about "Pattern Recognition and Machine Learning" by Christopher Bishop?

Re: Ask HN: What maths are critical to pursuing ML/AI?

#110

Post by cf »

CuriouslyC wrote: post #25
cf wrote: post #18

Earlier quoted context omitted.

I disagree about the graph theory as well. Unless you are doing things with learning on networks you won't need it. I think a solid background in linear algebra, multivariate calculus, and convex optimization will take you really far.

A lot of data is best represented graphically, and while you can shoehorn this sort of data into a vector space by projecting using a graph distance metric, the results are likely to be inferior.

I agree a lot of data can be represented graphically, but if you look at the literature it mostly is getting shoehorned into vector spaces. This doesn't mean people shouldn't learn about Graph Laplacians and friends, but I don't think it's an entry requirement.
Post reply on HN