Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#81

Post by kgwgk »

rdudekul wrote: post #69
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

Free PDFs of some of the books mentioned: "Information Theory, Inference and Learning Algorithms" by David MacKaye http://www.inference.org.uk/itprnn/book.pdf "Probability Theory: the Logic of Science" by E. T. Jaynes http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia... "Elements of Statistical Learning" by Tibshirani https://web.stanford.edu/~hastie/Papers/ESLII.pdf "Bayesian Data Analysis" by Andrew Gelm…

Note that only MacKay (that’s the correct spelling) and Hastie/Tibshirani/Friedman are legally available online.

edit: Goodfellow/Bengio/Courville, not mentioned in the previous comment, is also available online: http://www.deeplearningbook.org

Re: Ask HN: What maths are critical to pursuing ML/AI?

#82

Post by catnaroek »

mindcrime wrote: post #77
digitalzombie wrote: post #26

Earlier quoted context omitted.

> "Bayesian Data Analysis" by Andrew Gelman is another great read. If you want to read that book you need real analysis more specifically measure theory (unless that subject is in probability theory for you). You cannot get into the last few chapters without it. Dirichlet Process are described using measures. I don't believe you need multivar calc or info theory. Info theory stuff are used but not as often. I believe…

What's required as a prereq to Measure Theory? Any suggestions on good resources for learning Measure Theory? I have a vague notion that Probability and Measure Theory are intertwined / related somehow, but have never studied the latter specifically.

I'm taking a measure theory course right now, and we primarily use some set theory and some topology of R^n.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#83

Post by mindcrime »

mendeza wrote: post #65

I wish the people who answer this question are people that are current deep learning engineers or data scientist that use deep learning in real world settings, I am worried that people who are not credible are giving advice, which is not valuable. I am a masters student taking a PhD class in Bayesian machine learning to figure this out as well. I hope to have a better answer for this by the end of the course!

I wish the people who answer this question are people that are current deep learning engineers or data scientist that use deep learning in real world settings,

Why do you want answers only from people doing deep learning? Deep learning is just a subset of the overall field (albeit an incredibly popular and useful one).

Anyway, the simple solution is just to use some simple machine learning of your own to analyze the data set which these threads constitute. Look for patterns... are certain answers being repeated over and over again, by different posters? Then I'd argue that your Bayesian posterior for "this is legitimately important" should go up.

Take Linear Algebra for example... given the sheer number of people saying "linear algebra" in their answers, it seems a reasonably bet to me that LA is really, truly useful. Either that or there's some really freaking group-think shit going on. :-)

Re: Ask HN: What maths are critical to pursuing ML/AI?

#84

Post by thanatropism »

leecarraher wrote: post #58
anst wrote: post #49

Earlier quoted context omitted.

What "maths" is keras? Or scikit-learn? For what it's worth, to understand scikit-learn doc/tutorial I'd say you'll need Probability, Linear Algebra, Multivariate Calculus and, yeah, Stats. Not necessarily at a PhD level but still. And more you understand maths farther you can get in AL/ML.

Those are ML and AI frameworks that use a tremendous amount of mathematics under the hood, but you can also reliably treat them as blackbox learning systems too. Understanding the model generation procedure and setup is often unneeded. And many tools will help direct you toward what algorithms makes the most sense for your data, and even have competitions to figure out which actually works best. I agree, it's a littl…

Competition (labor supply side) and ultimate success of current ML approaches.

As the market starts to overheat, it seems that there will be a labor shortage/good quality workers will be scarce and we'll have to make simple tools for simpletons. But this is all a huge "if". Eventually the market will contract a lot and slack labor market conditions will have companies hiring them PhDs.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#85

Post by srean »

bluetwo wrote: post #50

Not a mention so far about game theory or Nash equilibrium. I'm no expert but does anyone think these apply?

It very much does. Boosting, a one of the best off the shelf ensemble classifier is derived from a game theoretic formulation. Besides that there is this huge body of literature about prediction under non-probabilistic sequence of test cases. This line of work is primarily held up by game theoretic arguments and that of online convex optimization.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#88

Post by laurus »

graycat wrote: post #71
graycat wrote: post #70

Part I (1) Calculus Generally should have college freshman and sophomore calculus. (1.1) Functions So, there can understand better what a function is. E.g., function f(x) = 3x^2 + 1. (1.2) Derivatives Then will learn how to find the slope of the graph of a function. That is the derivative of the function. E.g., for function f with f(x) = 3x + 2, as in high school algebra, the slope is 3. Then for each x, the derivati…

Part II (2) Linear Algebra (2.1) Linear Equations The start of linear algebra was seen in high school algebra, solving systems of linear equations. E.g., we seek numerical values of x and y so that 3 x - 2 y = 7 -x + 2 y = 8 So, that is two equations in the two unknowns x and y. Well, for positive integers m and n, we can have m linear ( linear is in the above example but omitting here a careful definition) equations…

I have been inspired by some of your past posts suggesting a path for studying mathematics and doing graduate level work, and have changed my direction to try and follow what you suggest. Is there any way I can get in touch with you privately? (I'm not looking for help with specific technical questions if you're concerned about that.)

Re: Ask HN: What maths are critical to pursuing ML/AI?

#89

Post by capkutay »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

Do you have any good guides for the calculus required to do ML? Is it just the basic Calc AB from high school?

Re: Ask HN: What maths are critical to pursuing ML/AI?

#90

Post by torbjorn »

mindcrime wrote: post #9

It depends on how deep you want to go and what your goals are, but I'd say that CuriouslyC pretty much nailed it. Multi-variable calculus, linear algebra, and probability / stats are definitely the core. If you're interested in finding more "freely available online" maths references, check out: http://people.math.gatech.edu/~cain/textbooks/onlinebooks.ht... http://www.openculture.com/free-math-textbooks https://open.…

3blue1brown is great.

I also recommend Siraj Raval's Youtube course the Math of Intelligence: https://www.youtube.com/watch?v=xRJCOz3AfYY&list=PL2-dafEMk2...

Post reply on HN