Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

61–70 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#61

It will depend on the level you plan to engage in the ML/AI space. If you just want a job in ML/AI , you are in luck. Due to the growing assortment of available, mostly to fully automated, solutions like Datarobot, H2O, sckit-learn, keras(w/ tensorflow) the only math you will absolutely 'need' is probably just Statistics. Regardless of what's going on behind the scenes with whatever automatically tuned and selected a…

Can you recommend a Stats course that would be most relevant for people trying to be more practitioners (not researchers)?

This https://www.amazon.com/Probability-Statistics-Engineers-Scie... is the newer version of the stats book i had in undergrad, But @anst makes a good point about scikit learn. there is alot of good math to learn just from the docs and you can then investigate further on wiki, quora, stackexchange.

for the what's up in Data Science i like datatau.com. and there are some great podcasts too, like datascienceathome and partiallyderivative (there are lists).

Re: Ask HN: What maths are critical to pursuing ML/AI?

#62
post #39

Earlier quoted context omitted.

I disagree that you need a solid founding in information theory. Almost all that I've seen about IT in ML is minimizing the KL divergence, which can be learned by browsing the wiki page.

Well, information theory isn't much more than the logarithm of probability theory, so it doesn't hurt to learn it anyway. The only thing you need to know is that given a probability distribution P there exist a compression scheme to encode a value X with a message of P_length(X) = log(1/P(X)) bits. This can be summarised as BITS = log(1/PROBABILITY). Entropy is just the average number of bits you need to encode a ran…

> information theory isn't much more than the logarithm of probability theory

stealing

Re: Ask HN: What maths are critical to pursuing ML/AI?

#63
post #50

Not a mention so far about game theory or Nash equilibrium. I'm no expert but does anyone think these apply?

Not a mention so far about game theory or Nash equilibrium.

It depends on what you're doing. I was literally just watching a video on Generative Adversarial Networks this morning, and game theory did come up there, at least in passing. If one sat down and started reading the papers on this subject and trying to implement / improve stuff in this area, I suspect game theory would be at least moderately important.

There is also the field of Competitive Learning where game theory has some application. See, for example:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.71....

Re: Ask HN: What maths are critical to pursuing ML/AI?

#64

Earlier quoted context omitted.

Want to try using a completely automated black-box ML pipeline like TPOT? Go right ahead. Good luck selling it to your product manager.

Can you please expand on this comment for us ML/AI naive?

There exists a tool called TPOT (Tree-based Pipeline Optimization Tool) [0] that aims to automate the knob-twiddling that tends to go with optimizing Machine Learning models. As these models often have a number of parameters to tune and tweak over large scales, such a tool can be useful to identify performant combinations of these parameters and save time in doing so.

However, many ML practitioners are wary of similar automated ML pipelines, especially as they focus on non-expert users. A huge part of "data science" is the "data" itself. It often has idiosyncrasies and quirks that must be identified and accounted for in any model that hopes to make useful predictions. There are many pitfalls that come from not understanding the base statistical/mathematical assumptions of these tools, and a simplified Automatic ML Suite runs the risk of providing misleading results when used as a one-size-fits-all solution. Even for expert users, such tools often make it difficult (either by mathematical need or software design) to interpret the reasons and causes for their results. "Black boxes" like this are definitely hard to sell up the chain.

[0] https://github.com/rhiever/tpot

Re: Ask HN: What maths are critical to pursuing ML/AI?

#65
I wish the people who answer this question are people that are current deep learning engineers or data scientist that use deep learning in real world settings, I am worried that people who are not credible are giving advice, which is not valuable. I am a masters student taking a PhD class in Bayesian machine learning to figure this out as well. I hope to have a better answer for this by the end of the course!

Re: Ask HN: What maths are critical to pursuing ML/AI?

#66

It will depend on the level you plan to engage in the ML/AI space. If you just want a job in ML/AI , you are in luck. Due to the growing assortment of available, mostly to fully automated, solutions like Datarobot, H2O, sckit-learn, keras(w/ tensorflow) the only math you will absolutely 'need' is probably just Statistics. Regardless of what's going on behind the scenes with whatever automatically tuned and selected a…

Can you recommend a Stats course that would be most relevant for people trying to be more practitioners (not researchers)?

I found this course to be very helpful, it has a good balance of reading material and labs to apply what you learn. The course is from the Austin’s Department of Statistics and Data Sciences.

[Foundations of Data Analysis](https://courses.edx.org/courses/course-v1:UTAustinX+UT.7.11x...)

Note: In this course, Dr. Michael J. Mahometa uses R. But I'd recommend you not to focus on R vs Python debates; the goal of this course is to learn about Statistics & Data Analysis in real-world scenarios. With that in mind, even just going through the reading material and lecture videos will be valuable enough if you're starting from scratch (but I'd recommend you to take the extra step and complete the Labs too).

Re: Ask HN: What maths are critical to pursuing ML/AI?

#67
> What maths must be understood to enable pursuit of either of the above fields?

None.

> Are there any seminal texts/courses/content which should be consumed before starting?

No.

You don't need to know binary to start being a programmer/developer either. Just start already. As long as you are not in charge of a medical diagnosis or financial model, you don't get any drawback in experimenting (and failing miserably).

Assuming applied ML, the most difficult part will be the human-political business element of it: People not understanding your model or using its output correctly, bias, feedback loops, acquiring enough resources, etc. The more you can explain to them, without resorting to heavy maths, the better communicator you are.

That said, it can't hurt to do Ng's Coursera course (a lot of top performers started out with this course). Learning from Data by Caltech's Abu-Mostafa goes very wide on machine learning. "Programming Collective Intelligence" is a, somewhat dated, good book.

As for seminal texts, the field is too wide for this. A better bet is: Find a professor in the field you are interested in. Say "Deep Learning", you could have a look at LeCun, Hinton, Schmidhuber, Bengio, ... Now look at their PhD-students, their papers, their courses, their conference talks, their software, their current research. Basically become a student under the most authoritative professor in the subfield you can find and resonate with, without ever paying any university tuition or them knowing you exist. This is very possible these days.

But by all means: Just start out. Machine learning is fun. Learning about dry 100 year old maths not so much. Make mistakes. Learn to detect and avoid overfit. Find out if you are passionate and curious about parts of the field, then the theory will come eventually. A lot of the time these questions seem to demand answers like: "You need a PhD-level understanding of mathematics" Just so your brain can go: "I am not good enough for this, so let's look at something easier". Don't use this as an excuse. Start making intelligent stuff. There are 16-year-olds on Kaggle routinely beating maths PhD's.

Also remember that, despite the current trend of calling everything "AI", that AI is a very wide field, of which mathematics is only a small part. There is philosophy, linguistics, cognitive science, physics, neuroscience, psychology, computer science, robotics, logic, ... all these parts vary wildly in their prerequisite maths knowledge.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#68
post #65

I wish the people who answer this question are people that are current deep learning engineers or data scientist that use deep learning in real world settings, I am worried that people who are not credible are giving advice, which is not valuable. I am a masters student taking a PhD class in Bayesian machine learning to figure this out as well. I hope to have a better answer for this by the end of the course!

>I wish the people who answer this question are people that are current deep learning engineers or data scientist that use deep learning in real world settings

There simply aren't very many people in those roles because the number of ML/AI/DL jobs out there are still limited, I think.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#69

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

Free PDFs of some of the books mentioned:

"Information Theory, Inference and Learning Algorithms" by David MacKaye

http://www.inference.org.uk/itprnn/book.pdf

"Probability Theory: the Logic of Science" by E. T. Jaynes

http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...

"Elements of Statistical Learning" by Tibshirani

https://web.stanford.edu/~hastie/Papers/ESLII.pdf

"Bayesian Data Analysis" by Andrew Gelman

http://hbanaszak.mjr.uw.edu.pl/TempTxt/(Chapman%20&%20Hall_C...

Re: Ask HN: What maths are critical to pursuing ML/AI?

#70
Part I

(1) Calculus

Generally should have college freshman and sophomore calculus.

(1.1) Functions

So, there can understand better what a function is. E.g., function

     f(x) = 3x^2 + 1.
(1.2) Derivatives

Then will learn how to find the slope of the graph of a function. That is the derivative of the function. E.g., for function f with f(x) = 3x + 2, as in high school algebra, the slope is 3. Then for each x, the derivative of f at x is just 3.

The derivative of function f is denoted by either of

     f'(x) = d/dx f(x)
E.g., for function f(x) = 3x^2 + 1 it turns out that

     f'(x) = 6x.
(1.3) Integration

For function

     g(x) = 6 x
maybe we want to know what function f(x) will give us

     f'(x) = g(x)
Finding such a function f is anti-differentiation, that is, undoes differentiation. So, sure,

     f(x) = 3x^2 + C
for any constant C.

Such anti-differentiation is also the way to find the area under a curve. So, can use that to find the area of a circle, volume of a cylinder, etc. Doing that the anti-differentiation is integration.

The fundamental theorem of calculus shows how differentiation and integration are related.

(1.4) Analytic Geometry

Commonly taught at the beginning of a calculus course is analytic geometry.

So, take a cone an cut it. Then the cut surfaces will be one of a circle, an ellipse, a parabola, a hyperbola, or just two crossed straight lines. So, those curves are from a cone and are the conic sections.

There is some simple associated algebra.

Conic sections are important off and on; e.g., applied math is awash in circles; the planets move in ellipses; a baseball moves in a parabola or nearly so; an electron moving toward a negative charge will turn away from that charge in a hyperbola.

It turns out that in linear algebra (below) circles and ellipses are important.

(1.5) Role of Calculus

Calculus was invented by Newton as part of working with force and acceleration for understanding the motion of the planets.

E.g., if at time t function d(t) gives distance traveled, then function v(t) = d'(t) is the velocity at time t and function a(t) = v'(t) is the acceleration at time t.

Then Newton's second law is

     F(t) = m a(t)
where F(t) is the force at time t applied to mass m.

Calculus is the first approach to the analysis of continuous change and is a pillar of civilization.

Knowledge of calculus will commonly be assumed in work in ML/AL, data science, statistics, optimization, applied math, engineering, etc.

E.g., a lot in ML, AI, and data science is getting best fits to data; best fitting is to minimize errors in the fit; such minimization is mostly a calculus problem; one of the main steps in ML is steepest descent, and that is from a derivative.

Probability theory (e.g., evaluating coin tossing, poker hands, accuracy in ML) will be important in ML/AI, etc.; two of the basic notions in probability are cumulative distributions and density distributions; the cumulative is from an integration, and the density is from a differentiation.

Post reply on HN