Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#121

Post by nerdponx »

samfriedman wrote: post #64
samstave wrote: post #59

Earlier quoted context omitted.

Can you please expand on this comment for us ML/AI naive?

There exists a tool called TPOT (Tree-based Pipeline Optimization Tool) [0] that aims to automate the knob-twiddling that tends to go with optimizing Machine Learning models. As these models often have a number of parameters to tune and tweak over large scales, such a tool can be useful to identify performant combinations of these parameters and save time in doing so. However, many ML practitioners are wary of simila…

Thanks for clarifying.

These tools do, however, have an important place in saving practitioners time and energy on the "knob-twiddling". It's a little like robot-assisted surgery: the robot doesn't actually do the surgery, but it makes the surgeon's job a whole lot easier.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#122

Post by Cacti »

popcorncolonel wrote: post #7
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

I disagree that you need a solid founding in information theory. Almost all that I've seen about IT in ML is minimizing the KL divergence, which can be learned by browsing the wiki page.

Information theory and probability are basically the same thing.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#123

Post by tel »

tptacek wrote: post #20
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

I'm not super interested in ML but I am very interested in applied mathematics in computer science. I've got a fair bit of linear algebra due to cryptography, but have had virtually no need of any form of calculus (unless I'm relying on it without knowing it) in my career. So beyond just saying that you'd need grounding in multivariable calculus to do serious ML work, I would be super interested in hearing more about…

Calculus essentially discusses how things change smoothly and it has a very nice mechanism for talking about smooth changes algebraically.

A system which is at an optimum will, at that exact point, be no longer increasing or decreasing: a metal sheet balanced at the peak of a hill rests flat.

Many problems in ML are optimization problems: given some set of constraints, what choices of unknown parameters minimizes error? This can be very hard (NP-hard) in general, but if you design your situation to be "smooth" then you can use calculus and its very nice set of algebraic solutions.

You also need multivariate calculus because typically while you're only trying to minimize "error", you do so by changing many, many parameters at once. This means that you've got to talk about smooth changes in a high-dimensional space.

--

The other side of calculus is integration which talks about "measuring" how big things are. Most of probability is discussing very generalized ratios: of the total, "how big is this piece" is analogous to "what are the odds this will happen".

The general discussion of measure is complex and essentially the only tool to tackle it involves gigantic (infinite, really) sums of small, well-behaved pieces to form a complex whole.

It just happens to turn out (and this is the big secret of calculus) that this machinery (integration) is dual to the study of smooth changes and you can knock them both out together.

--

So ultimately, ML hinges upon being able to measure things (integration) and talk about how they change (derivation). Those two happen to be the same concept in a way and they are essentially what you study in calculus.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#124

Post by jonahx »

mindcrime wrote: post #9

It depends on how deep you want to go and what your goals are, but I'd say that CuriouslyC pretty much nailed it. Multi-variable calculus, linear algebra, and probability / stats are definitely the core. If you're interested in finding more "freely available online" maths references, check out: http://people.math.gatech.edu/~cain/textbooks/onlinebooks.ht... http://www.openculture.com/free-math-textbooks https://open.…

Another upvote for 3blue1brown. I just watched his linear algebra series and it's probably the most outstanding math instruction I've encountered.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#126

Post by Jasamba »

nerdponx wrote: post #121
samfriedman wrote: post #64

Earlier quoted context omitted.

There exists a tool called TPOT (Tree-based Pipeline Optimization Tool) [0] that aims to automate the knob-twiddling that tends to go with optimizing Machine Learning models. As these models often have a number of parameters to tune and tweak over large scales, such a tool can be useful to identify performant combinations of these parameters and save time in doing so. However, many ML practitioners are wary of simila…

Thanks for clarifying. These tools do, however, have an important place in saving practitioners time and energy on the "knob-twiddling". It's a little like robot-assisted surgery: the robot doesn't actually do the surgery, but it makes the surgeon's job a whole lot easier.

That is making the assumption that the person using the tool is a surgeon (an expert in the field who could function independently if needed) which is not who the targeted demographic of such tools is. No-one who understands ML to some non-zero extent would use a plug-and-play ML tool, given that there is ML left to do otherwise. A better analogy would be a janitor activating the red button of the robot machine, which then does its complex surgery where if something goes wrong, the janitor would not be able to replace/understand the problem other than trying to restart it/kick it.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#127

Post by KirinDave »

cynicaldevil wrote: post #111
KirinDave wrote: post #53

If you care about actually reading the nournals, as I do, and you had a very poor math education (as mine was abysmally opposed to both math and science as enemies of religion) then here are things I've determined I need to know to read journals: - Core statistics. You need to be familiar with how statisticians treat data, because it comes up a lot. - Calculus. You do not need to be a wizard at working the numbers bu…

How did you learn these topics? Did you solve problems for each of them?

Reading, study, and textbooks.

Tbh, I'm not where I want to be with them. So maybe next year I can talk about 2017 and my math oddessy.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#128

Post by coconut_crab »

Maybe unrelated to OP's question but I have always felt that it is impossible to get a job in AI/ML without a PhD in that field (by getting a job I mean do something new/useful and not just coding algorithm devised by other people). I studied mechatronics in university and fairly comfortable with math (calculus, linear algebra and stats), I have even written a small neural network back then to optimise parameters for lathe machining. But that's no where near enough for a job in AI/ML. Unlike writing a web page, which someone can learn within a week to produce something usable, I feel like you need years and years of studying to barely get a start in ML/AI and there is no hope for us non computer scientist at all.

[Added] Of course writing webpages pays well enough, but I still can't shake off this feeling that I am missing something by not jumping on the AI/ML train though.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#129

Post by sunsu »

None. You can be a productive ML engineer without understanding the math. Many elitist engineers here will downvote me, but its true. ML libraries that allow you to quickly get productive have come a long way. BUT, you have to have a solid understanding of WHICH algorithms/tools to use WHEN. There is also a lot of "voodoo" knowledge to gain that isn't well documented or explained (unrelated to maths).
Post reply on HN