Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Math for Machine Learning

blog.ycombinator.com

Post reply on HN
Print view View original

21–30 of 119 posts

Re: Learning Math for Machine Learning

#21

Post by ivansavz »

mindcrime wrote: post #16
cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. I can tell you at least part of it, from my subjective perspective. I tend to "think" in a very verbal fashion and I instinctively try to sub-vocalize everything I read. So when I see math, as soon as I see a symbol that I can't "say" to myself (eg, a greek…

Very good point about reading/verbalizing symbols and notation. Once you know what they mean, they are super useful for expressing complex concepts precisely and compactly, but when you're getting started they look like an alien language...

Re: Learning Math for Machine Learning

#22

Post by SpaceManNabs »

My bullet list, which might be too ambitious and theory-focused, but this is what I used from my physics background.

Learn some:

Calc up to 3 (you can skip some of the divergence and curl stuff)

Linear algebra (no need for Jordan change of basis)

Real analysis

Intermediate probability theory (MAE, MAP, conjugate priors minus the measure theory stuff)

A little bit of differential geometry (at least geodesics. This is for dimension reduction)

Discrete math (know counting and sums really well)

Learn a little bit of Physics (at least know Lagrangians and Hamiltonians)

A little bit of complex analysis (to know contour integration and fourier/laplace transforms)

Some differential equations (up to Frobenius and wave equations)

Some graph theory (my weak spot, but I have used the matrix representations a few times)

After all that, read some Kevin Murphy and Peter Norvig.

Congrats, now you can read most machine learning papers. The above will also give you the toolkit to learn things as they come up like Robbins-Monro.

OP's article is much better if you are trying to be a ML developer/practitioner. Like I said, this list might be too theory focused, but it lets me read lots of applied math papers that aren't ML focused.

Re: Learning Math for Machine Learning

#23

Post by mindcrime »

ivansavz wrote: post #21
mindcrime wrote: post #16

Earlier quoted context omitted.

I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. I can tell you at least part of it, from my subjective perspective. I tend to "think" in a very verbal fashion and I instinctively try to sub-vocalize everything I read. So when I see math, as soon as I see a symbol that I can't "say" to myself (eg, a greek…

Very good point about reading/verbalizing symbols and notation. Once you know what they mean, they are super useful for expressing complex concepts precisely and compactly, but when you're getting started they look like an alien language...

This is why I'm using Anki to memorize the Greek alphabet, and to keep basic algebraic (h.s. algebra that is) stuff in mind. It might seem like a small thing, but remembering the various rules for factoring, working with fractions, dealing with exponents / root, etc. is not easy when you don't do math all the time.

Re: Learning Math for Machine Learning

#24

Post by xenihn »

ivansavz wrote: post #18
xenihn wrote: post #14

I recommend the No Bullshit books for anyone with no real math background past trig to get their feet wet, and/or anyone who hasn't done any serious math study for years. https://minireference.com/

Thx! Had I known you'll post this, I wouldn't have self-promoted so shamelessly :) I'll add some direct links to PDF previews: MATH & PHYS book: https://minireference.com/static/excerpts/noBSguide_v5_previ... LA book: https://minireference.com/static/excerpts/noBSguide2LA_previ... + free tutorial: https://minireference.com/static/tutorials/linear_algebra_in...

Hah I beat you by 2 minutes. Thanks for the great books!

Re: Learning Math for Machine Learning

#25

Post by ssivark »

cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

The impression I get is that many people want to be system designers stringing together pieces to create systems to solve problems (part of the motivation might be that it is easier to extract economic value from such integrated solutions, rather than better functioning pieces).

The problem is that in an immature field that's still evolving, the components are not yet well-understood or well-designed, so available abstractions are all leaky. However, modern software engineering is mostly built on the ability to abstract away enormous complexity behind libraries, so that a developer who is plumbing/composing them together can ignore a lot of details [1]. People with that background now expect similarly effective abstractions for machine learning, but the truth is that machine learning is simply NOT at that level of maturity, and might take decades to get there. It is the price you pay for the thrill of working in a nascent field doing something genuinely uncharted.

"Math in machine learning" is a bit of a red herring. We hear the same complaints about putting in effort to grok ideas in functional programming, thinking about hardware/physics details, understanding the effects of software on human systems [2], etc. Fundamentally, I think a lot of people have not developed the skill to fluidly move between different levels of abstraction, and a variety of approximately correct models. And to be fair, it seems like most of software engineering is basically blind to this, so one can't shift all the blame on individuals.

[1] Why the MIT CS curriculum moved away from Scheme towards Python -- https://www.wisdomandwonder.com/link/2110/why-mit-switched-f...

[2] Building software through REPL-it-till-it-works leads to implicitly ignoring important factors (such as ethics) -- https://news.ycombinator.com/item?id=16431008

Re: Learning Math for Machine Learning

#26

Post by jdminhbg »

cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

> The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every day.

`user_id` says what it is; something like `β` does not. It's more like reading minified JavaScript than literate programming. Math notation is frequently horribly overloaded and needlessly terse.

Re: Learning Math for Machine Learning

#28

Post by ksdale »

hmmm5 wrote: post #9
cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

> I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The biggest turn off about math is the way people are taught math. Most people are taught math as if it's an infinite set of cold formulas to memorize and regurgitate. Most students in my statistics class didn't know where and when to use the formulas tau…

This is exactly my experience! I breezed through math all the way until I got to calculus because I was excellent at rote memorization. Calculus made me realize that I didn't really understand most of what I had learned for the past several years.

Years later, I'm trying to relearn math, but I'm taking the exact opposite approach. No calculator, no rote memorization, just reading about the concepts and thinking about what they mean until I can do the manipulations in my head. When I do practice problems, I don't care so much about the specific numbers, but about my ability to understand what's happening to each part of an equation, what the graph looks like, etc.

Re: Learning Math for Machine Learning

#29

Post by throwawaymath »

lordfoom wrote: post #27

Anyone have a suggestion for a good online course in linear algebra?

Yes, UIUC offers very good online math courses: https://netmath.illinois.edu/college/math-415. There is also a more pure/abstract version of that course available.

If you don’t care about accreditation and are patient, sit down with Axler’s Linear Algebra Done Right and Hoffman & Kunze’s Linear Algebra, in that order.

I would caution you against trying to learn linear algebra using a “take what you need” approach. A random walk approach to learning the material is faster than an accumulation approach, but it’s more brittle and prone to confusion. A lot of things which appear to be irrelevant or unnecessary for machine learning (computation or research) can be imperative for understanding or implementing much more complex concepts later on.

Re: Learning Math for Machine Learning

#30

Post by randcraw »

lordfoom wrote: post #27

Anyone have a suggestion for a good online course in linear algebra?

I like "Coding the Matrix" by Philip Klein of Brown delivered via Coursera. It's a deep content intro to linear algebra (and more), with a focus on applications in computer science. The course is accompanied by a textbook written by Klein, which makes the course material better organized and more in-depth than slides and videos alone would allow.

http://codingthematrix.com/

Post reply on HN