Learning Math for Machine Learning • Hacker News Board

Re: Learning Math for Machine Learning

#51

Post by Xeronate » Wed, Aug 01, 2018, 10:39 PM UTC

throwawaymath wrote: post #39

ssivark wrote: post #31

Earlier quoted context omitted.

> I think anyone interested in learning ML should invest the time needed to deeply understand Linear Algebra: vectors, linear transformations, representations, vector spaces, matrix methods, etc. Linear algebra knowledge and intuition is key to all things ML, probably even more important than calculus. To play devil's advocate, (EDIT: an intuitive understanding of) probabilistic reasoning (probability theory, stochas…

> To play devil's advocate, probabilistic reasoning (probability theory, stochastic processes, Bayesian reasoning, graphical models, variational inference) might be equally if not more important. And of course, you're not going to get very far with probability theory and stochastic processes unless you have a mature understanding of analysis and measure theory :) This comment exchange neatly demonstrates the intrinsi…

I get what you are saying, but is the right way to learn math with a "connected path". I've heard "The art of problem solving" series works through math in the correct way, but I'm not sure how far I would get on that alone. Right now I'm trying to gain intuition in linear algebra via OCW with Strang, but I would like to truly understand. Is the only way to just to do a second bachelors in math?

Re: Learning Math for Machine Learning

#52

Post by blt » Wed, Aug 01, 2018, 10:40 PM UTC

cfcf14 wrote: post #49

mlevental wrote: post #43

Earlier quoted context omitted.

ya lol and Hamiltonians. sometimes people just reel off all the math they've heard of to sound impressive. next we'll have people talking about de rham cohomology because of TDA (or something like that)

Hamiltonian mechanics, along with many other seemingly out of place 'advanced' maths, show up in modern Bayesian statistics pretty frequently. Hamiltonian Monte Carlo/Riemannian Manifold Monte Carlo are pretty cutting edge (although are implemented in popular libraries like MC-Stan and Pymc3) and both require fairly advanced physics to really understand. Additionally, we're seeing the introduction of even more sophis…

I agree. My original question came from curiosity, not incredulity :)

Re: Learning Math for Machine Learning

#53

Post by thanatropism » Wed, Aug 01, 2018, 10:59 PM UTC

JesseAldridge wrote: post #44

I think a lot of people need to start from the basics because they don't have a good foundation in math. The core problem is schools will push you along if you can somehow produce the correct answer for 70% of the problems on a test. Combine this with intense pressure not to fail and you will very likely end up in higher level math courses with many gaping holes in your foundational knowledge. You thus end up relying…

I've heard of similar stories with adult iliteracy or almost-iliteracy.

I applaud your counter-Dunning-Krugerish inquisitiveness about your own skills. I hope some of that rubs on me.

Re: Learning Math for Machine Learning

#54

Post by cfcf14 » Wed, Aug 01, 2018, 11:01 PM UTC

blt wrote: post #52

cfcf14 wrote: post #49

Earlier quoted context omitted.

Hamiltonian mechanics, along with many other seemingly out of place 'advanced' maths, show up in modern Bayesian statistics pretty frequently. Hamiltonian Monte Carlo/Riemannian Manifold Monte Carlo are pretty cutting edge (although are implemented in popular libraries like MC-Stan and Pymc3) and both require fairly advanced physics to really understand. Additionally, we're seeing the introduction of even more sophis…

I agree. My original question came from curiosity, not incredulity :)

:) Personally I've never used any serious complex analysis in my job (I'm very grateful too, because I always struggled a bit with it). The closest thing I've seen, which I did run into recently, is the use of complex numbers to compute very accurate finite differences. It's one of those delightful tricks that is both elegant and useful: https://blogs.mathworks.com/cleve/2013/10/14/complex-step-di...

I've been working in golang, which fortunately has built-in complex128 types, so it's proved very helpful in a project!

Re: Learning Math for Machine Learning

#55

Post by thanatropism » Wed, Aug 01, 2018, 11:03 PM UTC

g9yuayon wrote: post #45

I'm kinda curious why so many people think that Linear Algebra Done Right is an introductory book for beginners who have math anxiety. Don't get me wrong, the book is great and I enjoyed working it through. It was a magical experience when I saw how simple it was to prove some seemingly hard theorems by just linking the right definitions and theorems. That said, the book does require certain level of math maturity as…

As a teenager I thought I was bad at math and even went to get a film studies/communications bachelors (I've defended this week my masters thesis in computational mathematical physical; this after an undegraduare degree in econ with lots of math of course).

The thing is, I couldn't write the damn matrices well lined up and made mistakes when doing calculations. This was really a (de)formative experience. In college Linear Algebra for econ was 40% gaussian elimination, 40% eigenvalues and 20% linear programming. I mean, I still can't do gausian elimination by hand right.

I started crawling out of it when I started seeing (in self-study) a book on linear algebra that takes the linear transform/vector space-first approach.

Re: Learning Math for Machine Learning

#56

Post by bhl » Wed, Aug 01, 2018, 11:17 PM UTC

ssivark wrote: post #31

ivansavz wrote: post #15

Here is a nice "cheat sheet" that introduces many math concepts needed for ML: https://ml-cheatsheet.readthedocs.io/en/latest/ > As soft prerequisites, we assume basic comfortability with linear algebra/matrix calc [...] > That's a bit of an understatement. I think anyone interested in learning ML should invest the time needed to deeply understand Linear Algebra: vectors, linear transformations, representations, vect…

> I think anyone interested in learning ML should invest the time needed to deeply understand Linear Algebra: vectors, linear transformations, representations, vector spaces, matrix methods, etc. Linear algebra knowledge and intuition is key to all things ML, probably even more important than calculus. To play devil's advocate, (EDIT: an intuitive understanding of) probabilistic reasoning (probability theory, stochas…

For ML, you need both—probability to justify the setup of the problem, and linear algebra and calculus to optimize for a solution.

A simple example is with linear regression: find w such that the squared l2 norm of (Xw - y) is minimized.

Linear algebra will help with generalizing to n data points; and calculus will help with taking the gradient and setting equal to 0.

Probability will help with understanding why the squared l2 norm is an appropriate cost function; we assumed y = Xw + z, where z is Gaussian, and tried to maximize the likelihood of seeing y given x.

I’m sure there’s more examples of this duality since linear regression is one of the more basic topics in ML.

Re: Learning Math for Machine Learning

#57

Post by cs702 » Wed, Aug 01, 2018, 11:19 PM UTC

throwawaymath wrote: post #36

cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

> The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every day. The entirety of mathematical knowledge is very much like a gigantic computer language (a formal system) in which every object is and must be precisely defined in terms of other objects, using and reusing symbols like…

True. I cannot disagree with any of this.

That said, the math we're talking about (that is, the math necessary for understanding, say, the sequence of transformations that make up a convnet) lies far below the ceiling of complexity you mention.

I'm not sure symbol reuse and overloading are as much of an issue. I've run into people who are quite proficient with Perl and routinely use complicated regular expressions who say they didn't like math growing up.

Re: Learning Math for Machine Learning

#58

Post by Bizarro » Wed, Aug 01, 2018, 11:23 PM UTC

The LAFF: Linear Algebra class just started again for the "fall semester" https://courses.edx.org/courses/UTAustinX/UT.5.02x/1T2015/co...

Maybe one of these days I'll complete it :)

I really like 3Blue1Brown for a wide range of math topics. He's just a great teacher.

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

Frankly, I find the UTAustin linear algebra class less than ideal or optimal, but it's free and lots of classmates, material, so...

Re: Learning Math for Machine Learning

#59

Post by cs702 » Wed, Aug 01, 2018, 11:24 PM UTC

ssivark wrote: post #25

cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

The impression I get is that many people want to be system designers stringing together pieces to create systems to solve problems (part of the motivation might be that it is easier to extract economic value from such integrated solutions, rather than better functioning pieces). The problem is that in an immature field that's still evolving, the components are not yet well-understood or well-designed, so available ab…

Yes, I see your point.

Deep learning, in particular, is a trade today. If we want to be generous we can call it an "experimental science"... but my perception is that only a minority of papers in the field actually deserve that moniker.

(Speaking as a deep learning practitioner with expertise in a narrow domain.)

Re: Learning Math for Machine Learning

#60

Post by Bizarro » Wed, Aug 01, 2018, 11:27 PM UTC

I really want a shallow-dive into machine learning and I know I need linear-algebra as a foundation. I would love an interactive course in linear algebra where we could input matrices and see some visual stuff with animations.