Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Math for Machine Learning

blog.ycombinator.com

Post reply on HN
Print view View original

71–80 of 119 posts

Re: Learning Math for Machine Learning

#71

Post by throwawaymath »

Xeronate wrote: post #51
throwawaymath wrote: post #39

Earlier quoted context omitted.

> To play devil's advocate, probabilistic reasoning (probability theory, stochastic processes, Bayesian reasoning, graphical models, variational inference) might be equally if not more important. And of course, you're not going to get very far with probability theory and stochastic processes unless you have a mature understanding of analysis and measure theory :) This comment exchange neatly demonstrates the intrinsi…

I get what you are saying, but is the right way to learn math with a "connected path". I've heard "The art of problem solving" series works through math in the correct way, but I'm not sure how far I would get on that alone. Right now I'm trying to gain intuition in linear algebra via OCW with Strang, but I would like to truly understand. Is the only way to just to do a second bachelors in math?

You don't need to do a second bachelors - you really need four or so courses. If you have the patience and dedication you can sit down with the textbooks and work through them on your own.

Re: Learning Math for Machine Learning

#72

Post by tnecniv »

g9yuayon wrote: post #45

I'm kinda curious why so many people think that Linear Algebra Done Right is an introductory book for beginners who have math anxiety. Don't get me wrong, the book is great and I enjoyed working it through. It was a magical experience when I saw how simple it was to prove some seemingly hard theorems by just linking the right definitions and theorems. That said, the book does require certain level of math maturity as…

For me, it really was the first time math had clicked. I had a non-proof based linear algebra course before going through the book, but it made very little sense to me. After doing LADR, I understood the subject intimately, lost my math anxiety, and performed better in every class I took afterwards than I would have otherwise.

Re: Learning Math for Machine Learning

#73

Post by throwawaymath »

JesseAldridge wrote: post #44

I think a lot of people need to start from the basics because they don't have a good foundation in math. The core problem is schools will push you along if you can somehow produce the correct answer for 70% of the problems on a test. Combine this with intense pressure not to fail and you will very likely end up in higher level math courses with many gaping holes in your foundational knowledge. You thus end up relying…

> Like, I didn't understand how division works -- if someone were to ask me what (3/4) / (5/6) even means conceptually I would not have been able to provide a coherent, accurate explanation. "Uh... it's like taking 5/6 of 3/4... wait no that's multiplication... you need to flip the second fraction over... for some reason..."

In case you (or others reading this) still struggle to formalize division, a very nice way to conceptualize it is as the inverse of multiplication. This neatly sidesteps the problem of trying to figure out a clean analogue for what it means to to multiply a fraction of something by another fraction of something, since the intuitive group-adding idea of multiplication sort of breaks down with ratios.

Addition is a straightforward operation, but subtraction is trickier. For all real x there exists an additive inverse -x satisfying x + (-x) = 0. So to subtract 3 from 4 we instead take the sum 4 + (-3) = 1.

Likewise to multiply 3 by 4 we add four groups of 3: 3 + 3 + 3 + 3 = 12. We accomplish division by using a multiplicative inverse: for all real x there exists a 1/x such that x(1/x) = 1.

So (3/4) / (5/6) is equal to (3 * 1/4) / (5 * 1/6). In other words, take the multiplicative inverse of 4 and 6 and multiply them by 3 and 5 respectively. Then multiply the first product by the inverse of the second product.

This is the axiomatic basis of division as "repeated subtraction": subtraction is the sum of a number and another number's additive inverse, and multiplication is repeated addition. Then division is the product of a number and another number's multiplicative inverse. From this perspective you need not even understand division computationally if all you'll ever deal with are fractions and not decimals.

Re: Learning Math for Machine Learning

#74

Post by imh »

SpaceManNabs wrote: post #22

My bullet list, which might be too ambitious and theory-focused, but this is what I used from my physics background. Learn some: Calc up to 3 (you can skip some of the divergence and curl stuff) Linear algebra (no need for Jordan change of basis) Real analysis Intermediate probability theory (MAE, MAP, conjugate priors minus the measure theory stuff) A little bit of differential geometry (at least geodesics. This is…

Can you give any recommendations for a little bit of differential geometry?

Re: Learning Math for Machine Learning

#75

Post by asafira »

throwawaymath wrote: post #71
Xeronate wrote: post #51

Earlier quoted context omitted.

I get what you are saying, but is the right way to learn math with a "connected path". I've heard "The art of problem solving" series works through math in the correct way, but I'm not sure how far I would get on that alone. Right now I'm trying to gain intuition in linear algebra via OCW with Strang, but I would like to truly understand. Is the only way to just to do a second bachelors in math?

You don't need to do a second bachelors - you really need four or so courses. If you have the patience and dedication you can sit down with the textbooks and work through them on your own.

This.

There's always more you might want to learn, but when people talk about these basics, it's really just being super focused in 4 or so classes, not a whole ivy league undergrad curriculum in math.

probability & stats, multivariable calculus, and linear algebra will take you a long way.

Re: Learning Math for Machine Learning

#76

Post by woolvalley »

jessaustin wrote: post #68
woolvalley wrote: post #63

Earlier quoted context omitted.

Math education from the undergraduate level on is fairly horrible and not communicated well. Just go read the typical calculus textbook and realize that they reference a lot of stuff that no pre-calculus student would typically know, such as proof by induction, lemmas and so on. The textbooks are written to the professors, not the actual students. Various non-intuitive concepts are handwaved, the foundations skipped…

I attended a not-superb high school in rural Missouri, and we studied proof by induction in 11th grade, before calculus in 12th. (Only over the naturals, but that's enough to get the flavor...) Lemmas came in 10-grade geometry, although frankly I may not understand what you mean because that's not really a full "concept", just kind of an arbitrary detail. That was the early 90s, though, so perhaps standards have slip…

My recollection is vague, since it was quite a while ago, but there were other things that weren't introduced in my high school curriculum that I remember my calculus textbook containing. I'm glad your school taught you proof of induction and other such things although. I also remember talking to my classmates about how near indecipherable our calculus textbook was.

Re: Learning Math for Machine Learning

#77

Post by bitcoinmoney »

throwawaymath wrote: post #71
Xeronate wrote: post #51

Earlier quoted context omitted.

I get what you are saying, but is the right way to learn math with a "connected path". I've heard "The art of problem solving" series works through math in the correct way, but I'm not sure how far I would get on that alone. Right now I'm trying to gain intuition in linear algebra via OCW with Strang, but I would like to truly understand. Is the only way to just to do a second bachelors in math?

You don't need to do a second bachelors - you really need four or so courses. If you have the patience and dedication you can sit down with the textbooks and work through them on your own.

Can you recommend books and online courses to hammer these concepts down? I used PCA and k-means for my masters thesis but didn’t really know how well they work under the covers.

Re: Learning Math for Machine Learning

#78

Post by Xeronate »

asafira wrote: post #75
throwawaymath wrote: post #71

Earlier quoted context omitted.

You don't need to do a second bachelors - you really need four or so courses. If you have the patience and dedication you can sit down with the textbooks and work through them on your own.

This. There's always more you might want to learn, but when people talk about these basics, it's really just being super focused in 4 or so classes, not a whole ivy league undergrad curriculum in math. probability & stats, multivariable calculus, and linear algebra will take you a long way.

Cool. I will look into those, but I was asking as a general interest in math question. I actually have no interest in machine learning. I'm bored of chasing money. Interested in 3D computer graphics and math for math's sake.

Re: Learning Math for Machine Learning

#79

Post by ivansavz »

Bizarro wrote: post #60

I really want a shallow-dive into machine learning and I know I need linear-algebra as a foundation. I would love an interactive course in linear algebra where we could input matrices and see some visual stuff with animations.

Check out 3blue1brown on youtube https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x... Everything this guy does is gold!

This is also really good for connecting LA concepts with visuals http://immersivemath.com/ila/index.html

Re: Learning Math for Machine Learning

#80

Post by shostack »

JesseAldridge wrote: post #44

I think a lot of people need to start from the basics because they don't have a good foundation in math. The core problem is schools will push you along if you can somehow produce the correct answer for 70% of the problems on a test. Combine this with intense pressure not to fail and you will very likely end up in higher level math courses with many gaping holes in your foundational knowledge. You thus end up relying…

I had a similar Khan Academy experience. What caught my attention was how much more relevant everything was because I had a wealth of work and life experiences that made the concepts much more relevant and applicable than they might have been when I was in HS.
Post reply on HN