Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Math for Machine Learning

blog.ycombinator.com

Post reply on HN
Print view View original

61–70 of 119 posts

Re: Learning Math for Machine Learning

#61

Post by Bizarro »

cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

I don't think we "don't like math", but in my case, I just need an accelerated version of the "math" that I need without the deep-dive.

Here's a crazy idea that machine learning might one day help with software engineers understanding algorithms and data structures.

You write some code to traverse a list or something and do some naive sorting, or maybe you're "everyday way of doing some operations on your lists is inefficent". I want some cool machine learning where I can submit my code and it does analysis.

I think Microsoft is working on that. https://techcrunch.com/2018/05/07/microsofts-new-intellicode...

Let's take it a step further. Explain to the programmer why what their doing is wrong.

I would pay big bucks for a "machine intelligence" IDE

Re: Learning Math for Machine Learning

#62

Post by _spoonman »

tenaciousDaniel wrote: post #46
ivansavz wrote: post #15

Here is a nice "cheat sheet" that introduces many math concepts needed for ML: https://ml-cheatsheet.readthedocs.io/en/latest/ > As soft prerequisites, we assume basic comfortability with linear algebra/matrix calc [...] > That's a bit of an understatement. I think anyone interested in learning ML should invest the time needed to deeply understand Linear Algebra: vectors, linear transformations, representations, vect…

Awesome. I had a really difficult time with math in HS, and never pursued it at all in college, so even though I'm a programmer my math skills are barely at a high school level. I'd love to get into ML but the math keeps me at bay.

We should find or start a Slack / Discord where we go through a math textbook and conquer our fear of mathematics together.

Re: Learning Math for Machine Learning

#63

Post by woolvalley »

cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

Math education from the undergraduate level on is fairly horrible and not communicated well. Just go read the typical calculus textbook and realize that they reference a lot of stuff that no pre-calculus student would typically know, such as proof by induction, lemmas and so on. The textbooks are written to the professors, not the actual students.

Various non-intuitive concepts are handwaved, the foundations skipped over and students then start struggling because they don't understand the foundation of what they are trying to learn. Reading from the textbook is fairly useless and it ends up being used as a problem set source.

I argued to a few math professors about teaching things like calculus with the textbooks referencing concepts that were not actually taught until 5 classes later is a bad idea.

In return I got a shrug of indifference telling me that's just the status quo and the status quo is OK.

Thank god khan academy exists now.

Re: Learning Math for Machine Learning

#64

Post by SpaceManNabs »

blt wrote: post #35
SpaceManNabs wrote: post #22

My bullet list, which might be too ambitious and theory-focused, but this is what I used from my physics background. Learn some: Calc up to 3 (you can skip some of the divergence and curl stuff) Linear algebra (no need for Jordan change of basis) Real analysis Intermediate probability theory (MAE, MAP, conjugate priors minus the measure theory stuff) A little bit of differential geometry (at least geodesics. This is…

I'm interested to know where you encountered contour integrals in machine learning?

I have only seen it used to evaluate some integrals in a few papers via Residue Theorem, so maybe I should have said Residue theorem instead of contour integration. I am sure there were other methods that the authors could have used, but I was sure glad to know of contour integration then. I'd say some complex analysis still deserves to be on the list to have a base understanding of Fourier transforms. Of course, you can arrive there without complex analysis.

Re: Learning Math for Machine Learning

#65

Post by SpaceManNabs »

mlevental wrote: post #43
blt wrote: post #35

Earlier quoted context omitted.

I'm interested to know where you encountered contour integrals in machine learning?

ya lol and Hamiltonians. sometimes people just reel off all the math they've heard of to sound impressive. next we'll have people talking about de rham cohomology because of TDA (or something like that)

https://arxiv.org/abs/1111.4246

I believe the above was used in sk-learn or PyMC3 at some point.

Like I said, the list was a bit too theory focused and not just for ML. Hope that clears things up.

Also, maybe a bit out of place, but it makes my day happier when I assume good intentions out of random tidbits posted online.

Re: Learning Math for Machine Learning

#66

Post by [deleted] »

cfcf14 wrote: post #54
blt wrote: post #52

Earlier quoted context omitted.

I agree. My original question came from curiosity, not incredulity :)

:) Personally I've never used any serious complex analysis in my job (I'm very grateful too, because I always struggled a bit with it). The closest thing I've seen, which I did run into recently, is the use of complex numbers to compute very accurate finite differences. It's one of those delightful tricks that is both elegant and useful: https://blogs.mathworks.com/cleve/2013/10/14/complex-step-di... I've been workin…

[deleted]

Re: Learning Math for Machine Learning

#67

Post by blt »

cfcf14 wrote: post #54
blt wrote: post #52

Earlier quoted context omitted.

I agree. My original question came from curiosity, not incredulity :)

:) Personally I've never used any serious complex analysis in my job (I'm very grateful too, because I always struggled a bit with it). The closest thing I've seen, which I did run into recently, is the use of complex numbers to compute very accurate finite differences. It's one of those delightful tricks that is both elegant and useful: https://blogs.mathworks.com/cleve/2013/10/14/complex-step-di... I've been workin…

oh cool, I really like Cleve Moler's Matlab posts.

I took a complex analysis class and did OK, but I get the feeling that EEs are the ones who really benefit from it (at least in the applied world). They seem to have some very rich analyses of linear dynamical systems using frequency domain methods.

Re: Learning Math for Machine Learning

#68

Post by jessaustin »

woolvalley wrote: post #63
cs702 wrote: post #5

This is excellent. Thank you for taking the time to write it. I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people. The fact that so many software developers "don't like math" is ironic, because they're perfectly happy to manipulate symbols such as "x", "file", or "user_id" that stand in for other things every…

Math education from the undergraduate level on is fairly horrible and not communicated well. Just go read the typical calculus textbook and realize that they reference a lot of stuff that no pre-calculus student would typically know, such as proof by induction, lemmas and so on. The textbooks are written to the professors, not the actual students. Various non-intuitive concepts are handwaved, the foundations skipped…

I attended a not-superb high school in rural Missouri, and we studied proof by induction in 11th grade, before calculus in 12th. (Only over the naturals, but that's enough to get the flavor...) Lemmas came in 10-grade geometry, although frankly I may not understand what you mean because that's not really a full "concept", just kind of an arbitrary detail. That was the early 90s, though, so perhaps standards have slipped.

Of course, colleges should cater to a range of preparatory educations, but the textbooks you're talking about are pitched at the correct level for somebody.

Re: Learning Math for Machine Learning

#69

Post by tntn »

SpaceManNabs wrote: post #22

My bullet list, which might be too ambitious and theory-focused, but this is what I used from my physics background. Learn some: Calc up to 3 (you can skip some of the divergence and curl stuff) Linear algebra (no need for Jordan change of basis) Real analysis Intermediate probability theory (MAE, MAP, conjugate priors minus the measure theory stuff) A little bit of differential geometry (at least geodesics. This is…

Sounds like EE.

Re: Learning Math for Machine Learning

#70

Post by smallnamespace »

throwawaymath wrote: post #48
adw wrote: post #37

Earlier quoted context omitted.

> To play devil's advocate, probabilistic reasoning (probability theory, stochastic processes, Bayesian reasoning, graphical models, variational inference) might be equally if not more important. For intuition, particularly if you care about vision applications, I think one field of math which is severely underrated by the community is group theory. Trying to understand methods which largely proceed by divining struc…

Can you talk about the use of group theory for computer vision or crystallography a bit? I'm familiar with the math but I'm not familiar with group theory's applications in those areas. That sounds pretty interesting. Is it primarily group theory, or does it so venture into abstract algebra more generally?

For crystallography, the use of group theory in part originates in X-ray crystallography [1], where the goal is to take 2D projections of a repeating 3D structure (crystal), and use that along with other rules that you know to re-infer what the 3D structure is.

Repeating structures have symmetries, so seeing the symmetries in your diffraction pattern inform you of the possible symmetries (and hence possible arrangements) in your crystal. Group theory is the study of symmetry.

By the way, this is also how the structure of DNA was inferred [2], although not from a crystal.

[1] https://en.wikipedia.org/wiki/X-ray_crystallography#Crystal_...

[2] https://www.dnalc.org/view/15014-Franklin-s-X-ray-diffractio...

Post reply on HN