Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What maths are critical to pursuing ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 139 posts

Re: Ask HN: What maths are critical to pursuing ML/AI?

#91

Post by mljoe »

anst wrote: post #49
leecarraher wrote: post #42

It will depend on the level you plan to engage in the ML/AI space. If you just want a job in ML/AI , you are in luck. Due to the growing assortment of available, mostly to fully automated, solutions like Datarobot, H2O, sckit-learn, keras(w/ tensorflow) the only math you will absolutely 'need' is probably just Statistics. Regardless of what's going on behind the scenes with whatever automatically tuned and selected a…

What "maths" is keras? Or scikit-learn? For what it's worth, to understand scikit-learn doc/tutorial I'd say you'll need Probability, Linear Algebra, Multivariate Calculus and, yeah, Stats. Not necessarily at a PhD level but still. And more you understand maths farther you can get in AL/ML.

These libraries leave most of the actual day to day work for ETL. ETL happens to be highly data and problem dependent, so it can't be easily automated or reused. For this reason I think the best thing to be a good applied ML person is a solid programming background. You should have a working knowledge of statistics and linear algebra, but the most useful skill really is being able to write good code. It's different for research of course.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#92

Post by ratsimihah »

Make sure to differentiate between AI researcher and applied AI software engineer, or whatever that is called.

The former needs the mathematical background mentioned here to develop groundbreaking algorithms or improve on existing ones, while the latter merely implements them and requires a much smaller mathematical toolset.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#94

Post by VT_Drew »

CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

Game Theory would probably be more valuable to understand than graph theory. Just my 2 cents.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#96

Post by sgt101 »

srean wrote: post #23
popcorncolonel wrote: post #7

Earlier quoted context omitted.

I disagree that you need a solid founding in information theory. Almost all that I've seen about IT in ML is minimizing the KL divergence, which can be learned by browsing the wiki page.

It depends. All that is essential for an autombile engineer is not essential for a taxi driver.

Maybe more all that is essential for a molecular biologist isn't necessary for a general practitioner? It's just... those conference calls where you're explaining that because the classifier is working really well now doesn't mean that we can use it in production, those calls can get difficult and annoying, and sometimes the "other side" wins - with predictable results.

ha ha ha!

Re: Ask HN: What maths are critical to pursuing ML/AI?

#97

Post by srean »

sgt101 wrote: post #96
srean wrote: post #23

Earlier quoted context omitted.

It depends. All that is essential for an autombile engineer is not essential for a taxi driver.

Maybe more all that is essential for a molecular biologist isn't necessary for a general practitioner? It's just... those conference calls where you're explaining that because the classifier is working really well now doesn't mean that we can use it in production, those calls can get difficult and annoying, and sometimes the "other side" wins - with predictable results. ha ha ha!

You bring up a very important point and a difficult one which is, if the decision making is in the hands of someone who does not understand the nuances too well nor has the time or inclination, what do you do ?

If your salary is going to depend on how many models you pushed out and not how well they continued to perform, many will optimize over the number of models pushed out.

A major source of problem (and sometimes a gift) is that you cannot prove a empirical statistical claim true or false in finite time. There is always this non-zero probability that the weirdest thing would happen. It could be just sheer bad luck that the model did so poorly in this cycle.

Re: Ask HN: What maths are critical to pursuing ML/AI?

#98

Post by [deleted] »

rdudekul wrote: post #69
CuriouslyC wrote: post #2

You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye. It's a bit long in the tooth now, but it is still one of the most approachable and well written books in the fiel…

Free PDFs of some of the books mentioned: "Information Theory, Inference and Learning Algorithms" by David MacKaye http://www.inference.org.uk/itprnn/book.pdf "Probability Theory: the Logic of Science" by E. T. Jaynes http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia... "Elements of Statistical Learning" by Tibshirani https://web.stanford.edu/~hastie/Papers/ESLII.pdf "Bayesian Data Analysis" by Andrew Gelm…

[deleted]

Re: Ask HN: What maths are critical to pursuing ML/AI?

#100

Post by mtzet »

mindcrime wrote: post #77
digitalzombie wrote: post #26

Earlier quoted context omitted.

> "Bayesian Data Analysis" by Andrew Gelman is another great read. If you want to read that book you need real analysis more specifically measure theory (unless that subject is in probability theory for you). You cannot get into the last few chapters without it. Dirichlet Process are described using measures. I don't believe you need multivar calc or info theory. Info theory stuff are used but not as often. I believe…

What's required as a prereq to Measure Theory? Any suggestions on good resources for learning Measure Theory? I have a vague notion that Probability and Measure Theory are intertwined / related somehow, but have never studied the latter specifically.

The relationship is that measure theory provides the theoretical framework for making probability theory rigorous.

The only formal prerequisite for learning measure theory is that you should know series and sequences. For a reference, I'm not so sure, maybe Halmos's book. The important parts are probably:

- Monotone convergence theorem

- Dominated convergence theorem

- The construction of the Lebesgue integral

- Fubini's theorem and Tonelli's theorem

I would probably try not to get bogged down in details of construction of measures (unless you like that) and take the Lebesgue measure (essentially length) as given. Also check out the Radon-Nikodym theorem which states that we can always (ish) work with density functions.

Post reply on HN