Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Machine Learning: From Theory to Algorithms (2014)

cs.huji.ac.il

Post reply on HN
Print view View original

51–58 of 58 posts

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#51

Post by mindcrime »

davmre wrote: post #12
cdnsteve wrote: post #7

I feel like the barrier to machine learning for me, as I've seen in many tutorials and books and is an immediate discouragement, is the massive amount of math thrown in your face. Many of us didn't just graduate, need glasses and fall asleep at 8pm on the couch when the kids are in bed... Math is this distant fragment of memory buried under years of everything not Math. It feels like machine learning is only taught b…

Andrew Ng's Coursera ML course is supposed to be pretty accessible. I've also heard good things about Machine Learning for Hackers ( http://www.amazon.com/Machine-Learning-Hackers-Drew-Conway/d... ). Ultimately, ML is a mathematical discipline. You can ask for a gentle approach that gets you to the foot of the mountain, but "if you want to learn about nature, to appreciate nature, it is necessary to understand the la…

so it shouldn't be beyond reach of a motivated developer able to set aside some time to learn. MOOCs and organizing study groups with friends/co-workers can help a lot here as well.

I'm in the middle of that process right now. I only took Calc I in college and that was 20 years ago, so I have decided to work my way through a Calc sequence, Differential Equations, Linear Algebra, and Probability and Statistics through a combination of MOOCs, "X For Dummies" books, Youtube videos (hello, Gilbert Strang!), Schaum's Outlines books, Khan Academy, a mammoth stack of college maths texts that I've picked up at used book stores, and questions on stats.stackexchange.com, math.stackexchange.com, learnmath.reddit.com, etc.

I'm doing the Ohio State MOOC on Calc I now on Coursera, and accompanying that with the Gilbert Strang "Highlights of Calculus" video series[1]. So far so good. I definitely think this stuff is learnable if one is willing to put in the time and work, even without going back to taking "on campus" classes at a university.

[1]: https://www.youtube.com/playlist?list=PLFW_V3qDH5jRyfpD9uiq6...

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#52

Post by nafizh »

gajjanag wrote: post #37
arbitrage314 wrote: post #31

I'm a math geek, but I'm also a mostly self-taught data scientist. "The Elements of Statistical Learning" ( https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI... ) is far and away the best book I've seen. It took me hundreds of hours to get through it, but if you're looking to understand things at a pretty deep level, I'd say it's well-worth it. Even if you stop at chapter 3, you'll still know more than mos…

I am a graduate student at MIT, and can second this recommendation. It is a fantastic book for machine learning and nothing else I have seen comes close.

You meant ESL or UML?

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#53

Post by mailshanx »

dxbydt wrote: post #26
cdnsteve wrote: post #7

I feel like the barrier to machine learning for me, as I've seen in many tutorials and books and is an immediate discouragement, is the massive amount of math thrown in your face. Many of us didn't just graduate, need glasses and fall asleep at 8pm on the couch when the kids are in bed... Math is this distant fragment of memory buried under years of everything not Math. It feels like machine learning is only taught b…

I interview a lot of developers for ML positions at our company. The first red flag is always a lack of math. Candidates who come in with API-level competence ie. can implement an ML algo using this,that or the other API, without any understanding of some basic math behind it, always fare poorly. Atleast in ML, not having an understanding of math is pretty much like claiming expertise in riding a bicycle by watching…

>> He'll happily throw in your zip code and credit card number into the mix, not realizing that those two features have super high entropy

You will be surprised: you can make significant gains by including the zip code - i've seen that happen in a competitive setting. Where you live probably contains some signal about your credit worthiness.

Having said that, of course it doesn't make sense to simply feed the raw zip code to the tree. An appropriate encoding (most people would use a one-hot encoding, though there exist better ones) of the zip code will be key to extracting signal in a robust way.

>> ... the tree will have serious overfitting issues

Isn't it an almost standard practise now to use an ensemble of trees, such as a random forest? Decision trees have long been known to be prone to overfitting.

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#54

Post by lakeeffect »

nafizh wrote: post #52
gajjanag wrote: post #37

Earlier quoted context omitted.

I am a graduate student at MIT, and can second this recommendation. It is a fantastic book for machine learning and nothing else I have seen comes close.

You meant ESL or UML?

ESL. His post was an hour before the reader5000 uml post.

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#55

Post by gtani »

jzymbaluk wrote: post #49
dxbydt wrote: post #26

Earlier quoted context omitted.

I interview a lot of developers for ML positions at our company. The first red flag is always a lack of math. Candidates who come in with API-level competence ie. can implement an ML algo using this,that or the other API, without any understanding of some basic math behind it, always fare poorly. Atleast in ML, not having an understanding of math is pretty much like claiming expertise in riding a bicycle by watching…

As an undergrad interested in machine learning, what areas of math specifically should I be focusing on?

here's a list of open content (free pdf) texts that have appendixes or review chapters

- UML

- http://goodfeli.github.io/dlbook/

- 3 books: Barber, MacKay, and Rasmussen/Williams from this list: https://www.reddit.com/r/MachineLearning/comments/1jeawf/mac...

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#56

Post by onaclov2000 »

sotojuan wrote: post #43
Hockenbrizzle wrote: post #29

Earlier quoted context omitted.

A lack of mathematical intuition is a serious problem for many people from engineering to biology to economics. It certainly plagued me throughout my engineering bachelors studies and is something I continually work to get better at. In my opinion, physics students learn the best framework for thinking and get a very good mathematical intuition. For example, here's a problem from an introductory QM book that really t…

> A lack of mathematical intuition is a serious problem for many people from engineering to biology to economics. This is true. I can't really say why, but after my Discrete Mathematics class a lot of my Computer Science problems became a lot easier to reason about.

I will say that after my discrete math class we really only talked about how to write a proof, and well, it really didn't help for me. (I think we were supposed to get further into stuff, but well, the class wasn't paced well, new professor, etc).

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#57

Post by arbitrage314 »

reader5000 wrote: post #40
arbitrage314 wrote: post #31

I'm a math geek, but I'm also a mostly self-taught data scientist. "The Elements of Statistical Learning" ( https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI... ) is far and away the best book I've seen. It took me hundreds of hours to get through it, but if you're looking to understand things at a pretty deep level, I'd say it's well-worth it. Even if you stop at chapter 3, you'll still know more than mos…

Having read significant chunks of both ESL and Understanding Machine Learning (albeit UML much more recently) I would argue that for many readers UML is superior. ESL pays short shrift to the computational complexity of learning whereas UML explicitly handles both statistical and computational complexity concerns. It doesnt matter how statistically pure your algorithm is if its running time scales exponentially with…

Thanks for the interesting comparison!

Re: Understanding Machine Learning: From Theory to Algorithms (2014)

#58

Post by kewlb »

cdnsteve wrote: post #7

I feel like the barrier to machine learning for me, as I've seen in many tutorials and books and is an immediate discouragement, is the massive amount of math thrown in your face. Many of us didn't just graduate, need glasses and fall asleep at 8pm on the couch when the kids are in bed... Math is this distant fragment of memory buried under years of everything not Math. It feels like machine learning is only taught b…

Certainly understanding the math is very important but it is harder to get expertise on the pre-requisite math because the horizon is much bigger. I would recommend taking a case study approach and side by side learning the math stuff needed. If you are looking for an example then take a look at this, https://www.coursera.org/learn/ml-foundations/
Post reply on HN