Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Crash Course: Part 2

ml.berkeley.edu

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Machine Learning Crash Course: Part 2

#21

Post by wjn0 »

davmre wrote: post #15
wjn0 wrote: post #6

Earlier quoted context omitted.

Where would you place "Elements of Statistical Learning" in relation to these, if you know?

I haven't read it in detail, but my impression is that it is mathy, like Bishop, but focuses more on 'classical' frequentist analysis, whereas Bishop takes a more open-ended Bayesian perspective and covers important machinery like graphical models and inference algorithms that I don't think are in ESL. As a researcher I tend to prefer the Bayesian perspective in Bishop, because it gives you a unifying framework for t…

I appreciate the comparison. My field is computational biology, and the latest edition of ESL has some specific, relevant examples, which is what drew me to it in the first place. I'm currently working on getting my statistics background up to par, and then I'll be choosing an ML textbook. However, your note about ESL focusing on frequentist models gives me pause about my original choice, as it would seem to me biological applications naturally lend themselves to Bayesian methods. I think in the end I'll have to crack open several books to decide. Thanks for the input.

Re: Machine Learning Crash Course: Part 2

#22

Post by wjn0 »

platz wrote: post #10
wjn0 wrote: post #6

Earlier quoted context omitted.

Where would you place "Elements of Statistical Learning" in relation to these, if you know?

You want ISLr ( http://www-bcf.usc.edu/~gareth/ISL/ ), not ESL. ESL is the prototype for the former.

Interesting, thanks - I wasn't aware of this text.

But according to your link, ESL isn't a prototype of ISL, but a more 'advanced treatment'. The R applications in ISL seem like they might be very useful, though.

Re: Machine Learning Crash Course: Part 2

#23

Post by visarga »

wiradikusuma wrote: post #8

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…

> Forget ML, if you can't hardcode if-else, just use rules engine. What do you think?

Load data in X, model.fit(X, y), predictions = model.predict(new_X). It's not more complicated to apply ML.

Re: Machine Learning Crash Course: Part 2

#24

Post by argonaut »

wjn0 wrote: post #22
platz wrote: post #10

Earlier quoted context omitted.

You want ISLr ( http://www-bcf.usc.edu/~gareth/ISL/ ), not ESL. ESL is the prototype for the former.

Interesting, thanks - I wasn't aware of this text. But according to your link, ESL isn't a prototype of ISL, but a more 'advanced treatment'. The R applications in ISL seem like they might be very useful, though.

ISLR was written after ESL to be an easier to read version. I guarantee you will have trouble if you try to teach yourself ML with ESL.

Re: Machine Learning Crash Course: Part 2

#25

Post by davmre »

wjn0 wrote: post #21
davmre wrote: post #15

Earlier quoted context omitted.

I haven't read it in detail, but my impression is that it is mathy, like Bishop, but focuses more on 'classical' frequentist analysis, whereas Bishop takes a more open-ended Bayesian perspective and covers important machinery like graphical models and inference algorithms that I don't think are in ESL. As a researcher I tend to prefer the Bayesian perspective in Bishop, because it gives you a unifying framework for t…

I appreciate the comparison. My field is computational biology, and the latest edition of ESL has some specific, relevant examples, which is what drew me to it in the first place. I'm currently working on getting my statistics background up to par, and then I'll be choosing an ML textbook. However, your note about ESL focusing on frequentist models gives me pause about my original choice, as it would seem to me biolo…

You're welcome! FWIW I mostly agree with argonaut's point elsewhere in this thread - very few people successfully self-teach ML from a textbook alone. So whichever book(s) you choose, it might also be worth working through some course materials. I've already suggested Stanford's CS229 for solid foundations, but depending on your interests in bioinformatics, Daphne Koller's Coursera course on probabilistic graphical models (https://www.coursera.org/learn/probabilistic-graphical-model...) might be especially relevant. Koller literally wrote the book on PGMs, has done a lot of work in comp bio, and her MOOC is apparently the real deal: very intense but well-reviewed by the people that make it through.

Re: Machine Learning Crash Course: Part 2

#26

Post by nilknarf »

gtani wrote: post #16
davmre wrote: post #5

Earlier quoted context omitted.

You can order the standard machine learning texts from most to least math-y, and least to most modern: - Pattern Recognition and Machine Learning (Bishop 2007) - Machine Learning: A Probabilistic Perspective (Murphy 2012) - Deep Learning (Goodfellow, Bengio, Courville 2016) If you want cutting-edge material, read the Deep Learning book (which is still quite technical, though some of its content may be outdated in a f…

Besides the DL book, there's other excellent texts that are freely available/open content on the web: - Elements statistical Learning, Hastie et al - Shalev-Shwartz and Ben-David: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning... - the late David MacKay's Info Theory - Bayesian Reasoning in ML, Barber - Hopcroft/Kannan (this is an older version, you can google latest: http://www.cs.cornell.edu/jeh/book1…

I didn't expect to see Hopcroft/Kannan in that list. I used it in a course taught by Kannan and back then it was called "Computer Science Theory for the Information Age". Apparently the book is now called "Foundations of Data Science". I always thought the old naming was terrible but compared to the new title it actually describes the content a lot better since the book is mainly about CS theory and mathematical foundations for them. It was one of my favorite courses but I would not classify it as ML book.

[1]https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/

[2]https://www.cs.cornell.edu/jeh/book2016June9.pdf

Re: Machine Learning Crash Course: Part 2

#27

Post by akg_67 »

wiradikusuma wrote: post #8

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…

What you are describing is the basis for decision tree and random forest.

Re: Machine Learning Crash Course: Part 2

#28

Post by signa11 »

wiradikusuma wrote: post #8

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…

> (1) we're too lazy to formulate the rules, or (2) the data is too complex/big to analyze by rules

in most of the cases we don't even know _how_ to formulate the rules. for example, if someone asks you, what makes an 'a' a 'a' ? what would be your response ?

Re: Machine Learning Crash Course: Part 2

#29

Post by [deleted] »

wiradikusuma wrote: post #8

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…

[deleted]

Re: Machine Learning Crash Course: Part 2

#30

Post by nrao123 »

wiradikusuma wrote: post #8

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, fin…

I would recommend reading Paul Graham's Paul Graham's Plan for Spam- one of the early extremely useful ML approaches to fighting spam. http://www.paulgraham.com/spam.html

In the article he outlines the challenges of taking a rules based approach:

The statistical approach is not usually the first one people try when they write spam filters. Most hackers' first instinct is to try to write software that recognizes individual properties of spam. You look at spams and you think, the gall of these guys to try sending me mail that begins "Dear Friend" or has a subject line that's all uppercase and ends in eight exclamation points. I can filter out that stuff with about one line of code.

And so you do, and in the beginning it works. A few simple rules will take a big bite out of your incoming spam. Merely looking for the word "click" will catch 79.7% of the emails in my spam corpus, with only 1.2% false positives.

I spent about six months writing software that looked for individual spam features before I tried the statistical approach. What I found was that recognizing that last few percent of spams got very hard, and that as I made the filters stricter I got more false positives.

False positives are innocent emails that get mistakenly identified as spams. For most users, missing legitimate email is an order of magnitude worse than receiving spam, so a filter that yields false positives is like an acne cure that carries a risk of death to the patient.

The more spam a user gets, the less likely he'll be to notice one innocent mail sitting in his spam folder. And strangely enough, the better your spam filters get, the more dangerous false positives become, because when the filters are really good, users will be more likely to ignore everything they catch.

Post reply on HN