Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introduction to Support Vector Machines in Machine Learning

ranvir.xyz

Post reply on HN
Print view View original

31–40 of 90 posts

Re: Introduction to Support Vector Machines in Machine Learning

#31

Post by ska »

uoaei wrote: post #17
astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Great comments. I heartily agree and support the statement about probabilistic graphical models. Just to add a couple more facets to this perspective: 'State of the art' does not always mean 'best for your task', and in fact lately depending on your field SOTA sometimes simply means 'unaffordable' for anyone whose budget is under 1 million dollars. Try linear methods first. Ensembles of decent models are usually good…

> Try linear methods first

This bears repeating.

Re: Introduction to Support Vector Machines in Machine Learning

#32

Post by mrslave »

astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Requesting best book(s) on probability estimation: techniques, model accuracy, and strategies in applying them (e.g. markets, marketing, business operations)?

Re: Introduction to Support Vector Machines in Machine Learning

#33

Post by bitL »

astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Starting with PGMs would kill 99.9% of aspiring ML practitioners. Classes related to PGM at Stanford and MIT are considered to be some of the most difficult ones. I'd rather recommend to start with something they are enthusiastic about and once they become sufficiently advanced, to naturally learn (H)PGM.

Re: Introduction to Support Vector Machines in Machine Learning

#34

Post by salty_biscuits »

bitL wrote: post #33
astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Starting with PGMs would kill 99.9% of aspiring ML practitioners. Classes related to PGM at Stanford and MIT are considered to be some of the most difficult ones. I'd rather recommend to start with something they are enthusiastic about and once they become sufficiently advanced, to naturally learn (H)PGM.

I think they said "aspire to", not start with. I would start with a naive bayes classifier, understand why it is naive and then go from there.

Re: Introduction to Support Vector Machines in Machine Learning

#35

Post by bearzoo »

_pastel wrote: post #21
smbrian wrote: post #4

Stay away, in my opinion. I spent a year supporting a SVM in a production machine learning application, and it made me wish the ML research community hadn't been so in love with them for so long. They're the perfect blend of theoretically elegant and practically impractical. Training scales as O(n^3), serialized models are heavyweight, prediction is slow. They're like Gaussian Processes, except warped and without any…

100% agree. What's the use case for SVMs? Linear models are simpler. GBMs are more powerful, more flexible, and faster. Every ML course I took had 3 weeks of problem sets on VC dimension and convex quadratic optimization in Lagrangian dual-space, while decision tree ensembles were lucky to get a mention. Meanwhile GBMs continue to win almost all the competitions where neural nets don't dominate. I suspect my professo…

Svms are, by default, linear models. The decision boundary in the Svm problem is linear and since it’s the max margin we may enjoy nice generalization properties (as you probably know).

You probably also know that decision tree boundaries are non Linear And piecewise. It’s not so straightforward to find splits on continuous features.

Ie If the data is linearly separable then why not. Even using hinge loss with nns is not uncommon.

You probably see gbms winning a lot of competitions compared to svms because a lot of competitions may have a lot of data and non linear decision boundaries. some problems don’t have these characteristics.

Re: Introduction to Support Vector Machines in Machine Learning

#36

Post by [deleted] »

astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

[deleted]

Re: Introduction to Support Vector Machines in Machine Learning

#37

Post by unishark »

astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Personally I'd advise against both SVM's and Bayesian methods for a beginner. Bayesian statistics is very much the deep end of the pool. Graphical models and Bayesian methods generally may make a comeback but such approaches have been superseded by other methods for good reasons, i.e. scaling.

A strong basis in statistics is certainly a great thing, but that can be maximum likelihood plus Bayes law (i.e. "MAP" estimation which is more of a hack to ML than an actual Bayesian method) and provide the big picture for almost everything.

Meanwhile a strong basis in "deterministic methods", as an alternative way to spend that learning effort, has its own rewards. The training algorithms for deep learning are also the hottest algorithm research area in machine learning, and are certainly applicable beyond deep learning. For that matter a thorough understanding of SVM delves into convex optimization, an extremely powerful framework as well.

Re: Introduction to Support Vector Machines in Machine Learning

#38

Post by sibmike »

antipaul wrote: post #25
astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Can you give a concrete example how a Bayesian concept provides practical help in ML model training or another ML task? Want to whet my appetite for your suggestion.

The most recent example has been supply failure detection in sales timeseries data with intermittent demand. Ended up using approach described in The Longest Run of Heads by Mark F. Schilling, which is outstandingly well written stats paper and a pleasure to read.

Re: Introduction to Support Vector Machines in Machine Learning

#39

Post by sjg007 »

bitL wrote: post #33
astrophysician wrote: post #12

If you're new to ML or datascience, I would recommend working to build a strong basis in Bayesian statistics. It will help you understand how all of the "canonical" ML methods relate to one another, and will give you a basis for building off of them. In particular, aspire to learn probabilistic graphical models + the libraries to train them (like pyro, tensorflow probability, Edward, Stan). They have a steep learning…

Starting with PGMs would kill 99.9% of aspiring ML practitioners. Classes related to PGM at Stanford and MIT are considered to be some of the most difficult ones. I'd rather recommend to start with something they are enthusiastic about and once they become sufficiently advanced, to naturally learn (H)PGM.

PGMs are graphs mapped into probability theory and vice versa.

Re: Introduction to Support Vector Machines in Machine Learning

#40

Post by jclos »

astrophysician wrote: post #20
tesseract2 wrote: post #14

Earlier quoted context omitted.

Thanks for this insight. Can you kindly also suggest a good book for someone to start with Bayesian Statistics? I could really use a suggestion about first and second book on this. About Probabilistic Graphical Models, is there book other than Daphne Koller's book that you would suggest?

I think PGM's are covered by a lot of "standard" ML texts -- someone else mentioned Murphy's book which is great and is humongous but is a good reference for pretty much every method under the sun. Bishop's Pattern Recognition and Machine Learning has a chapter thats free online: https://www.microsoft.com/en-us/research/wp-content/uploads/...

I'd just like to add that the entire PRML book is now free online, not just the sample: https://www.microsoft.com/en-us/research/uploads/prod/2006/0...
Post reply on HN