A Quick Look at Support Vector Machines
generalabstractnonsense.com
A Quick Look at Support Vector Machines
1–10 of 39 posts
Re: A Quick Look at Support Vector Machines
#2This is great! Any follow-ups describing kernels?
Re: A Quick Look at Support Vector Machines
#3One thing that I discovered recently which surprised me (while taking the Udacity SDC)is how effective and resilient these "older" ML algorithms can be. Neural networks was always my go to method for most of my classification or regression problems for my small side projects. But now I learned with the minimal dataset I have (<5K samples), linear regression, SVM, or decision tress is the way to go. I got higher accuracy and it's about 10X faster in terms of computational time!
Re: A Quick Look at Support Vector Machines
#4One thing that I discovered recently which surprised me (while taking the Udacity SDC)is how effective and resilient these "older" ML algorithms can be. Neural networks was always my go to method for most of my classification or regression problems for my small side projects. But now I learned with the minimal dataset I have (<5K samples), linear regression, SVM, or decision tress is the way to go. I got higher accur…
[deleted]
Re: A Quick Look at Support Vector Machines
#5One thing that I discovered recently which surprised me (while taking the Udacity SDC)is how effective and resilient these "older" ML algorithms can be. Neural networks was always my go to method for most of my classification or regression problems for my small side projects. But now I learned with the minimal dataset I have (<5K samples), linear regression, SVM, or decision tress is the way to go. I got higher accur…
On the other hand SVM doesn't scale as well as neural networks do because it has computational complexity between O(n^2) and O(n^3) [1] where n is the number of samples in the training set. So if you plan to add more data later you may eventually encounter scaling problems with SVM.
[1] http://scikit-learn.org/stable/modules/svm.html#complexity
Re: A Quick Look at Support Vector Machines
#6One thing that I discovered recently which surprised me (while taking the Udacity SDC)is how effective and resilient these "older" ML algorithms can be. Neural networks was always my go to method for most of my classification or regression problems for my small side projects. But now I learned with the minimal dataset I have (<5K samples), linear regression, SVM, or decision tress is the way to go. I got higher accur…
On the other hand SVM doesn't scale as well as neural networks do because it has computational complexity between O(n^2) and O(n^3) [1] where n is the number of samples in the training set. So if you plan to add more data later you may eventually encounter scaling problems with SVM. [1] http://scikit-learn.org/stable/modules/svm.html#complexity
Good to know, I did not know that! I kind of wish scikit had some sort of CUDA capabilities to speed things up.
Re: A Quick Look at Support Vector Machines
#7One thing that I discovered recently which surprised me (while taking the Udacity SDC)is how effective and resilient these "older" ML algorithms can be. Neural networks was always my go to method for most of my classification or regression problems for my small side projects. But now I learned with the minimal dataset I have (<5K samples), linear regression, SVM, or decision tress is the way to go. I got higher accur…
On the other hand SVM doesn't scale as well as neural networks do because it has computational complexity between O(n^2) and O(n^3) [1] where n is the number of samples in the training set. So if you plan to add more data later you may eventually encounter scaling problems with SVM. [1] http://scikit-learn.org/stable/modules/svm.html#complexity
We found great success with the LIBLINEAR SVM implementation [1] though: Extremely good performance, to the point that it affects scalability too, with predictive performance acceptably close to libSVM with the RBF kernel, for a large cheminformatics dataset:
Paper (open acccess): http://dx.doi.org/10.1186/s13321-016-0151-5
As can be seen in fig 5 [2] in the paper, a dataset size that took ~1 week with libSVM (actually, the parallel piSVM implementation) on 64 cores, took less than a minute with LIBLINEAR, which runs on just one core.
[1] https://www.csie.ntu.edu.tw/~cjlin/liblinear
[2] http://jcheminf.springeropen.com/articles/10.1186/s13321-016...
Re: A Quick Look at Support Vector Machines
#8One thing that I discovered recently which surprised me (while taking the Udacity SDC)is how effective and resilient these "older" ML algorithms can be. Neural networks was always my go to method for most of my classification or regression problems for my small side projects. But now I learned with the minimal dataset I have (<5K samples), linear regression, SVM, or decision tress is the way to go. I got higher accur…
Yes SVMs are still great models. The advantage neural nets have over them are that they can do automatic feature extraction. By the time you get to the last layer of a neural net, you are basically just doing a simple logistic classification, but the features coming in have been learned from all of the previous layers.
I've even seen people use pretrained ImageNet classifiers, chop off the last layer and use an SVM as the actual classifier, and it works very well for some problems.
Re: A Quick Look at Support Vector Machines
#9Aaah, I was hoping for an explanation of the kernel trick. I think that is the hardest concept in support vector machines.
Re: A Quick Look at Support Vector Machines
#10Many aspects of Machine Learning boil down to optimization problems.