An Idiot’s guide to Support vector machines (2003) [pdf]
1–10 of 64 posts
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#2implementing a svm was my senior project in college. Brings back nightmares.
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#3It's interesting how quickly support vector machines went from the hot new thing to classify images to an afterthought after deep learning started having great results.
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#4Clearly idiots are not what they use to be in my day ...
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#5Back in around 2008, SVMs were all the rage in computer vision. We would use hand designed visual features and then a linear SVM on top. That was how object detectors were built (remember DPM?)
Funny how SVMs are just max-margin loss functions and we just took for granted that you needed domain expertise to craft features like HOG/SIFT by hand.
By 2018, we use ConvNets to learn BOTH the features and the classifier. In fact, it’s hard to separate where the features end and the classifier begins (in a modern CNN).
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#6It's interesting how quickly support vector machines went from the hot new thing to classify images to an afterthought after deep learning started having great results.
Noticed that too. It feels it was just a few years and all of the sudden everything is "deep" now.
The same thing happened with data storage. As soon as big data appeared everyone stopped doing just data and started doing "big data". Now the term is kind of a joke even.
I predict in a few years "deep learning" term will become mostly used in an ironic sense as well.
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#7Back in around 2008, SVMs were all the rage in computer vision. We would use hand designed visual features and then a linear SVM on top. That was how object detectors were built (remember DPM?) Funny how SVMs are just max-margin loss functions and we just took for granted that you needed domain expertise to craft features like HOG/SIFT by hand. By 2018, we use ConvNets to learn BOTH the features and the classifier. I…
It’s also hard to separate the design of the neural architecture from the definition of the feature extractor.
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#8It's interesting how quickly support vector machines went from the hot new thing to classify images to an afterthought after deep learning started having great results.
Noticed that too. It feels it was just a few years and all of the sudden everything is "deep" now. The same thing happened with data storage. As soon as big data appeared everyone stopped doing just data and started doing "big data". Now the term is kind of a joke even. I predict in a few years "deep learning" term will become mostly used in an ironic sense as well.
> I predict in a few years "deep learning" term will become mostly used in an ironic sense as well.
I may be a bit behind the times, but I'm also mystified by "deep learning's" popularity. Both giant neural nets and kernel methods have overfitting problems: torture a billion-parameter model long enough, and it will tell you what you want to hear.
SVMs address this by finding a large margin for error, which will hopefully improve generalization. DNNs (I think) do this by throwing more ("big") data at the problem and hoping that the training set covers all possible inputs. Work on adversarial learning suggests that DNNs go completely off the rails when presented with anything slightly unexpected.
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#9Back in around 2008, SVMs were all the rage in computer vision. We would use hand designed visual features and then a linear SVM on top. That was how object detectors were built (remember DPM?) Funny how SVMs are just max-margin loss functions and we just took for granted that you needed domain expertise to craft features like HOG/SIFT by hand. By 2018, we use ConvNets to learn BOTH the features and the classifier. I…
If you use the right sort of kernel for an SVM it becomes a neural network with automatic architecture derivation.
See slide 7:
http://www.cs.rpi.edu/~magdon/courses/LFD-Slides/SlidesLect2...
Re: An Idiot’s guide to Support vector machines (2003) [pdf]
#10Earlier quoted context omitted.
Noticed that too. It feels it was just a few years and all of the sudden everything is "deep" now. The same thing happened with data storage. As soon as big data appeared everyone stopped doing just data and started doing "big data". Now the term is kind of a joke even. I predict in a few years "deep learning" term will become mostly used in an ironic sense as well.
> I predict in a few years "deep learning" term will become mostly used in an ironic sense as well. I may be a bit behind the times, but I'm also mystified by "deep learning's" popularity. Both giant neural nets and kernel methods have overfitting problems: torture a billion-parameter model long enough, and it will tell you what you want to hear. SVMs address this by finding a large margin for error, which will hopef…
Empericially, CNNs generalize better on image recognition tasks than hand built features. This comment doesn't make much sense and is needlessly obtuse in the face of progress, tbh.