31–38 of 38 posts
Earlier quoted context omitted.
New frontier? They've been popular for over 20 years.
That's AI. Every generation invents everything again. Probably works this way in other fields too. (Yes this is hyperbole, but any old timers out there will know the feeling)
Also "Past decades was AI winter. But no worries, now we'll start seeing human intelligence in a about next year" probably heard that since late 90's.
Could someone explain in layman words what is the qualitative contribution of the paper and why it is important?
Sure. Random forests are a very powerful machine learning method that has proven to give excellent performance with very little tuning. Neural networks with many layers (aka deep learning) is the new frontier of machine learning - they are very powerful but require extensive tuning (the architecture of the network) and lots of examples before they become practical. In this paper, the authors combine the two methods -…
Yay! Now Decision Trees too can get stuck in local optima! :D
(totally tongue in cheek)
Why do they only compare their method against GoogLeNet? VGG was the winner in 2014, most papers seem to use VGG and my team has observed it to always gets higher accuracy than GoogLeNet.
I'm not entirely sure, but I think that the multiple softmax layers in GoogLeNet might make it easier to modify for this purpose than the VGG architecture.
[1] http://image-net.org/challenges/LSVRC/2014/results (look for "Classification+localization with provided training data: Ordered by classification error")
Earlier quoted context omitted.
Deep learning has really only recently become successful with new learning algorithms such as constrastive divergence and convolutional neural networks. Previous efforts were focused around backpropagation, but due to the signal loss across many layers there was never enough information in the output layer to successfully train the network.
Time for a fact update! My, my, how time flies. Deep learning has really only recently become successful hinton coined the term "deep learning" around 2006/2007 (more around deep belief nets/RBMs, but still, same thing), if that's considered "recently." constrastive divergence and convolutional neural networks. CD was also ~10 years ago. CNNs were reading your checks and postal zipcodes in the mid 90s. successfully t…
Before that there was little evidence that any form of neural network was massively better than other forms of machine learning. Now it has become clear that isn't the case.
Earlier quoted context omitted.
Sure. Random forests are a very powerful machine learning method that has proven to give excellent performance with very little tuning. Neural networks with many layers (aka deep learning) is the new frontier of machine learning - they are very powerful but require extensive tuning (the architecture of the network) and lots of examples before they become practical. In this paper, the authors combine the two methods -…
>> One of the main theoretical results in the paper is that they propose a differentiable .. technique to optimize the parameters of the trees. Yay! Now Decision Trees too can get stuck in local optima! :D (totally tongue in cheek)
Why do they only compare their method against GoogLeNet? VGG was the winner in 2014, most papers seem to use VGG and my team has observed it to always gets higher accuracy than GoogLeNet.
GoogLeNet "won" ImageNet 2014[1], but VGG is often found to be more flexible. I'm not entirely sure, but I think that the multiple softmax layers in GoogLeNet might make it easier to modify for this purpose than the VGG architecture. [1] http://image-net.org/challenges/LSVRC/2014/results (look for "Classification+localization with provided training data: Ordered by classification error")
Earlier quoted context omitted.
New frontier? They've been popular for over 20 years.
Old time neural networks not even came close to current models in terms of performance. I clearly remember in my machine learning class, one professor mentions neural networks and says it is slow and impossible to tame when the layers goes up thus loses its popularity. That is just 3-4 years ago.
Earlier quoted context omitted.
>> One of the main theoretical results in the paper is that they propose a differentiable .. technique to optimize the parameters of the trees. Yay! Now Decision Trees too can get stuck in local optima! :D (totally tongue in cheek)
Even though this is tongue-in-cheek, it makes no sense. Decision trees are traditionally trained in a greedy, non-globally-optimal fashion.
That's the traditional view. I guess it's always possible to see decision trees as optimising a binary function, but that's probably to be filed under "original research". Or maybe not- I'm not sure.
In any case, I've never heard of anyone worried that a tree learner would get stuck anywhere. The concern is usually with overfitting.