What's Wrong with Deep Learning?
docs.google.com
What's Wrong with Deep Learning?
1–10 of 48 posts
Re: What's Wrong with Deep Learning?
#2Wow what in incredible amount of knowledge in those slides. Is there a video of the keynote?
Re: What's Wrong with Deep Learning?
#3Wow what in incredible amount of knowledge in those slides. Is there a video of the keynote?
CVPR usually makes all talks available when the conference has concluded. Watch this page http://www.pamitc.org/cvpr15/.
Re: What's Wrong with Deep Learning?
#4It is always impressive to me how both Professor LeCun, Bengio & Hinton stuck to their guns and worked on these problems while others were not so interested. I'm very glad Hinton's group at DNN Research was able to blow away the competition in the Imagenet challenge. Now many more people are working on these ideas and some really amazing things have been accomplished in a very short time. I can't wait to see where we are in five years, and I love that LeCun's points out the areas where we should focus and what we are not good at yet.
Re: What's Wrong with Deep Learning?
#5It is always impressive to me how both Professor LeCun, Bengio & Hinton stuck to their guns and worked on these problems while others were not so interested. I'm very glad Hinton's group at DNN Research was able to blow away the competition in the Imagenet challenge. Now many more people are working on these ideas and some really amazing things have been accomplished in a very short time. I can't wait to see where we…
Do you have any information about what were the hot topics just before this new resurgence of NNs (meaning around late 90's, and early 2000's).
I know symbolic AI was big in 60s, and 80s, but not sure about recent past.
Re: What's Wrong with Deep Learning?
#6It is always impressive to me how both Professor LeCun, Bengio & Hinton stuck to their guns and worked on these problems while others were not so interested. I'm very glad Hinton's group at DNN Research was able to blow away the competition in the Imagenet challenge. Now many more people are working on these ideas and some really amazing things have been accomplished in a very short time. I can't wait to see where we…
Do you have any information about what were the hot topics just before this new resurgence of NNs (meaning around late 90's, and early 2000's). I know symbolic AI was big in 60s, and 80s, but not sure about recent past.
Support Vector Machines was one of the hot topics.
Re: What's Wrong with Deep Learning?
#7It is always impressive to me how both Professor LeCun, Bengio & Hinton stuck to their guns and worked on these problems while others were not so interested. I'm very glad Hinton's group at DNN Research was able to blow away the competition in the Imagenet challenge. Now many more people are working on these ideas and some really amazing things have been accomplished in a very short time. I can't wait to see where we…
Do you have any information about what were the hot topics just before this new resurgence of NNs (meaning around late 90's, and early 2000's). I know symbolic AI was big in 60s, and 80s, but not sure about recent past.
Probabilistic models. Recent research often focuses on Bayesian models.
Probabilistic models have never really gone away. This presentation by LeCun actually suggests embedding neural networks inside of various types of probabilistic models: factor graphs and conditional random fields. This is, for example, how speech recognition works: the output of a neural network is fed into a probabilistic model (a hidden Markov model).
Re: What's Wrong with Deep Learning?
#8Earlier quoted context omitted.
Do you have any information about what were the hot topics just before this new resurgence of NNs (meaning around late 90's, and early 2000's). I know symbolic AI was big in 60s, and 80s, but not sure about recent past.
Probabilistic models. Recent research often focuses on Bayesian models. Probabilistic models have never really gone away. This presentation by LeCun actually suggests embedding neural networks inside of various types of probabilistic models: factor graphs and conditional random fields. This is, for example, how speech recognition works: the output of a neural network is fed into a probabilistic model (a hidden Markov…
Actually, state of the art speech recognition has switched over to having a Recursive Neural Network directly run over the audio input. Take a look at the paper at http://arxiv.org/abs/1412.5567 and http://usa.baidu.com/deep-speech-lessons-from-deep-learning/
However combining learning features with other systems is a very powerful approach and combining SVM's on top of the learned features of a Neural Network I would say is common. I personally am more interested in approaches like Deep Fried Convnets (http://arxiv.org/abs/1412.7149) that combine kernel methods as part of the Neural Networks themselves.
Re: What's Wrong with Deep Learning?
#9Earlier quoted context omitted.
Probabilistic models. Recent research often focuses on Bayesian models. Probabilistic models have never really gone away. This presentation by LeCun actually suggests embedding neural networks inside of various types of probabilistic models: factor graphs and conditional random fields. This is, for example, how speech recognition works: the output of a neural network is fed into a probabilistic model (a hidden Markov…
Actually, state of the art speech recognition has switched over to having a Recursive Neural Network directly run over the audio input. Take a look at the paper at http://arxiv.org/abs/1412.5567 and http://usa.baidu.com/deep-speech-lessons-from-deep-learning/ However combining learning features with other systems is a very powerful approach and combining SVM's on top of the learned features of a Neural Network I woul…
I know that Andrew Ng and colleagues say that they don't use HMMs. I haven't spoken with them (I haven't seen them at speech conferences) so I do not know whether they actually believe this themselves.
I believe the best comparison between "CTC" (which is billed as recurrent neural networks without the HMMs) and the traditional approach is by people at Google, Sak et al, "Learning Acoustic frame labeling for speech recognition with recurrent neural networks", ICASSP 2015. (I can't find a PDF online.)
Re: What's Wrong with Deep Learning?
#10I would like to see or hear more regarding the theory slides - in particular on the objective being a piecewise polynomial, and the distribution of weights using random matrix theory. Anyone know where I could find more?