A Brief Overview of Deep Learning
yyue.blogspot.com
A Brief Overview of Deep Learning
1–10 of 42 posts
Re: A Brief Overview of Deep Learning
#2Good article, but I still don't understand why they're suddenly popular again. So processing is faster, but is there some development in processing which has improved this domain especially? Developments in "big data" processing? Concepts like mapreduce? What's with the resurgence :S
Re: A Brief Overview of Deep Learning
#3Good article, but I still don't understand why they're suddenly popular again. So processing is faster, but is there some development in processing which has improved this domain especially? Developments in "big data" processing? Concepts like mapreduce? What's with the resurgence :S
Deep neural networks are quite powerful because they don't require someone to design the network specific to the task. Large deep nets will generally be able to learn feature representations by themselves, which makes them extremely powerful for tasks where we aren't quite sure of exactly what features we want.
However, deep nets require large amounts of training data and computational power. The fairly recent widespread adoption of general purpose GPUs has allowed much faster training. Combine this with the popularity of "big data", and you've got a perfect storm for deep neural nets.
Of course, the hype may be overvaluing deep nets as the future of AI. DNNs work well in practical applications, but they're poorly defined theoretically and the AI community suspects that we're still bad at training them -- a recent paper showed that a simpler shallow net can perform as well as a deep net if a deep net is trained first[1]. We're also fairly certain that deep nets are not how the brain actually works, and thus we'll need a different architecture in order to achieve human level performance on some tasks.
Re: A Brief Overview of Deep Learning
#4Good article, but I still don't understand why they're suddenly popular again. So processing is faster, but is there some development in processing which has improved this domain especially? Developments in "big data" processing? Concepts like mapreduce? What's with the resurgence :S
[deleted]
Re: A Brief Overview of Deep Learning
#5Good article, but I still don't understand why they're suddenly popular again. So processing is faster, but is there some development in processing which has improved this domain especially? Developments in "big data" processing? Concepts like mapreduce? What's with the resurgence :S
I feel like cloud technology may be a contributing factor as well.
At the same time, once Geoffrey Hinton used a deep neural network and participated in the ImageNet contest (2012: http://www.image-net.org/challenges/LSVRC/2012/results.html), his results beat the next best thing by a full 10%. The results were so astounding that many people immediately began re-visiting neural networks. Shortly afterwards, people proved it could beat the current technology for language processing and more. Now a days, it seems like a major leap has been in real-time translation with Skype and now Google launching machine translation applications/functions.
Side note, in my opinion, start-ups that are looking to compete with large giants like Google will have a pretty hard time. In the end, implementing deep neural nets that work is still extremely hard. The companies that do it right usually get bought up by one of the giants. Google has some of the leading researchers in academia on its side as well.
Re: A Brief Overview of Deep Learning
#6This post, while very interesting, attempts to draw a completely unwarranted parallel between deep nets and the human brain, as if layers of artificial neurons running on a GPU and the cortical layers of your brain were two interchangeable things.
So far, there has been no evidence that the brain works anything like an artificial neural network. Maybe it does, and there are several theories in that direction, but at the moment we have no solid reason to think so.
Re: A Brief Overview of Deep Learning
#7Good article, but I still don't understand why they're suddenly popular again. So processing is faster, but is there some development in processing which has improved this domain especially? Developments in "big data" processing? Concepts like mapreduce? What's with the resurgence :S
Hinton published a much more efficient training algoritm in the last couple decades. They've also shown extremely impressive performance in vision and speech.
Re: A Brief Overview of Deep Learning
#8This post, while very interesting, attempts to draw a completely unwarranted parallel between deep nets and the human brain, as if layers of artificial neurons running on a GPU and the cortical layers of your brain were two interchangeable things. So far, there has been no evidence that the brain works anything like an artificial neural network. Maybe it does, and there are several theories in that direction, but at…
[deleted]
Re: A Brief Overview of Deep Learning
#9Has anyone had experience training deep nets for domains in which examples are large heterogenous collections (as opposed to speech, or text, or images), like say transactional or click-stream data?
Re: A Brief Overview of Deep Learning
#10"so I implemented a small neural network and trained it to sort 10 6-bit numbers, which was easy to do to my surprise"
Does anyone know what the inputs and outputs of a neural network that sorts numbers would look like?