21–30 of 42 posts
This post, while very interesting, attempts to draw a completely unwarranted parallel between deep nets and the human brain, as if layers of artificial neurons running on a GPU and the cortical layers of your brain were two interchangeable things. So far, there has been no evidence that the brain works anything like an artificial neural network. Maybe it does, and there are several theories in that direction, but at…
Earlier quoted context omitted.
I feel like cloud technology may be a contributing factor as well. At the same time, once Geoffrey Hinton used a deep neural network and participated in the ImageNet contest (2012: http://www.image-net.org/challenges/LSVRC/2012/results.html ), his results beat the next best thing by a full 10%. The results were so astounding that many people immediately began re-visiting neural networks. Shortly afterwards, people pr…
But don't kid yourself, if you have ever been to CVPR, or one of the machine learning conferences you see all of the papers trying to squeeze a few more % accuracy on a test set out of existing algorithms. Maybe one or two papers actually do something novel. Rarely will their be a new approach altogether. The point I am trying to make is that one shouldn't get star eyed by leading researchers in the field and assume…
"so I implemented a small neural network and trained it to sort 10 6-bit numbers, which was easy to do to my surprise" Does anyone know what the inputs and outputs of a neural network that sorts numbers would look like?
Input: a 60-dimensional vector that is the concatenation of 10 6-dimensional binary vectors encoding the binary representation of the input numbers. Output: the same, sorted. At least that's one dead simple way to formulate the problem, multiple other solutions would work as well, and some would probably work better.
Example sorting 987654 and 123456
Input: 1, .9, .8, .7, .6, .5, .2, .3, .4, .5, .6, .7
Expected output: .2, .3, .4, .5, .6, .7, 1, .9, .8, .7, .6, .5
Whatever happened to shallow learning (or you know the regular learning) everyone did before deep learning. Anyone still doing that? Is this like BigData. As soon as someone mentioned BigData, anyone in the world who touched data all of the sudden did BigData. So is this something coming out of Google and Facebook and such and everyone else in Academia is happily building SVMs and 2 layer neural networks or some new…
Deep learning happened, and it pretty much always beats other approaches. Saying that sounds unbelievable, so here's a quote from Pete Warden:
I know I’m a broken record on deep learning, but almost everywhere it’s being applied it’s doing better than techniques that people have been developing for decades[1]
There's a great paper from a group of researchers who set out to prove that their technique, which they had many years of experience in (SVMs?) was just as good as deep learning (I can't remember their field). They ended up proving the opposite, and switched their whole lab over to doing deep learning. I can't find the paper (!!) so I'll refer you to [2] instead.
[1] http://petewarden.com/2015/01/01/five-short-links-76/
[2] http://petewarden.com/2014/06/10/why-is-everyone-so-excited-...
> ... human neurons are slow yet humans can perform lots of complicated tasks in a fraction of a second. More specifically, it is well-known that a human neuron fires no more than 100 times per second. This means that, if a human can solve a problem in 0.1 seconds, then our neurons have enough time to fire only 10 times --- definitely not much more than that. It therefore follows that a large neural network with 10 layers can do anything a human can in 0.1 seconds.
I found this to be insightful: > ... human neurons are slow yet humans can perform lots of complicated tasks in a fraction of a second. More specifically, it is well-known that a human neuron fires no more than 100 times per second. This means that, if a human can solve a problem in 0.1 seconds, then our neurons have enough time to fire only 10 times --- definitely not much more than that. It therefore follows that a…
This sounds presumptuous. Couldn't there be thousands of neural networks which all receive an input signal, and we know how to interpret the output of all of these, as a signal. Maybe the neural networks themselves are all only 10 layers deep, but if they're all running in parallel - this defeats this point, right?
I don't really know anything about it, though.
This post, while very interesting, attempts to draw a completely unwarranted parallel between deep nets and the human brain, as if layers of artificial neurons running on a GPU and the cortical layers of your brain were two interchangeable things. So far, there has been no evidence that the brain works anything like an artificial neural network. Maybe it does, and there are several theories in that direction, but at…
The point of drawing comparisons to the human brain is that we know how quickly humans can perform visual recognition tasks and speed of signal propagation between neurons. Combining these two properties implies that the human brain is able to solve these tasks without feedback, i.e. no loops. Thus, a DNN should be able to perform similar tasks if it can be trained (which it can). Recurrent neural nets add feedback a…
Current approach of feeding raw bitmaps to DNN falls short of that and is very sensitive to training data[1]
I remember an old paper I cannot find now about how to normalize image for NN processing in face recognition. Software extracted the face, centered it on a square and projected that square on a circle around the center to make face orientation irrelevant (hard to explain without images)
Anyway, it is unfair to expect a DNN to perform vision recognition tasks from raw bi-dimensional image points.
[1] http://www.i-programmer.info/news/105-artificial-intelligenc...
I found this to be insightful: > ... human neurons are slow yet humans can perform lots of complicated tasks in a fraction of a second. More specifically, it is well-known that a human neuron fires no more than 100 times per second. This means that, if a human can solve a problem in 0.1 seconds, then our neurons have enough time to fire only 10 times --- definitely not much more than that. It therefore follows that a…
I actually came in here to ask for clarification on this. This sounds presumptuous. Couldn't there be thousands of neural networks which all receive an input signal, and we know how to interpret the output of all of these, as a signal. Maybe the neural networks themselves are all only 10 layers deep, but if they're all running in parallel - this defeats this point, right? I don't really know anything about it, though…
The observation was a comment on how deep a network needs to be to perform useful tasks.
Good article, but I still don't understand why they're suddenly popular again. So processing is faster, but is there some development in processing which has improved this domain especially? Developments in "big data" processing? Concepts like mapreduce? What's with the resurgence :S
very funny ... as if ANNs are sufficiently comparable to actual neural activity. also I think it is naive to assess the "powerful"-ness of the brain to what is going on in a single neuron - it is certainly the parallel interaction which creates the human intelligence.
> And if human neurons turn out to be noisy (for example), which m...
it is pretty naive to consider noise as something of only handicapping nature - a lot of algorithms are as powerful as they are by utilizing noise and stochasticity
> What is learning? Learning is the problem of finding a setting of the neural network’s weights that achieves the best possible results on our training data.
Wrong - this is memorizing ... learning is the process leading to a low out-of-sample error.