A Brief Overview of Deep Learning

Post by im2w1l » Fri, Jan 16, 2015, 10:36 AM UTC

sesqu wrote: post #28

gravity13 wrote: post #26

Earlier quoted context omitted.

I actually came in here to ask for clarification on this. This sounds presumptuous. Couldn't there be thousands of neural networks which all receive an input signal, and we know how to interpret the output of all of these, as a signal. Maybe the neural networks themselves are all only 10 layers deep, but if they're all running in parallel - this defeats this point, right? I don't really know anything about it, though…

Combining a thousand networks into one signal is effectively one extra layer (though I don't think human neurons can actually handle a thousand incoming connections). The observation was a comment on how deep a network needs to be to perform useful tasks.

Humans have >10^14 synapses and slighly short of 10^11 neurons. Neurons thus have more than thousand incoming connections on average.

Purkinje cells can have on the order of hundreds of thousands of inputs.

Post by Houshalter » Sun, Jan 18, 2015, 8:30 AM UTC

dharma1 wrote: post #33

I've been playing with Caffe for recognising images. It's kind of mind blowing how well it works. Yet the networks I tested could "only" recognise photos, not drawings or anything abstract. A human could easily attribute meaning to a drawing, even if the drawing was very abstract or she had never seen a similar drawing before. Whereas a deep networks seem to rely on visual similarity to things it has seen in the past…

NN's have been able to represent fairly abstract art, e.g. http://i.imgur.com/HU66Vo7.png?1

They can also generate abstract images when the images are optimized to be recognized by the NN: http://i.imgur.com/Mixk96V.png?1

I think it's likely that cartoons contain a lot of meaning and symbols that is specific to human culture. Imagine a stick figure in the simplest case. It's not obvious that a circle and sticks should be a person. Same with a lot of other cartoon features that look nothing like reality.

Re: A Brief Overview of Deep Learning

Re: A Brief Overview of Deep Learning