11–20 of 62 posts
Here's a very good tech talk from him about RBMs: http://www.youtube.com/watch?v=AyzOUbkUf3M
That said, both approaches loosely mirror the function of the brain, as neurons are not simple threshold devices, and both backpropagation and the RBMs training algorithms do not have a biophysical equivalent.
Really interesting stuff. I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback. One of the performanc…
First, it's Geoffrey, not Gregory Hinton. Here's a very good tech talk from him about RBMs: http://www.youtube.com/watch?v=AyzOUbkUf3M That said, both approaches loosely mirror the function of the brain, as neurons are not simple threshold devices, and both backpropagation and the RBMs training algorithms do not have a biophysical equivalent.
Really interesting stuff. I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback. One of the performanc…
PS: Still a fun project, just harder than you might think to scale.
Really interesting stuff. I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback. One of the performanc…
There are many ways to avoid this, for example have a look at:
http://en.wikipedia.org/wiki/Rprop http://en.wikipedia.org/wiki/Conjugate_gradient_method
In traditional (I am not talking about the "deep" stuff) neural networks optimization is hardly ever the problem though, most often under- or over- fitting is the issue that produces poor performance.
One of the performance problems with neural networks is that the number of cores on a typical machine are far less than the number of input and intermediate nodes in the network.
This sounds odd. Certainly doing neural networks in hardware is interesting, but it sounds a bit like an imagined problem, I mean, when multiplying 20 numbers one does not complain that the number of cores is less than 20. And most often it is the training of the network that is resource-intensive not the actual running of it.
Really interesting stuff. I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback. One of the performanc…
If you're really interested in understanding more about he "hierarchy of filters" quote, and much more related to that theory of how our brains operate, I strongly suggest the book On Intelligence by Jeff Hawkins. Super interesting stuff!
Here's a page that gives a high level overview of the technology that he has helped to develop: https://www.numenta.com/technology.html On Intelligence has dramatically changed the way I think about thinking. It's an awesome book.
Now, it looks like those Vitamin D people have their own company: http://www.vitamindinc.com/
Even the really early versions of Vitamin D were impressive. Anybody use it for anything interesting now?
If you're really interested in understanding more about he "hierarchy of filters" quote, and much more related to that theory of how our brains operate, I strongly suggest the book On Intelligence by Jeff Hawkins. Super interesting stuff!
I cannot agree more, this is one of the most interesting books I have ever read.
Really interesting stuff. I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback. One of the performanc…
It sounds almost like you want to recreate a Connection Machine [0] using modern(ish) hardware. 0: http://en.wikipedia.org/wiki/Connection_Machine