Resurgence of Neural Networks
tjake.github.com
Resurgence of Neural Networks
1–10 of 62 posts
Re: Resurgence of Neural Networks
#2If you're really interested in understanding more about he "hierarchy of filters" quote, and much more related to that theory of how our brains operate, I strongly suggest the book On Intelligence by Jeff Hawkins. Super interesting stuff!
Re: Resurgence of Neural Networks
#3In fact we're only scratching the surface when it comes to the generative capabilities of deep models. See e.g. our recent work on using Deep Boltzmann Machines to learn how to draw object silhouettes: http://arkitus.com/ShapeBM/
Re: Resurgence of Neural Networks
#4[deleted]
Re: Resurgence of Neural Networks
#5Really interesting stuff.
I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback.
One of the performance problems with neural networks is that the number of cores on a typical machine are far less than the number of input and intermediate nodes in the network. The output nodes are less of a concern as you're trying to distill a lot of data down to a little data, but there is no reason to treat them differently. There are (very few) examples of NNs on GPUs, so that helps, but I've recently been curious to try a different, more hardware-driven approach, just because one could.
Texas Instruments has a cheap DSP chip that you'ns are probably familiar with called the MSP430. It's pretty easy to use, the tool chain is free and fairly easy to setup (especially for a bunch of professional software devs like us, right? right? Well, there's an Arduino-like tool now, too, if not), costs around 10 cents in bulk for the simplest version, requires very few external parts to run (something like a power source, 1 cap, and two resistors), and it has a couple of serial communication protocols built in. I'm quite fond of the chip; I've used it to build a number of digital synthesizers and toys.
For about $50 and quite a bit of soldering time, you could build a grid of 100 of these, each running at 16Mhz, and I bet with a clever design you could make them self programmable, i.e. propagate the program EEPROM over the grid. Load up a simple neural network program, maybe even having each chip simulating more than one node, and interface it with a PC to pump data in one end and draw it out the other. It might not be more useful than the GPGPU approach, but having physical hardware to play with and visualize node activity through other hardware would be a lot of fun.
Re: Resurgence of Neural Networks
#6Check out the rest of Geoffrey Hinton's work as well- http://scholr.ly/person/3595934/geoffrey-e-hinton
Re: Resurgence of Neural Networks
#7If you're really interested in understanding more about he "hierarchy of filters" quote, and much more related to that theory of how our brains operate, I strongly suggest the book On Intelligence by Jeff Hawkins. Super interesting stuff!
I cannot agree more, this is one of the most interesting books I have ever read.
Re: Resurgence of Neural Networks
#8I've been working for several years as the "applications developer" for a neural net lab. The neural lab has spent 11 years developing and refining a neural net pipeline - a series of neural nets which given one or more photos of a person's face, the pipeline performs forensically accurate 3D Reconstructions of the person's face and head. The system is used by government & police agencies the world over when trying to determine what a "person of interest" looks like given random photos of their subject. I've additionally exposed an "entertainment" version of the technology which can be seen at www.3d-avatar-store.com. There one can create a 3D avatar, get a Maya rigged version for professional quality animation, as well as license my WebAPI to embed avatar creation into your own software. And the best part is the avatars look just like the person in the source photo.
Re: Resurgence of Neural Networks
#9If you're really interested in understanding more about he "hierarchy of filters" quote, and much more related to that theory of how our brains operate, I strongly suggest the book On Intelligence by Jeff Hawkins. Super interesting stuff!
Here's a page that gives a high level overview of the technology that he has helped to develop:
https://www.numenta.com/technology.html
On Intelligence has dramatically changed the way I think about thinking. It's an awesome book.
Re: Resurgence of Neural Networks
#10[deleted]