51–60 of 62 posts
That Coursera class has been showing a start date of "Oct 1st 2012" for a while. Does anyone know when the next class might be?
Haven't heard anything about the next class starting, but I believe you can still sign up and download all the videos.
Neural Network techniques which work so well in small, easy and trivial datasets like MNIST do not generalize to more serious datasets, and that's where the "and this is where the magic happens" component is needed.
I've been working for several years as the "applications developer" for a neural net lab. The neural lab has spent 11 years developing and refining a neural net pipeline - a series of neural nets which given one or more photos of a person's face, the pipeline performs forensically accurate 3D Reconstructions of the person's face and head. The system is used by government & police agencies the world over when trying t…
MNIST is not a good dataset to show any artificial intelligence on. The dataset is so simple, a good programmer can probably write 100 lines of python to write a classifier for it, based on no machine learning. Neural Network techniques which work so well in small, easy and trivial datasets like MNIST do not generalize to more serious datasets, and that's where the "and this is where the magic happens" component is n…
Great post, and thank you for the link to Hinton's coursera page - I didn't know about that. I also hope to learn a thing or two from your github code. But it was so depressing to read this: > Now, when I say Artificial Intelligence I’m really only referring to Neural Networks. There are many other kinds of A.I. out there (e.g. Expert Systems, Classifiers and the like) but none of those store information like our bra…
Oh, backprop isn't so bad... After all, a deep belief network starts with an RBM for unsupervised pre-training, but the finetuning stage that follows just treats the network as a standard MLP using backprop. Also, you can use an autoencoder instead of an RBM, which I think are getting better results these days? And there are better regularization techniques for backprop now--weight decay, momentum, L1/L2 regularizati…
We could also pipe the raw data through an RBM and then slap a SVM or some other classifier on top.
Does anyone have a link to a web page (or to a book) that would be useful if you want to learn to program a Deep Belief Network?
It discusses how to choose a learning algorithm, selecting hyperparameters, number of hidden units, etc.
Really interesting stuff. I had once attempted to build a genetic algorithm for manipulating the synapse weights, specifically because of the problems of traditional back-propagation falling into local minima (unfortunately, some serious shit at work made it drop by the wayside). This RBM approach sounds better than back-propagation, but it also sounds like it would be prone to runaway feedback. One of the performanc…
Great post, and thank you for the link to Hinton's coursera page - I didn't know about that. I also hope to learn a thing or two from your github code. But it was so depressing to read this: > Now, when I say Artificial Intelligence I’m really only referring to Neural Networks. There are many other kinds of A.I. out there (e.g. Expert Systems, Classifiers and the like) but none of those store information like our bra…
By the way, is there a simple command line tool implementing RBM? Something easy to pick up and use.