Building high-level features using large scale unsupervised learning

Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches).

This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can be applied to any data type (financial data, text, audio, images/video) without any human involvement (except gatheting of unlabelled data and running the system). Pretty much the artificial intelligence holy grail!

I only read the abstract, so I'm sure this is a basic/dumb question... but if you don't label images as faces or not, what makes it a face detector? :) How do you get an elbow detector or a butt detector out of the same algorithm?

Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly disconnected from machine and humanity merging.

It automatically creates a set features that you can then use a final layer of machine learning to get what you want.

In machine learning, normally you have to create a set of features (call feature engineering - basically think algorithms to better represent your data). The amazing thing about deep learning is that the computer does this for you!

You just need a few 10s/100s face/nonface images - same for 20,000 other objects - this is called fine-tuning.

For more, andrew ng, geoff hinton, yann lecun have given talks on this at google and they are up on youtube.

I was going to make the same request. The singularity should be discussed where relevant, not added to everything. This paper is producing high level features from noisy data in an unsupervised fashion -- a human still needs to indicate the task it should be targeted for and a human still needs to provide labelled training data for these high level features to be of use.

This work is interesting enough to warrant detailed discussion on the topic at hand, large scale machine learning, rather than just rehashing discussions of the singularity.

Added: As I can't reply to the comment below I'll do it here =] The network provides learned representations that are discriminative. The aim of the network is to learn high level features representative of the content. One of the many features it produced was one which accurately indicated the presence of a face in the image. Note that they said train a face detector and not classify. For example, from the same network there was a feature which accurate detected cats yet they didn't explicitly train a cat detector either (see the section "Cat and human body detectors"). As the network represents the content as generic features it is clear that, if it reaches a high enough level, those features are essentially classifications themselves.

tldr; High-level features generated by this unsupervised network are so high-level that one of them aligns with "has a face in the image", others with "has cat in image", etc, but these features cannot be used without labelled training.