Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building high-level features using large scale unsupervised learning

research.google.com

Post reply on HN
Print view View original

11–20 of 196 posts

Re: Building high-level features using large scale unsupervised learning

#11

Post by jfoutz »

chubot wrote: post #4
marshallp wrote: post #2

Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches). This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can…

I only read the abstract, so I'm sure this is a basic/dumb question... but if you don't label images as faces or not, what makes it a face detector? :) How do you get an elbow detector or a butt detector out of the same algorithm?

Show it a zillion pictures. then show it a face and see what gets activated. that's your face detector. show it an elbow or butt, and see what gets activated, that's your elbow or butt detector.

Re: Building high-level features using large scale unsupervised learning

#12

Post by marshallp »

forgotusername wrote: post #6

Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly disconnected from machine and humanity merging.

It does this for 20,000 different objects categories - this is getting close to matching human visual ability (and there are huge societal implications if computer reach that standard).

This is the most powerful AI experiment yet conducted (publicly known).

Re: Building high-level features using large scale unsupervised learning

#13

Post by Cyranix »

marshallp wrote: post #2

Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches). This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can…

"Singularity is near"? "Holy grail"? You may be getting a little carried away here.

The outcome shows a very nice improvement on an unsupervised classification and feature detection task, but it also highlights that unsupervised machine learning still has a long way to go. 16% accuracy from a network with 1bn connections and 100m inputs using (if my math is right) 1.15m hours of CPU time. Which of these would be the easiest way to continue making gains: investing more time/hardware, increasing the complexity of the model, or developing a new and improved algorithm altogether? All of these sound pretty intensive to me.

Re: Building high-level features using large scale unsupervised learning

#14

Post by marshallp »

kenrikm wrote: post #7

While I'm excited about progress, 15.8% accuracy is not exactly "Singularity is near"

They just have to scale it up - more computers, more days, and the accuracy level should increase accordingly (that is my intuition and hope on this, though i could be wrong).

Re: Building high-level features using large scale unsupervised learning

#15

Post by reitzensteinm »

Smerity wrote: post #10
forgotusername wrote: post #6

Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly disconnected from machine and humanity merging.

I was going to make the same request. The singularity should be discussed where relevant, not added to everything. This paper is producing high level features from noisy data in an unsupervised fashion -- a human still needs to indicate the task it should be targeted for and a human still needs to provide labelled training data for these high level features to be of use. This work is interesting enough to warrant det…

Actually, what's significant about this work is that labeled training data was not required:

"Contrary to what appears to be a widely-held intuition, our experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not."

Re: Building high-level features using large scale unsupervised learning

#16

Post by ADRIANFR »

chrischen wrote: post #3
marshallp wrote: post #2

Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches). This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can…

Wait until they become conscious and demand human rights. Then we can no longer exploit them and we're back to square one.

Seriously, a Phd thesis not far from now may have the title: "The limits of AI: how far can we exploit the machines before we are limited by machine rights"

Re: Building high-level features using large scale unsupervised learning

#17

Post by Smerity »

jpeterson wrote: post #5

Maybe I'm missing something here, but how exactly is it "unlabeled" data if they're specifically feeding it millions of pictures of "faces"? I mean, if you make a specific selection of the type of images you train the network on, isn't that basically equivalent to labeling them?

The aim of the paper was to produce an unsupervised system that would generate high level features from noisy data. These high level features could then be used in supervised systems where labelled data is added.

Thus, the paper is about using an unsupervised system to help a later supervised system. An advantage of this is that, as the unsupervised system isn't trained to recognise object X, it instead learns features that are discriminative. This same network could be used to recognise arbitrary objects (which is what they do later on in the paper with ImageNet).

Re: Building high-level features using large scale unsupervised learning

#18

Post by heretohelp »

marshallp wrote: post #14
kenrikm wrote: post #7

While I'm excited about progress, 15.8% accuracy is not exactly "Singularity is near"

They just have to scale it up - more computers, more days, and the accuracy level should increase accordingly (that is my intuition and hope on this, though i could be wrong).

Haha, no.

--- last company was in computer vision.

Re: Building high-level features using large scale unsupervised learning

#19

Post by marshallp »

Cyranix wrote: post #13
marshallp wrote: post #2

Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches). This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can…

"Singularity is near"? "Holy grail"? You may be getting a little carried away here. The outcome shows a very nice improvement on an unsupervised classification and feature detection task, but it also highlights that unsupervised machine learning still has a long way to go. 16% accuracy from a network with 1bn connections and 100m inputs using (if my math is right) 1.15m hours of CPU time. Which of these would be the…

If the algorithm keeps increasing in accuracy as you scale up computation and add more unlabeled data that is pretty amazing. You might get something that matches human performance on vision/speech recognition etc.
Post reply on HN