Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building high-level features using large scale unsupervised learning

research.google.com

Post reply on HN
Print view View original

91–100 of 196 posts

Re: Building high-level features using large scale unsupervised learning

#91

Post by JohnHaugeland »

marshallp wrote: post #12
forgotusername wrote: post #6

Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly disconnected from machine and humanity merging.

It does this for 20,000 different objects categories - this is getting close to matching human visual ability (and there are huge societal implications if computer reach that standard). This is the most powerful AI experiment yet conducted (publicly known).

"It does this for 20,000 different objects categories - this is getting close to matching human visual ability"

No, it isn't. This classifier cannot identify theme variations, unknown rotations, will confuse new objects for objects it already knows, is unable to cope with camera distortion, needs fixed lighting, has no capacity for weather, does not work in the time you need to run away from a tiger, requires hundreds of times more data than a human eye presents, and does a far lower quality job, all while completely losing the ability to give a non-boolean response.

To say this is approaching human abilities is to have no idea what human abilities actually are.

"This is the most powerful AI experiment yet conducted"

No, it isn't. Please stop presenting your guesses as facts. Cyc runs circles around this, as do quite a few things from the Netflix challenge, as well as dozens of other things.

I personally have run far larger unsupervised neural networks than this, and I am not a cutting edge researcher.

Re: Building high-level features using large scale unsupervised learning

#92

Post by JohnHaugeland »

marshallp wrote: post #23
frisco wrote: post #21

Earlier quoted context omitted.

> This is the most powerful AI experiment yet conducted (publicly known). That's an ill-defined statement. AI is a vast and diverse field: what makes one demonstration more "powerful" than another? There are definitely other projects that could be viewed as being in the same class of "powerful" as this cluster. This is certainly an interesting paper, but it has to be viewed in the context of a large and active field.

That's true. Let's say in machine learning then.

Let's not, because it's still wrong on the order of thirty counterexamples.

Let's say we'll stop making broad proclamations about the global best in a field we know very little about.

Re: Building high-level features using large scale unsupervised learning

#93

Post by JohnHaugeland »

jpeterson wrote: post #5

Maybe I'm missing something here, but how exactly is it "unlabeled" data if they're specifically feeding it millions of pictures of "faces"? I mean, if you make a specific selection of the type of images you train the network on, isn't that basically equivalent to labeling them?

You're correct: it isn't an unlabelled system, and the article author is deeply confused about basic topics in artificial intelligence.

What he's trying to talk about is "this is an unsupervised feature detector in a large dataset which is only categorized, and where no human has provided correct answers up front to verify progress."

The reason this matters (and it doesn't matter very much) is that that means that in cases where it's prohibitive to provide training sets, such as where you don't know the good answer yourself, or where giving a decent range of good answers would be difficult, this sort of approach can still be used.

"isn't that basically equivalent to labeling them?"

Yes. It is. The original poster is confused.

What he meant to say was "there is no training set."

Re: Building high-level features using large scale unsupervised learning

#94

Post by jules »

seiji wrote: post #76

16,000 cores sounds impressive until you realize it's just five to ten modern GPUs. For Google, it's easier to just run a 1,000 machine job than requisition some GPUs. See: http://www.nvidia.com/object/tesla-servers.html (4.5 teraflops in one card ) Reminder: GPUs will destroy the world.

The problem is not the theoretical peak teraflops. The problem is actually achieving those teraflops with useful work. Due to architecture that is easier on a CPU than on a GPU, so you can't directly compare teraflops and conclude that GPUs are superior. Getting something to run fast on a GPU is very difficult.

And actually the thing that does 4.5 teraflops in single precision does only 95 gigaflops in double precision per GPU. A good x86 CPU does ~100 gigaflops in double precision as well, and you're much more likely to actually achieve that number on a x86. Although another one on the page you linked to theoretically does 665 gigaflops double precision.

Re: Building high-level features using large scale unsupervised learning

#95

Post by nik61 »

Chlorus wrote: post #42
marshallp wrote: post #32

Earlier quoted context omitted.

I put the singularity bit in to make it relevant for those who are non-technical. This experiment is significant because it shows that large artificial neural networks can be made to work. People have tried and failed at this for decades. This technigue was "discovered" by geoff hinton at the university of toronto in 2005. However, nobody at tried (or maybe got enough funds) to try it this scale. If this continues to…

"I put the singularity bit in to make it relevant for those who are non-technical." Yeah, I'm sure there's a lot of those on HN...I'd expect this kind of crap in something like Wired, but not here.

This is a Troll response, and gratuitously insulting. HN readers may be broadly technical (or not, who knows?) but few of them have the insight into AI provided here, especially if they got their IT experience or education during the AI Winter referenced above.

Re: Building high-level features using large scale unsupervised learning

#96

Post by Smerity »

dmoney wrote: post #84
reitzensteinm wrote: post #47

Earlier quoted context omitted.

But the difference is that you can show it 1 billion unclassified images, then show it 1000 images you know to be faces, analyzing how its neurons respond to the known inputs to use it to classify the rest of the images. Strictly speaking, you do need to have some labeled data at the end in order to determine how the neural net views faces, but I think that obscures what's notable about this system. The amount of hum…

How does this work? I thought neural nets only learned when they got some kind of feedback that let them know whether what their classification was right (back propagation).

The neural network in this paper, an autoencoder, doesn't require labelled data.

Autoencoders take high dimensional input, map it to a lower dimensional space and then try to recreate the original high dimensional input as closely as possible. The idea is to learn a compressed representation for the data and hope that this compressed representation works as a high level featureset.

As the model is just trying to represent the original input, no labelled data is required for the initial part. Labelled data is later introduced when the high level features are used for classification. What's most interesting about this paper is that one of the features learned by the model maps quite well to "image contains a face" without any prompting by the researchers.

For more details, check out http://www.stanford.edu/class/cs294a/sparseAutoencoder.pdf

Re: Building high-level features using large scale unsupervised learning

#97

Post by jules »

roel_v wrote: post #62
sown wrote: post #41

I'm seriously considering quitting my job and studying ML for a few months in a desperate attempt to get work in projects like this. I feel like I'm missing out but too dumb for traditional grad school.

"and studying ML for a few months" Could anyone with expertise say if this would be enough to build a foundation? How much math background do you need?

You need at least probability/statistics, linear algebra, calculus, and numerical methods. Once you know this machine learning is a relatively thin layer built on this math foundation. The real problem is learning the math foundation.

Re: Building high-level features using large scale unsupervised learning

#98

Post by mikedmiked »

ralfd wrote: post #83
p1esk wrote: post #50

Earlier quoted context omitted.

That's what I did a few months ago - quit my job and decided to go to a grad school to study AI (with focus on neural nets and ML).

What did you learn so far about neural nets? I recently looked into machine learning and naively thought I could find at least one practical fun tutorial "Here is a neural network API in C, you have to do that and this to let a simulated robot evade obstacles or learn to play Asteroids". Instead my (extremely superficial) search did find that neural nets are pretty arcane, genetic algorithms trapped in local minima a…

try python's scikit.learn library

Re: Building high-level features using large scale unsupervised learning

#99

Post by ippisl »

ralfd wrote: post #83
p1esk wrote: post #50

Earlier quoted context omitted.

That's what I did a few months ago - quit my job and decided to go to a grad school to study AI (with focus on neural nets and ML).

What did you learn so far about neural nets? I recently looked into machine learning and naively thought I could find at least one practical fun tutorial "Here is a neural network API in C, you have to do that and this to let a simulated robot evade obstacles or learn to play Asteroids". Instead my (extremely superficial) search did find that neural nets are pretty arcane, genetic algorithms trapped in local minima a…

Here's a fun toy project using neural networks.

http://slowping.com/2012/self-driving-lego-mindstorms-robot/

Re: Building high-level features using large scale unsupervised learning

#100

Post by jules »

marshallp wrote: post #19
Cyranix wrote: post #13

Earlier quoted context omitted.

"Singularity is near"? "Holy grail"? You may be getting a little carried away here. The outcome shows a very nice improvement on an unsupervised classification and feature detection task, but it also highlights that unsupervised machine learning still has a long way to go. 16% accuracy from a network with 1bn connections and 100m inputs using (if my math is right) 1.15m hours of CPU time. Which of these would be the…

If the algorithm keeps increasing in accuracy as you scale up computation and add more unlabeled data that is pretty amazing. You might get something that matches human performance on vision/speech recognition etc.

If you extrapolate that way you'd conclude that naive Bayes is the solution to AI. Improvements tend to tail off fairly quickly as you add more data and computation, unfortunately.
Post reply on HN