Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building high-level features using large scale unsupervised learning

research.google.com

Post reply on HN
Print view View original

81–90 of 196 posts

Re: Building high-level features using large scale unsupervised learning

#81

Post by eric_bullington »

Homunculiheaded wrote: post #53

There was an interesting discussion on Quora about this recently[0] The most relevant quote being perhaps: "The magic of the brain is not the number of neurons, but how the circuits are wired and how they function dynamically. If you put 1 billion transistors together, you don't get a functioning CPU. And if you put 100 billion neurons together, you don't get an intelligent brain." 0. http://www.quora.com/How-big-is-…

That's an interesting discussion, but this experiment suggests exactly the opposite (perhaps that's why you included the discussion). Who knows, if we put 1 billion cores together, and fed it a massive amount of data (akin to what a baby receives as he/she matures), perhaps we would get a brain we would consider "intelligent". The fact that this system was able to pick out high-level features like "face" and "cat" without any prior training -- and with only 1000 cores, not 1 billion -- is quite suggestive that they're on to something.

EDIT: Mistyped number of cores. 1000, not 100.

Re: Building high-level features using large scale unsupervised learning

#82

Post by tonfa »

seiji wrote: post #76

16,000 cores sounds impressive until you realize it's just five to ten modern GPUs. For Google, it's easier to just run a 1,000 machine job than requisition some GPUs. See: http://www.nvidia.com/object/tesla-servers.html (4.5 teraflops in one card ) Reminder: GPUs will destroy the world.

I don't think that one CPU core is exactly comparable to one GPU core.

Re: Building high-level features using large scale unsupervised learning

#83

Post by ralfd »

p1esk wrote: post #50
sown wrote: post #41

I'm seriously considering quitting my job and studying ML for a few months in a desperate attempt to get work in projects like this. I feel like I'm missing out but too dumb for traditional grad school.

That's what I did a few months ago - quit my job and decided to go to a grad school to study AI (with focus on neural nets and ML).

What did you learn so far about neural nets? I recently looked into machine learning and naively thought I could find at least one practical fun tutorial "Here is a neural network API in C, you have to do that and this to let a simulated robot evade obstacles or learn to play Asteroids". Instead my (extremely superficial) search did find that neural nets are pretty arcane, genetic algorithms trapped in local minima and you are faster and better of coding logic yourself, developing a mathematical model to calculate results, instead searching for patterns in vast sets of data.

Re: Building high-level features using large scale unsupervised learning

#84

Post by dmoney »

reitzensteinm wrote: post #47
Smerity wrote: post #37

Earlier quoted context omitted.

I replied by adding to my comment above as it wouldn't allow me to reply earlier. Reference that. tldr; High-level features generated by this unsupervised network are so high-level that one of them aligns with "has a face in the image", another to "has cat in image" (see the section "Cat and human body detectors") and so on. Note however that they select the "best neuron" for face classification -- the only way they…

But the difference is that you can show it 1 billion unclassified images, then show it 1000 images you know to be faces, analyzing how its neurons respond to the known inputs to use it to classify the rest of the images. Strictly speaking, you do need to have some labeled data at the end in order to determine how the neural net views faces, but I think that obscures what's notable about this system. The amount of hum…

How does this work? I thought neural nets only learned when they got some kind of feedback that let them know whether what their classification was right (back propagation).

Re: Building high-level features using large scale unsupervised learning

#85

Post by DanBC »

manmal wrote: post #74
DanBC wrote: post #71

> the dataset has 10 million 200x200 pixel images downloaded from the Internet They take the frames from YouTube. It is weird to me that YouTube, (derided as a way of sharing funny cat videos) is able to contribute something actually useful to the world.

Youtube contains a lot of educational material you would not find otherwise. If I wanted to learn sewing on a sewing machine, I would just watch some video tutorials - try that with a book on sewing. Same thing with instructions on how to play instruments. Heck you can even watch videos on how to fix problems with your car engine. Many procedural instructions can't be transported properly via papers or books. It also…

I hope that Google look after it a bit better than they've looked after their Usenet archives. Google groups search is a particularly frustrating experience.

I agree that there is some great content on Youtube. Interesting that you mention sewing machines, because that's something I've used and they are particularly helpful. (See also all those other crafting videos; latch-hooking etc.)

Re: Building high-level features using large scale unsupervised learning

#87

Post by JohnHaugeland »

The singularity is a poorly constructed myth. It is built around the presumption that intelligence is a linear function of CPU power, and that surely as CPU power rises, so shall intelligence; the problem is, that prediction was made in the 1970s, since which CPU power has risen ten decimal orders of magnitude, and we still don't have much better speech recognition than we did back then, let alone anything even approaching simple reasoning.

The ability to detect faces is not a signal that general intelligence is right around the corner.

Re: Building high-level features using large scale unsupervised learning

#88

Post by Jabbles »

nl wrote: post #33

People are missing the point here. Yes, 15% accuracy doesn't seem great. BUT the detector built its own categories(!). It managed to find 20,000 different categories of objects in Youtube videos, and one of these categories corresponded to human faces, and another to cats. Once the experimenters found the "face detection neuron" and used it to test faces THAT neuron managed 81.7% detection rate(!). Forget the singula…

You're in danger of missing the point too far in the other direction. The system just returns yes/no as to whether an image has a face in it, and if it was hard-coded to respond "no" it would score 64.8%.

Obviously this is extremely impressive work, and given that Google gives away 1e9 core hours a year, I'd like to see how much further they can push this network (which only used 16e3x3x24 ~ 1e6 hours). But this isn't like scoring 80% in a written exam.

I'm also impressed by how readable the paper was. Apart from a few paragraphs of detailed maths this should be accessible to anyone who's read the wikipedia article on neural networks.

http://googleblog.blogspot.com/2011/04/1-billion-computing-c...

Re: Building high-level features using large scale unsupervised learning

#89

Post by JohnHaugeland »

reitzensteinm wrote: post #15
Smerity wrote: post #10

Earlier quoted context omitted.

I was going to make the same request. The singularity should be discussed where relevant, not added to everything. This paper is producing high level features from noisy data in an unsupervised fashion -- a human still needs to indicate the task it should be targeted for and a human still needs to provide labelled training data for these high level features to be of use. This work is interesting enough to warrant det…

Actually, what's significant about this work is that labeled training data was not required: "Contrary to what appears to be a widely-held intuition, our experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not."

Unsupervised training is not particularly significant, and was the original form of neural network.

"our experimental results reveal"

No, they don't. We had one of these in the 1980s.

Re: Building high-level features using large scale unsupervised learning

#90

Post by kalleboo »

ekianjo wrote: post #68
ehsanu1 wrote: post #63

Earlier quoted context omitted.

scotty79 means to showing a human only the 10 million 200x200 images. We probably see a lot more than that, in 3D from different perspectives with continuity in motion, and in a lot more detail and possibilities for filtering.

> scotty79 means to showing a human only the 10 million 200x200 images Then, what kind of thing are you measuring? Recognizing patterns ? We know humans are very good at that. We can see thousands of different people everyday but we can recognize in the blink of an eye a familiar face. A computer or computer network is very, very, very far from being able to do that yet.

But why are humans are very good at that? Could it be because humans have gotten tons of high-resolution, 3D imagery with binaural audio? Whereas this computer got a handful of low-res still images. Is the solution to just throw more horsepower at the problem, or is there really some inherent quality of the brain that's different?

edit: this quote puts things into perspective a bit "It is worth noting that our network is still tiny compared to the human visual cortex, which is 1,000,000 times larger in terms of the number of neurons and synapses."

Post reply on HN