Building high-level features using large scale unsupervised learning • Hacker News Board

Re: Building high-level features using large scale unsupervised learning

#181

Post by sellsword » Sat, Jun 23, 2012, 4:36 AM UTC

Smerity wrote: post #73

reitzensteinm wrote: post #47

Earlier quoted context omitted.

But the difference is that you can show it 1 billion unclassified images, then show it 1000 images you know to be faces, analyzing how its neurons respond to the known inputs to use it to classify the rest of the images. Strictly speaking, you do need to have some labeled data at the end in order to determine how the neural net views faces, but I think that obscures what's notable about this system. The amount of hum…

In a more general response: I don't think what I stated obscures what's notable about the system, I feel I stated exactly what was notable and specifically avoided overstating it. Overhyping when it comes to machine learning and AI seems to be the norm and has already hurt AI/ML severely in the past[1]. More specifically: I didn't disagree with anything you've stated, simply pointed out that labeled training data is…

Having a bachelor's emphasis in AI, I think you described it perfectly. I was wondering too from their abstract how they were recognizing "faces" entirely without labels, this makes it clearer. As you said, unsupervised they can find extremely high-level categories. That is pretty impressive.

Re: Building high-level features using large scale unsupervised learning

#182

Post by scotty79 » Sat, Jun 23, 2012, 7:50 AM UTC

Dn_Ab wrote: post #123

scotty79 wrote: post #114

Earlier quoted context omitted.

I agree that CV is currently behind human sight. I just argue that this is more due to lacking input processing than to actual classification engine. The reason they used 200x200 px images, I think, was that bigger images couldn't be analysed in sufficient quantity. In a way their first input processing of the real images was pixelisation filter. If you feed pixelised image to a person you see how much information is…

>> To improve CV we should focus on finding best ways of converting full resolution visual data to something of smaller volume in such way that important features are preserved. You are exactly right! See Dictionary Learning, Random projection, compressive sensing. As for time, perhaps you are right I don't know. That question is: would a suitably written video trained classifier that preserved temporal features do b…

I think that it might be the case as I believe that there is a lot of information in movement (or in 3d that is made observable thanks to movement) than makes dividing objects into categories much easier task. There must be a lot of shortcuts that can be made thanks to the fact that objects are not something completely arbitrary but physical, usually solids and have to obey relatively simple rules of reality. Once this classifier is taught with reach data you can match newly observed object to one of the classes, relying only on very small amount of information, for example its 2d small resolution image.

Seeing is in my opinion very similar to understanding language in sense that the information that is transferred, observed image or words heard are just small fuzzy fragments. Sender (speaker, or in case of vision, physical world) has rich model and recipient has rich model of all the things they can communicate about and the actual information passed, only indicates the parts of the underlying model to the recipient, that he should select and how he should modify them to get the message.

Building usable model from small fuzzy fragments of information that are passed when recognizing image or hearing spoken words should be incredibly hard task and I think no biological brain could do that. I think that absorbing as much real information as possible at the time of training the classifier is absolutely crucial for achieving anything close to what humans or animals can do.

From the techniques you mentioned, dictionary learning looks most awesome to me, and most applicable to CV.

Re: Building high-level features using large scale unsupervised learning

#183

Post by agravier » Sat, Jun 23, 2012, 9:45 AM UTC

snikolov wrote: post #176

agravier wrote: post #43

Earlier quoted context omitted.

> BUT the detector built its own categories(!). It's not revolutionary. Clustering algorithms and neural nets are plenty. Really, what differentiates this network is its scale.

Clustering algorithms operate on features, which typically have to be designed by hand. The appeal of deep learning is that it discovers good features automatically.

Feature sensitivity is typically hand-crafted only because it's the practical thing to do. Neural nets can easily learn visual features. See the LISSOM neural nets for a good example of self-organized learning of features.

Re: Building high-level features using large scale unsupervised learning

#184

Post by mbq » Sat, Jun 23, 2012, 11:57 AM UTC

moultano wrote: post #174

mbq wrote: post #79

As far as I can tell, this is let's train a huge number of models and then cherry-pick few that works well on a test set, so an overfitted junk. What have I missed?

It seems like they are cherry picking the one that works well on the training set no? Where did you get the sense that they were doing it on test?

The train is not labeled, so it is not possible; and they do not mention that the labeled set was split or used in validation -- it is just called "test".

Re: Building high-level features using large scale unsupervised learning

#185

Post by campnic » Sat, Jun 23, 2012, 12:01 PM UTC

JohnHaugeland wrote: post #137

campnic wrote: post #121

Earlier quoted context omitted.

I appreciate your insight into the original article and your help placing it into the context of the broader field.

It's a kind claim, but I'm actually pretty much an outsider who dabbles. I haven't a clue what the state of the art is; Cyc is from the 1980s.

Ha' then I'm appreciating your honesty :-)

Re: Building high-level features using large scale unsupervised learning

#186

Post by moultano » Sat, Jun 23, 2012, 10:25 PM UTC

mbq wrote: post #184

moultano wrote: post #174

Earlier quoted context omitted.

It seems like they are cherry picking the one that works well on the training set no? Where did you get the sense that they were doing it on test?

The train is not labeled, so it is not possible; and they do not mention that the labeled set was split or used in validation -- it is just called "test".

"We followed the experimental protocols speciﬁed by (Deng et al., 2010; Sanchez & Perronnin, 2011), in which, the datasets are randomly split into two halves for training and validation. We report the performance on the validation set and compare against state-of-theart baselines in Table 2. Note that the splits are not identical to previous work but validation set performances vary slightly across diﬀerent splits."

Re: Building high-level features using large scale unsupervised learning

#187

Post by AeaeaActual » Sat, Jun 23, 2012, 10:40 PM UTC

JohnHaugeland wrote: post #145

Symmetry wrote: post #116

Earlier quoted context omitted.

Well, maybe. There are a whole lot of very different things called "The Singularity" and some of them are much more reasonable than others. There's the Cambpellian Singularity, which says that we won't be able to predict what will happen next. Pretty non-controversial as far as it goes. There's the Vingean Singularity, which says that if we ever develop AIs that can think as fast and as well as humans then due to Moo…

"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…

"Whose work do they all point to? Moore, Lanier, Holland, and Hawkins. Guess what all four of them say they think?"

Here are some pretty uninformed statements by tech luminaries: http://spectrum.ieee.org/computing/hardware/tech-luminaries-...

They mostly don't have the philosophical or synthetic chops to make intelligent statements about the singularity. Moore misunderstood his own observation for at least the first ten years after he made it, not to minimize his important contributions.

Singularity is an unfortunate term, because it's technically incorrect and logically contradictory. My work is not trying to make a singularity; it is trying to make recursively self-improving machine-human intelligence which interacts with and learns from its environment. This is not impossible; it is merely technically difficult.

It is also a hypothesis. That's all. It isn't a phenomenon, so we cannot yet be good Aristoteleans and observe it. Therefore, we can't develop a science of it. You'd avoid this entire webpage's worth of argument if you'd just simply remember that the singularity is nothing more than this hypothesis.

Repeat: The singularity is not a phenomenon, nor is it a theory; it is a hypothesis.

For those who wish to believe it is a correct hypothesis, and it is the future, well, get busy doing the hard work and developing the the technology to make it the future. For those who don't believe it could be a possible future, get out of our way, since you're so damn sure you're right.

Re: Building high-level features using large scale unsupervised learning

#188

Post by bfrs » Sun, Jun 24, 2012, 3:22 AM UTC

ekianjo wrote: post #25

15.8% accuracy != singularity is near. No more supplements-eating Kurzweil, walking Terminators and Skynet-like BS please.

"supplements-eating Kurzweil", ROTFL!

Re: Building high-level features using large scale unsupervised learning

#189

Post by bfrs » Sun, Jun 24, 2012, 3:56 AM UTC

karpathy wrote: post #171

If anyone is interested to read more on this topic, there is another recent, closely related and perhaps slightly more accessible paper ( "High-Level Invariant Features with Scalable Clustering Algorithms" http://bit.ly/KDuN04 ) from Stanford that also learns face neurons from unsupervised collection of images (disclaimer: I'm co-author). It uses a slightly different model based on layers of k-means clustering and li…

I think your comment http://news.ycombinator.com/item?id=3838971 is very relevant to this discussion.

Re: Building high-level features using large scale unsupervised learning

#190

Post by JohnHaugeland » Sun, Jun 24, 2012, 11:55 AM UTC

MaysonL wrote: post #180

JohnHaugeland wrote: post #120

Earlier quoted context omitted.

The early works in AI with regards to unsupervised learning were in the 1940s and 1950s. Claude Shannon had demonstrated a chess learning system which taught itself by playing him to defeat him in under two weeks as early as 1949. No, they weren't overstated. They were hyped by a clueless press. There's a pretty critical difference. It's a bit like how the early web pioneers didn't say that the web was going to revol…

Claude Shannon had demonstrated a chess learning system which taught itself by playing him to defeat him in under two weeks as early as 1949. I call "citation needed", and needed quite badly: Shannon was a much better chess player than any program available in 1949.

I have no interest in your presenting your unwillingness to do basic research as if it was a valid form of skepticism.

Whether or not you believe me, everyone else just went ahead and took a quick look, and learned something.

Frankly, I would be happier, given your seeming inability to be a part of this conversation in a polite way, yet also your seeming unwillingness to depart this conversation even after it was requested, that you actually believe I'm wrong, and go around "calling people on this," so that everyone has early warning just how much you actually know about this field, instead of having to wait to listen to you speak.

"Shannon was a much better chess player than any program available in 1949."

On a technicality, this is correct: he started his work on December 29, and it wasn't until five days later, January 2 of 1950, that it was able to beat him.

All the same, you have no idea what you're talking about, and are asserting your beliefs as fact.

The correct way to handle "that doesn't sound right" is a search engine, not putting your hands on your hips and telling someone they're wrong in public.

Go check now, little bird.