Live data from Hacker News

Building high-level features using large scale unsupervised learning

research.google.com

131–140 of 196 posts

Re: Building high-level features using large scale unsupervised learning

#131
post #115

Earlier quoted context omitted.

You're correct: it isn't an unlabelled system, and the article author is deeply confused about basic topics in artificial intelligence. What he's trying to talk about is "this is an unsupervised feature detector in a large dataset which is only categorized, and where no human has provided correct answers up front to verify progress." The reason this matters (and it doesn't matter very much) is that that means that in…

The article authors of the paper? How can you say they are deeply confused - have you not seen their previous work and presentations? Everything else you say I agree with.

No. The author of the Y! Combinator article.

I apologize for being vague, and shall endeavor to be clearer in the future.

Re: Building high-level features using large scale unsupervised learning

#132
post #28

Earlier quoted context omitted.

It does this for 20,000 different objects categories - this is getting close to matching human visual ability (and there are huge societal implications if computer reach that standard). This is the most powerful AI experiment yet conducted (publicly known).

> It does this for 20,000 different objects categories With 15.8% accuracy. > This is the most powerful AI experiment yet conducted (publicly known). It's only powerful because they threw more cores at it than anyone else has previously attempted. From a quick skimming of the paper, there does not appear to be a lot of novel algorithmic contribution here. It's the same basic autoencoder that Hinton proposed years ago…

To a computer vision researcher, 15.8% on 20k categories is phenomenal.

Re: Building high-level features using large scale unsupervised learning

#135

Earlier quoted context omitted.

The early works in AI with regards to unsupervised learning were in the 1940s and 1950s. Claude Shannon had demonstrated a chess learning system which taught itself by playing him to defeat him in under two weeks as early as 1949. No, they weren't overstated. They were hyped by a clueless press. There's a pretty critical difference. It's a bit like how the early web pioneers didn't say that the web was going to revol…

I find both of your comments extremely condescending, both toward saalweachter and the authors of this article. 1. The fact that Claude Shannon succeeded in training a chess system has virtually no impact on sallweachter's claim that many AI results were overstated. 2. Certainly the press overstated them, which supports saalweachter's premise rather than weakening it. Even if the _implied_ claim was that _researchers…

"I find both of your comments extremely condescending"

When a comment opens with a tone like this, I usually don't bother to respond, but I'll give you a chance, because you seem to have done a lot of honest mis-reading.

To wit, it may be of value for you to inspect your own tone, if you find public condescention inappropriate.

.

"1. The fact that Claude Shannon succeeded in training a chess system has virtually no impact on sallweachter's claim that many AI results were overstated."

It wasn't meant to. Sallweatcher's claim was silly. Who cares if many things were overstated? That has zero bearing on that valid work was, in fact, being done.

The purpose of that statement was to remind us that as early as the 1940s, machine learning was able to defeat its own creator at what remains today regarded as a highly intellectual pursuit. My goal was to ignore the FUD of "some people got it wrong" as an attempt to suggest that there was nothing right.

Some people always get some of everything wrong. His claim is tautological and disinteresting. I was politely declining to shame him for it, but since you've presented me as having false goals, I now have no choice but to clarify.

It is generally inappropriate, for reasons like these, to chastize strangers over imagined motivations. Frequently, you don't know strangers' motivations as well as you might imagine from a simple read of a few paragraphs.

.

"2. Certainly the press overstated them, which supports saalweachter's premise"

You are now repeating something I said to me back to me. From that, you are deriving the false conclusion that because a journalist somewhere said something wrong, an important thing has been discovered.

What I'd like to point out is that the net result of observing that journalists made mistakes is still "so what?"

"Even if the _implied_ claim was that _researchers_ overstated results"

It isn't.

"your argument does nothing to weaken this claim."

You have not correctly identified what I was speaking to. This is akin to telling someone discussing environmental damage that some farmer is talking about crop yield and the speaker hasn't weakened their claim.

Again: so what? I never argued that there are journalists who got things wrong. I'm the one who brought it up.

What does that have to do with my original discussion?

.

"Frito Lay solved a problem several orders of magnitude easier that of face recognition in natural images"

Discovering defects in potatoes moving at 45 miles an hour inside a water sluice from a single blurry image from a single angle in hard realtime using 1970s hardware is not several orders of magnitude easier than locating things on a face in slow time on modern hardware.

It's actually quite a bit more difficult even in fair conditions. Potato defects are under the surface, and have to be located by subtle color variation. It is not hard to find the characteristic shape and shadow of the nose.

With respect, sir, it's quite clear that this is not something you've done. You're claiming that easy things are more difficult than hard things, and you're forgetting the 40 year technology gap inbetween in your rush to show that a 2012 project is more impressive than a 1973 project.

To be clear, Babbage's mechanical calculator is also more impressive than an algebra solving system made in prolog. Why? Because it's more work and it's more difficult.

Your claim of several orders of magnitude simpler suggests that you are inventing data for the sake of feeling correct in an argument, and that you do not actually have the experience to show correct guesses in this field. That, combined with a tone suggesting that you feel it appropriate to rebuke strangers in public, suggests that I don't really want to much talk to you anymore.

.

"Frito Lay example contributes nothing to your goal of weakening saalweachter's claim that this is valuable research"

Again, you've misidentified my goal, and the way by which you've done that is to drop a critical piece of his actual claim.

I don't know why you feel that it's okay to guess at people's goals, then tell people how morally wrong your guesses are. I really don't.

My actual goal was to point out the jarring unfamiliarity with the field that both he and you evidence:

"It's extremely valuable for someone to actually go and do a thing, now that we can, even if someone had the idea for the thing eons ago"

The thing I was focussing on was to show him that this thing that he's applauding someone for doing in 2012 for the first time now that it's practical, even though it isn't being used in industry, was actually outclassed by a much more difficult problem on much more limited hardware in realtime 40 years ago by a company that nobody would think of as a technology giant.

The goal was to display just how far out of touch saalweatcher was with the state of the industry.

Please don't speak to my goals anymore. For someone who'd like to speak about condescention (when I think you actually mean arrogance,) for you to tell me what I meant and what I was getting at - incorrectly - then lambast me for it in a tone far more severe than that which you're criticizing is, I admit, difficult to swallow politely.

.

"I'm sure your father (respected AI researcher of the same name) would make them too"

Do not speak for, or involve, my recently deceased father in your attempt to be correct, sir. Especially not while you're telling someone else they're being rude.

"I would a less condescending one that made better use of evidence than the argument you've made here."

Unfortunately, though you suggest this, taking a brief look through your comment shows that this is not in fact correct. You have been radically uglier than that which you are criticizing, involving personal attacks, false claims of other people's intent, false claims of other people's goals, and the repeat involvement of a recently deceased relative.

I would prefer not to hear from you again. Thanks.

Re: Building high-level features using large scale unsupervised learning

#136
post #28

Earlier quoted context omitted.

> It does this for 20,000 different objects categories With 15.8% accuracy. > This is the most powerful AI experiment yet conducted (publicly known). It's only powerful because they threw more cores at it than anyone else has previously attempted. From a quick skimming of the paper, there does not appear to be a lot of novel algorithmic contribution here. It's the same basic autoencoder that Hinton proposed years ago…

To a computer vision researcher, 15.8% on 20k categories is phenomenal.

[deleted]

Re: Building high-level features using large scale unsupervised learning

#137

Earlier quoted context omitted.

I'm his son. Let's take the example of The Netflix Prize, a $1 million bounty that the movie shipping organization ran several years ago. Their purpose was to improve their ratings prediction algorithm, under the pretext that people frequently ran out of ideas of what to rent, and that a successful suggestion algorithm would keep people as customers longer after that point. So, they carefully defined the success rate…

I appreciate your insight into the original article and your help placing it into the context of the broader field.

It's a kind claim, but I'm actually pretty much an outsider who dabbles. I haven't a clue what the state of the art is; Cyc is from the 1980s.

Re: Building high-level features using large scale unsupervised learning

#138
post #76

16,000 cores sounds impressive until you realize it's just five to ten modern GPUs. For Google, it's easier to just run a 1,000 machine job than requisition some GPUs. See: http://www.nvidia.com/object/tesla-servers.html (4.5 teraflops in one card ) Reminder: GPUs will destroy the world.

What a GPU calls a "core" doesn't at all correspond to what a CPU calls a "core". Going by the CPU definition (something like "something that can schedule memory accesses") a high end GPU will only have 60 or so cores. And going by the the GPU definition (An execution unit) a high end CPU will tend to have 30-something cores. GPUs do have fundamentally more execution resources, but that comes at a price and not every…

OpenCV has rewritten several of its algorithms for GPU's. http://www.opencv.org.cn/opencvdoc/2.3.2/html/modules/gpu/do... In general, the GPU versions are faster but you need to be cognizant of data transfer times between memory and the GPU. Relative speed also depends on which CPU and GPU's you have access to and the quality of the GPU vs CPU algorithm implementations. For example, my 2012 Macbook CPU is faster than my 2011 Macbook GPU for certain OpenCV algorithms.

Re: Building high-level features using large scale unsupervised learning

#139

Earlier quoted context omitted.

Let's not, because it's still wrong on the order of thirty counterexamples. Let's say we'll stop making broad proclamations about the global best in a field we know very little about.

Andrew Ng has set many state of the art results on various data sets using similar approaches as the one described in the paper. Here is a reasonably approachable talk he gave about it. http://www.youtube.com/watch?v=ZmNOAtZIgIk

This is a very interesting talk.

Thank you for sharing with me. :)

In return, I will offer you two interesting non-sequiturs, because I don't have anything topical and a non-sequitur seems like it's worth half what something germane would be.

.

Bret Victor, "Inventing on Principle." First 5 minutes are terribly boring. Give him a chance; it's 100% worth it.

http://vimeo.com/36579366

.

Damian Conway, "Temporally Quaquaversal Virtual Nanomachine."

It's as funny as it sounds.

http://yow.eventer.com/events/1004/talks/1028

Post reply on HN