Is AI Riding a One-Trick Pony? • Hacker News Board

Re: Is AI Riding a One-Trick Pony?

#191

Post by Retric » Sat, Sep 30, 2017, 1:53 PM UTC

soVeryTired wrote: post #186

Retric wrote: post #178

Earlier quoted context omitted.

Deep learning does not seem to abstract very well. Train on a data set then test with images that are simply upside down and the preformance can be significant. Feature extraction also works much better when you toss a lot of data and processing power behind it. So, a lot of progress is simply more data and computing power vs better approaches. Consider how poorly deep leaning works when using a single 286.

> Deep learning does not seem to abstract very well. Train on a data set then test with images that are simply upside down and the preformance can be significant. But that's true of people too. How quickly can you read upside-down? If you trained on a mixture of upside-down and right way up images, and tested on upside-down images, performance wouldn't take that much of a hit.

> But that's true of people too.

Sure, the problem is we are more willing to ignore failures that are similar to how we fail. IMO, when we compare AI approach X vs. Y we need to consider absolute performance not just performance similar to human performance.

Deep learning for example gains a lot from texture detection in images. But, that also makes it really easy to fool.

Re: Is AI Riding a One-Trick Pony?

#192

Post by mannykannot » Sat, Sep 30, 2017, 2:25 PM UTC

aroman wrote: post #134

mannykannot wrote: post #84

Earlier quoted context omitted.

Human perception and intelligence are by no means infallible, but neither are they anywhere near being as brittle as current AI. The thing about illusions is that we generally know that we are being subject to an illusion, and we also usually have the depth of understanding to know when we don't understand something about what we are seeing or think we have heard, and we have the depth of understanding to think of ac…

> The thing about illusions is that we generally know that we are being subject to an illusion. Citation needed? I did my undergrad in cognitive science, and while my knowledge of illusions is very limited, I never came across anything to suggest that we have an innate awareness of when our perceptual system is being tricked.

Frankly, when I first saw your post, I thought I was being trolled, especially as the phrase 'citation needed' is over-used, frequently in an attempt to avoid the burden of proof. For a citation, there's hyperbovine's reply. On reflection, however, I think your post raises a reasonable question.

Firstly, whether it is innate, learned or some combination, all are equally valid here.

In general, we cannot know if we are being deceived by our senses, and if you follow this line of argument to its end, you reach solipsism. With regard to the illusions of the sort presented in the linked article, they seem to fall into three types. There are the ones where we are immediately aware of being subject to an illusion; this is especially true in the cases where there is apparent motion. There are some where we do not notice unless we investigate further or have our attention brought to it, such as those involving apparent differences in brightness or color. Then there are those that actually depend on us noticing that there is an illusion - Necker cubes, for example.

In real life, when faced with an ambiguous input from our senses, we are often aware the fact because of the dissonance with our general understanding of the world, and we are usually able to take actions specifically designed to resolve the ambiguity. In contrast, AI can be very confident about the most ridiculous conclusions.

So we don't infallibly know when we are being tricked, but even in the cases where we are, further investigation often reveals that what is going on. In contrast, has any AI ever demonstrated any understanding of the concept of an illusion? The fact that we can sometimes be tricked by illusions for a while does not imply that AI has reached parity with humans in this regard, or that the fragility of image recognition is not an issue.

Re: Is AI Riding a One-Trick Pony?

#193

Post by mustacheemperor » Sat, Sep 30, 2017, 2:29 PM UTC

mannykannot wrote: post #84

philipkglass wrote: post #20

I'm encouraged that so much fruitful work has come out of this one trick. If you can use the same basic framework for image labeling, playing Go, and translating natural languages, I'd say it's a powerful tool with broad applications. I think that there's a kernel of insight to "A real intelligence doesn’t break when you slightly change the problem." But human perception and intelligence are pretty brittle. The metho…

Human perception and intelligence are by no means infallible, but neither are they anywhere near being as brittle as current AI. The thing about illusions is that we generally know that we are being subject to an illusion, and we also usually have the depth of understanding to know when we don't understand something about what we are seeing or think we have heard, and we have the depth of understanding to think of ac…

>we generally know that we are being subject to an illusion,

Is this an inherent power or a result of those methodological and institutional systems that reduce our cognitive brittleness? The fata morgana is an illusion, but even today people see it and think it's a ghost ship, something that logically is completely and literally impossible. I'm not suggesting AI is on equal footing with humanity yet, but I think the comparison of these limitations is valid.

Re: Is AI Riding a One-Trick Pony?

#194

Post by gilbetron » Sat, Sep 30, 2017, 2:46 PM UTC

karpathy wrote: post #132

There is a bit of "can't see the forest for the trees" failure in the article. AI is spearheading a paradigm shift in how we write programs. Or rather, we don't write programs. We write much much shorter programs that search the program space for programs that satisfy some desiderata. The programs we get as the output of the search process are extremely flexible, work very well, are very homogeneous in compute (e.g.…

I'm trying to get the company I work for used to the idea of ML in general, and one thing I get push back on is that we don't understand what such system are doing. Beyond showing them the math, I also point out to some of our bug reports that end up spanning thousands of lines, with input by our best, most experienced people, and that how often the bug "fix" is of the "well, it works now" type, and that we actually don't really understand what our system is doing. Of course the most common response back is a bunch of "yeah, but"s :) I'm making progress, but it is slow going.

I've been steeped in code for nearly 40 years now, I'm ok with the fact that ML lets me step back a bit from tabs, semicolons, and objects!

Re: Is AI Riding a One-Trick Pony?

#195

Post by QAPereo » Sat, Sep 30, 2017, 2:57 PM UTC

mikeash wrote: post #55

QAPereo wrote: post #28

Earlier quoted context omitted.

Of course, we're the product of an evolutionary history which results in such human "failure modes" being rare. If staring at a zebra made you hallucinate, you'd be unlikely to be the most successful member of your species, nor would your offspring thrive. So while we only tend to run into our obvious failing whens we do the unusual, computers fail at what we consider mundane.

The other day I was walking out of my closet, turned, and nearly jumped out of my skin because some clothes hanging from the door briefly looked like a large man standing right next to me. I'm not sure that our failings only happen under unusual circumstances, but rather maybe we're just used to them and don't think about it much.

That's no failing, that's working as intended. It's far more beneficial for us (and more importantly, our successful ancestors) to be extremely wary of potential threats at the level of near-reflex. It's also important not to waste a bunch of energy running from phantoms. So you did something no computer today could; you had an instinctive reaction, which was then moderated by increasingly higher levels of reasoning. I'm guessing the whole process of panic->resolution took less than a few seconds.

That's no failure mode.

Re: Is AI Riding a One-Trick Pony?

#196

Post by Demiurge » Sat, Sep 30, 2017, 3:01 PM UTC

nesyt wrote: post #149

markan wrote: post #101

Earlier quoted context omitted.

Great quote from Hinton. The biggest deficiency in AI is that we still don't have artificial systems which simulate human thought with any fidelity. Sooner or later that's bound to become a focus of attention.

Why bother simulating human thought? It's not the only road to Rome.

It's probably more of a map than a road.

Re: Is AI Riding a One-Trick Pony?

#197

Post by abecedarius » Sat, Sep 30, 2017, 3:10 PM UTC

aws_ls wrote: post #188

karpathy wrote: post #132

There is a bit of "can't see the forest for the trees" failure in the article. AI is spearheading a paradigm shift in how we write programs. Or rather, we don't write programs. We write much much shorter programs that search the program space for programs that satisfy some desiderata. The programs we get as the output of the search process are extremely flexible, work very well, are very homogeneous in compute (e.g.…

>So sure, backprop (the credit assignment scheme that gives us a good search direction in program space, one of multiple techniques that could do so) is pervasive, but AI is starting to work primarily as a result of a deeper epiphany - that we are not very good at all at writing code. Isn't it applicable to a class of programs only? Best example of which is Computer Vision. Or do you imply your argument to hold for a…

Daniel Hillis wrote a nice paper on machine-learned sorting networks decades ago. I think they were for fixed-size inputs, but the good news is that you can validate them using a 0-1 principle.

Re: Is AI Riding a One-Trick Pony?

#198

Post by lanstin » Sat, Sep 30, 2017, 3:19 PM UTC

hyperion2010 wrote: post #61

I think that the idea that learning is what was missing from the prior generation of AI is the most important insight of this generation. There are many things that we don't know how to implement from first principals but that can be implemented by a system that can learn. The problem now is that the substrates for learning are extremely low level, practically the raw inputs to the retina or pure symbols. In order to…

Not just evolution but also infancy.

Re: Is AI Riding a One-Trick Pony?

#199

Post by mannykannot » Sat, Sep 30, 2017, 3:20 PM UTC

mustacheemperor wrote: post #193

mannykannot wrote: post #84

Earlier quoted context omitted.

Human perception and intelligence are by no means infallible, but neither are they anywhere near being as brittle as current AI. The thing about illusions is that we generally know that we are being subject to an illusion, and we also usually have the depth of understanding to know when we don't understand something about what we are seeing or think we have heard, and we have the depth of understanding to think of ac…

>we generally know that we are being subject to an illusion, Is this an inherent power or a result of those methodological and institutional systems that reduce our cognitive brittleness? The fata morgana is an illusion, but even today people see it and think it's a ghost ship, something that logically is completely and literally impossible. I'm not suggesting AI is on equal footing with humanity yet, but I think the…

Many illusions - especially those involving color or apparent motion - are consequences of relatively low-level signal processing. Putting those aside, what you call 'methodological and institutional systems' I think of as 'understanding'. It feels to me that how I make sense of my senses is, after the signal processing, loosely based on something like forming hypotheses about what is going on in the context of how we understand the world, and evaluating their consistency and credibility. I accept the possibility that all of this is nothing more than very sophisticated statistical pattern-matching, but that has yet to be demonstrated.

Re: Is AI Riding a One-Trick Pony?

#200

Post by zwischenzug » Sat, Sep 30, 2017, 3:27 PM UTC

gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

I feel like Hofstadter was one of those people thinking really deeply about AI.

Anyone who doesn't know what I'm talking about should read 'Goedel, Escher, Bach', or 'Fluid Analogies'. I haven't read them in a long while, but I'm sure they're going to be relevant for decades, because they deal with the fundamental challenge of what it means to think. Backpropagation may be part of the puzzle, but the brain (and intelligence) is so much more than that.