Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI Riding a One-Trick Pony?

technologyreview.com

Post reply on HN
Print view View original

81–90 of 219 posts

Re: Is AI Riding a One-Trick Pony?

#81

Post by shimon »

aub3bhat wrote: post #69
gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

Humanity used fire for a long time before combustion was understood. Even today Anesthesia is not well understood at biological/physiological level that has not stopped its safe use and innovation through Clinical Trials. Maybe competitions and empiricism are the best approaches to building intelligent systems. Why get caught up in Physics/Math envy?

Because an improved theoretical understanding of how complex systems work can be incredibly valuable. Anesthesia is a good example - we get a lot of value from it, yes, but it would be way better if we could tailor dosages to individuals based on an understanding of how that individual will experience pain. There would be fewer severe complications, but also maybe you could wake up refreshed an hour after surgery instead of in a stupor.

If this could work with computer-trained models, that would be incredible too. What could a great speech understanding system teach us about language? What tricks from a facial-expression classifier could help autistic kids understand their friends?

Re: Is AI Riding a One-Trick Pony?

#82

Post by openasocket »

bra-ket wrote: post #71
danieltillett wrote: post #63

Earlier quoted context omitted.

In defence of AI researchers 1 is very, very hard and to the best of our knowledge there is not one way the brain works. The brain is a complex, cobbled together set of systems all using different ways of problem solving.

most AI researchers have never opened a textbook on cognitive psychology or neurobiology , or any of these 'soft' sciences. how do you plan to build artificial intelligence with no model of intelligence, without learning about important experiments in learning and memory , it's the complete ignorance that drives me crazy. AI is not for specialists.

Most of those experts aren't looking to solve general AI problems, they're looking for solutions to specific problems like basic image recognition. And you don't need a full human brain to do that, and you don't need to conform to the way humans and other biological systems do it. You're not aiming for full human intelligence, so you don't need to care too much about how humans learn.

That said, I find when trying to solve a problem with ML techniques, it's better to use someone who knows the problem domain really well than someone who only knows ML really well. Someone who really understands the problem they're trying to solve can encode that knowledge into their models when training the system. While I've seen people who really know ML but lack the specific domain knowledge labor for weeks, coming back to me with "discoveries" that are already well known.

Re: Is AI Riding a One-Trick Pony?

#83

Post by ballenf »

aub3bhat wrote: post #69
gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

Humanity used fire for a long time before combustion was understood. Even today Anesthesia is not well understood at biological/physiological level that has not stopped its safe use and innovation through Clinical Trials. Maybe competitions and empiricism are the best approaches to building intelligent systems. Why get caught up in Physics/Math envy?

I took his point not to criticize those early stages, but simply to acknowledge them as such. Early fire users could not have built a rocket no matter how many experiments they performed until they understood combustion (and some other sciences).

In AI, we're not building rockets yet, but we have some really awesome and really powerful bonfires or whatever.

At least that's how I understood his point.

(And in anesthesia, when we do understand those things, we may very well look on our use today as barbaric or dangerous.)

Re: Is AI Riding a One-Trick Pony?

#84

Post by mannykannot »

philipkglass wrote: post #20

I'm encouraged that so much fruitful work has come out of this one trick. If you can use the same basic framework for image labeling, playing Go, and translating natural languages, I'd say it's a powerful tool with broad applications. I think that there's a kernel of insight to "A real intelligence doesn’t break when you slightly change the problem." But human perception and intelligence are pretty brittle. The metho…

Human perception and intelligence are by no means infallible, but neither are they anywhere near being as brittle as current AI. The thing about illusions is that we generally know that we are being subject to an illusion, and we also usually have the depth of understanding to know when we don't understand something about what we are seeing or think we have heard, and we have the depth of understanding to think of actions we can take to resolve the issue.

As for cognitive biases, has any AI even come close enough to be comparable on that issue? For that matter, has any AI come close to understanding the concept of an optical illusion?

I am also encouraged by recent progress, but there is nothing to be gained in playing down the distance to go.

Re: Is AI Riding a One-Trick Pony?

#85

Post by thephyber »

kazinator wrote: post #62

> To continue reading this article, please exit incognito mode or log in. Eat shit and fucking die.

I had a slightly more diplomatic thought when I encountered the same thing.

It looks like it's JS+CSS, so we should probably submit an AdBlock rule update for it.

Re: Is AI Riding a One-Trick Pony?

#86

Post by TimPC »

TimPC wrote: post #80
joe_the_user wrote: post #74

Earlier quoted context omitted.

It's not that bridges before Newton were bad, it's that Newton gave us the ability to design the strongest possible bridge of a given shape with the materials at hand - using not just calculus but calculus-of-variations, a subject nearly as old as Newtonian mechanics [1]. With this knowledge, what happens when one adds one or two columns to a bridge is now longer "news" the way it might have been before Newtonian mec…

Present day AI on the Deep Learning side is a lot like what you describe. We haven't really had the Newtonian foundations yet. The theoretical foundations are quite limited because they are hard to figure out. But the techniques with less established theory work far better on most applications in AI. Redirecting work into areas of AI that have more solid theoretical foundations but worse application performance is no…

I should point out in this case it's almost certainly a genuine call to research the foundations underlying the working techniques more as Duvenaud publishes research using mostly the techniques that work well on applications.

Re: Is AI Riding a One-Trick Pony?

#87

Post by dragontamer »

bra-ket wrote: post #57
gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

it's really simple 1) learn how the brain works 2) build a simulator most current AI research skips step 1

> 1) learn how the brain works 2) build a simulator

I disagree that step #1 is important.

Consider the "Air-foil", which led to flight. In one sense, its an approximation of the wings of birds and other animals.

But ultimately, the discovery that the "Air-foil" shape turns sideways blowing wind into an upward force now called "lift" is completely different from how most people understand bird wings.

Bird Wings flap, but Airplane Air Foils do not.

--------

Another example: Neural Networks are one of the best mathematical simulations of the human brain (as we understand it, as well as a few simplifications to make Artificial Neural Networks possible to run on modern GPUs / CPUs).

However, the big advances in "Game AI" the past few years are:

1. Monte Carlo Tree Search -- AlphaGo (although some of it is Neural Network training, the MCTS is the core of the algorithm)

2. Counterfactual Regret Minimization -- The Poker AI that out-bluffed humans

There are other methodologies which have proven very successful, despite little to no biological roots. IIRC, Bayesian Inference is a widely deployed machine learning technique (for some definition of Machine Learning at least), but has almost nothing to do with how a human brain works.

An interesting field of AI is "Genetic Algorithms", which have biological roots but not anything based on the biology of brains, to achieve machine learning. Overall, a "Genetic Algorithm" is really just a randomized search in a multidimensional problem, but the idea of it was inspired by Darwinian Evolution.

Re: Is AI Riding a One-Trick Pony?

#88

Post by psyc »

bra-ket wrote: post #57
gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

it's really simple 1) learn how the brain works 2) build a simulator most current AI research skips step 1

In industry, yes. In academia, there's computational neuroscience:

http://cocosci.mit.edu/people

https://web.stanford.edu/group/mbc/research.html

Re: Is AI Riding a One-Trick Pony?

#89

Post by thephyber »

badminton1 wrote: post #32

It's not a one trick pony. It's the distinction between AI research and applied AI. There's more applied AI than there's research.

From TFA:

> Just about every AI advance you've heard of depends on a breakthrough that's three decades old. Keeping up the pace of progress will require confronting AI's serious limitations.

The expression "one trick pony" means it does one thing, not that its foundational principles are based on one research paper.

If this author's analogy holds, electromagnetism is a "one-trick pony", even though it has millions of applications (which is what common usage of that phrase refers to).

Re: Is AI Riding a One-Trick Pony?

#90

Post by anakron »

bra-ket wrote: post #57
gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

it's really simple 1) learn how the brain works 2) build a simulator most current AI research skips step 1

That's certainly one way to do it. However, we didn't succeed at building modern aircraft or earth moving machinery by building simulations of birds or muscles. There's enough that is unknown out there for a variety of approaches.
Post reply on HN