Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI Riding a One-Trick Pony?

technologyreview.com

Post reply on HN
Print view View original

101–110 of 219 posts

Re: Is AI Riding a One-Trick Pony?

#101

Post by markan »

gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

Great quote from Hinton.

The biggest deficiency in AI is that we still don't have artificial systems which simulate human thought with any fidelity. Sooner or later that's bound to become a focus of attention.

Re: Is AI Riding a One-Trick Pony?

#102

Post by make3 »

gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

except that this has really only been going on for five years, which is nothing in the scale of human history or even of human rational thought. Some record number of people/scientists are working on getting the physics level understanding to happen, with crazy record breaking year after year quantity of people publishing and attending scientific conferences that fill up in like two days now. it is happening, and will happen even more in depth as time goes on

Re: Is AI Riding a One-Trick Pony?

#103

Post by bitL »

openasocket wrote: post #94
joe_the_user wrote: post #74

Earlier quoted context omitted.

It's not that bridges before Newton were bad, it's that Newton gave us the ability to design the strongest possible bridge of a given shape with the materials at hand - using not just calculus but calculus-of-variations, a subject nearly as old as Newtonian mechanics [1]. With this knowledge, what happens when one adds one or two columns to a bridge is now longer "news" the way it might have been before Newtonian mec…

I completely agree with your assessment, but the problem is a bit worse in my opinion. We already have a pretty firm grasp of how different ML systems learn and converge towards a solution in the average case. It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. We can't determine how well some ML architecture will perform at an object recognition…

The whole point of Deep Learning is that we don't want to describe math behind object recognition; it was the failed "classical" approach where people spent decades figuring out complex features which worked horribly. Deep Learning is actually pretty simple, well understood and parallelizable, and it's basically a billion-dimensional non-linear optimization. As optimization is infested with NP-hard problems, it's as difficult as it gets. It's actually amazing what we can do with it in the real world right now (and we are still far away from seeing all its fruits). Of course, it would frustrate academics that can't base AGI on top of it, but did they really think this approach would do it anyway?

Re: Is AI Riding a One-Trick Pony?

#104

Post by joe_the_user »

openasocket wrote: post #94
joe_the_user wrote: post #74

Earlier quoted context omitted.

It's not that bridges before Newton were bad, it's that Newton gave us the ability to design the strongest possible bridge of a given shape with the materials at hand - using not just calculus but calculus-of-variations, a subject nearly as old as Newtonian mechanics [1]. With this knowledge, what happens when one adds one or two columns to a bridge is now longer "news" the way it might have been before Newtonian mec…

I completely agree with your assessment, but the problem is a bit worse in my opinion. We already have a pretty firm grasp of how different ML systems learn and converge towards a solution in the average case. It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. We can't determine how well some ML architecture will perform at an object recognition…

It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better.

How 'bout "creating models that can work with more dimensions of the problem domain than are conveyed by standard data labeling"?

I mean, we don't simply want AI but actually "need" it in the sense that problems like biological system are too complex to understand without artificial enhancements to our comprehension processes - thus to "understand the problem domain better" we need AI. If it's true that "to build AI, we need to understand the problem domain better", it leaves us stuck in a chicken-and-problem. That might be the case but if we're going find a way out, we are going to need to build tools in the fashion humans used to solve problems many times before.

Re: Is AI Riding a One-Trick Pony?

#105

Post by epberry »

Does anyone know anything more about these “capsules”? I’ve seen some nature articles on them but these were from a neuroscience perspective. Has Hinton published anything on them?

Also if Hinton is the Einstein of deep learning, then will capsules be his “unified field theory”? I feel that if we embrace the Einstein analogy we should embrace it to its fullest.

Re: Is AI Riding a One-Trick Pony?

#106

Post by [deleted] »

QAPereo wrote: post #28
philipkglass wrote: post #20

I'm encouraged that so much fruitful work has come out of this one trick. If you can use the same basic framework for image labeling, playing Go, and translating natural languages, I'd say it's a powerful tool with broad applications. I think that there's a kernel of insight to "A real intelligence doesn’t break when you slightly change the problem." But human perception and intelligence are pretty brittle. The metho…

Of course, we're the product of an evolutionary history which results in such human "failure modes" being rare. If staring at a zebra made you hallucinate, you'd be unlikely to be the most successful member of your species, nor would your offspring thrive. So while we only tend to run into our obvious failing whens we do the unusual, computers fail at what we consider mundane.

[deleted]

Re: Is AI Riding a One-Trick Pony?

#107

Post by yters »

Xcelerate wrote: post #14
yters wrote: post #8

Earlier quoted context omitted.

All AI is just objective minimization. AI cannot solve the halting problem or perform Solomonoff induction.

> AI cannot [...] perform Solomonoff induction. Well it's clearly got to approximate it somehow, else humans wouldn't currently be making decently accurate predictions about the future.

You're making an assumption that is not necessarily true.

Re: Is AI Riding a One-Trick Pony?

#108

Post by openasocket »

joe_the_user wrote: post #104
openasocket wrote: post #94

Earlier quoted context omitted.

I completely agree with your assessment, but the problem is a bit worse in my opinion. We already have a pretty firm grasp of how different ML systems learn and converge towards a solution in the average case. It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. We can't determine how well some ML architecture will perform at an object recognition…

It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. How 'bout "creating models that can work with more dimensions of the problem domain than are conveyed by standard data labeling"? I mean, we don't simply want AI but actually "need" it in the sense that problems like biological system are too complex to understand without artificial enhancements…

It will probably play out like a conversation. A data scientist trains an ML model, and in analyzing the results discovers some intrinsic property or invariant of the problem domain. The scientist can then encode that information into the model and retrain. And that goes on and on, each time providing more accurate results.

As an aside, I think it's important that we find a way to examine and inspect how an ML model "works". If you have some neural network that does really well at the problem, it would be nice if you could somehow peer into it and explain, in human terms, what insight the model has made into the problem. That might not be feasible with neural networks, as they're really just a bunch of weights in a matrix, but this is practical for something like decision trees. Just food for thought.

Re: Is AI Riding a One-Trick Pony?

#109

Post by joe_the_user »

openasocket wrote: post #108
joe_the_user wrote: post #104

Earlier quoted context omitted.

It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. How 'bout "creating models that can work with more dimensions of the problem domain than are conveyed by standard data labeling"? I mean, we don't simply want AI but actually "need" it in the sense that problems like biological system are too complex to understand without artificial enhancements…

It will probably play out like a conversation. A data scientist trains an ML model, and in analyzing the results discovers some intrinsic property or invariant of the problem domain. The scientist can then encode that information into the model and retrain. And that goes on and on, each time providing more accurate results. As an aside, I think it's important that we find a way to examine and inspect how an ML model…

A data scientist trains an ML model, and in analyzing the results discovers some intrinsic property or invariant of the problem domain. The scientist can then encode that information into the model and retrain. And that goes on and on, each time providing more accurate results.

Mmmaybe,

It's tricky to articulate what pattern the data-scientist could see ... that an automated system couldn't see. Or otherwise, perhaps the whole "loop" could be automated. Or possibly the original neural already finds all the patterns available and what's left can't be interpreted.

Post reply on HN