The Limitations of Deep Learning • Hacker News Board

Re: The Limitations of Deep Learning

#181

Post by landon32 » Tue, Jul 18, 2017, 1:17 AM UTC

meh2frdf wrote: post #128

Correct me if I'm wrong but I don't see that with 'deep learning' we have answered/solved any of the philosophical problems of AI that existed 25 years ago (stopped paying attention about then). Yes we have engineered better NN implementations and have more compute power, and thus can solve a broader set of engineering problems with this tool, but is that it?

Yeah, I think the author's just priming the pump for a few more posts in this series that show a new way of abstraction/new NN architecture that solves some new problems.

Doesn't seem like he's trying to claim anything philosophical.

Re: The Limitations of Deep Learning

#182

Post by landon32 » Tue, Jul 18, 2017, 1:18 AM UTC

nv-vn wrote: post #173

meh2frdf wrote: post #128

Correct me if I'm wrong but I don't see that with 'deep learning' we have answered/solved any of the philosophical problems of AI that existed 25 years ago (stopped paying attention about then). Yes we have engineered better NN implementations and have more compute power, and thus can solve a broader set of engineering problems with this tool, but is that it?

Yep. The whole machine learning craze is just fueled by the fact that it's now feasible to create models for handwriting/voice/image recognition that actually work reliably. But in terms of the underlying technology, we haven't had some "breakthrough" that explained how the brain works or anything even close to that.

This is totally true, but I think it's still important to note that while something like Artificial General Intelligence is still way beyond the state of the art, the state of the art still has a huge impact on the world. A tiny slice of that can be seen in autonomous vehicles and the impact that they seem poised to have.

Re: The Limitations of Deep Learning

#183

Post by fnl » Tue, Jul 18, 2017, 1:32 AM UTC

ThomPete wrote: post #101

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

One can say that the human mind consist of millions of not very special parts. It's the aggregate, the complexity of which they interact that makes it special. Once you start to connect all these seemingly non-special abilities in deep learning the "magic" starts to happen. You get something that is more than the sum of it's parts. Of course it's not DL in itself thats interesting but the potential emergent complex r…

That's just another version of the trap GP spoke about. About a decade ago everybody was expecting emergent complex behavior from all kinds of evolutionary, intelligent ("swarm") systems. Didn't happen, seen that.

https://en.m.wikipedia.org/wiki/Swarm_intelligence

Re: The Limitations of Deep Learning

#184

Post by Abigail45 » Tue, Jul 18, 2017, 1:36 AM UTC

yters wrote: post #92

paganel wrote: post #58

Earlier quoted context omitted.

> The quality of the algo and I assume the deep learning model lies in the quality (breadth and depth) of the data, and how honest with himself the person choose to model it. I've only dabbled with machine-learning here and there for the past 10 years or so, but if there's one thing I've learned so far is that the data behind your ML code (and the way it is structured) is responsible for almost all the success or fai…

The data processing inequality says processing data does not increase its information content.

like Curtis said I can't believe that any body can make $8668 in one month on the internet . > Open this >>>>>>>>>>>http://usawork.cn.to

Re: The Limitations of Deep Learning

#185

Post by fnl » Tue, Jul 18, 2017, 1:40 AM UTC

raverbashing wrote: post #126

romaniv wrote: post #87

Earlier quoted context omitted.

I think it would help a lot if we brought random forests and SVMs to the same level of performance as DNNs. Demonstrating that more "mechanical" algorithms can be as efficient would dispel some of the anthropomorphism and allow for better analysis of why certain things work. I also believe that researches have responsibility to outline the limits of their own algorithms in research papers. (For example, presenting ex…

Not possible, unfortunately

True - until some clever guy proves us all wrong and finds ways to train some multidimensional/complex/deep/... kernel/forest/swarm/... that can learn those nonlinearities that currently only deep nets can be trained to detect (essentially, due to their relative simplicity, I'd say) :-)

Re: The Limitations of Deep Learning

#186

Post by [deleted] » Tue, Jul 18, 2017, 1:43 AM UTC

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

[deleted]

Re: The Limitations of Deep Learning

#187

Post by fnl » Tue, Jul 18, 2017, 1:49 AM UTC

L_Rahman wrote: post #61

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

I agree with your position. But I want to add a warning against the humanization of the brain. Many parts of it are complex in unknown ways, but some parts are truly mechanical. The parts of your central nervous system that respond to reflexes, that locate the source of sound or parse the color of retinal input are far more similar to deep learning algorithms than they are to what we think of as human consciousness.

Because that has nothing to do with consciousness... Every living cell can perceive such inputs, even the simplest of prokaryotes can "sniff" out their food sources.

Re: The Limitations of Deep Learning

#188

Post by Aubree- » Tue, Jul 18, 2017, 1:51 AM UTC

Cybiote wrote: post #149

mtremsal wrote: post #140

Earlier quoted context omitted.

My understanding is that innovation comes from reinforcement learning during self-play (rather than supervised learning of pro games), and thus goes against the best moves suggested by AlphaGo's policy network, in turn pushing it towards new options. In a sense, it seems innovation arises when the value network forces the policy network to expand the search space because an apparently unlikely move leads to downstrea…

It's not that simple. The creativity is that the combination of rollouts, policy and value networks allow for more efficient traversal of the search space. Which gets you better exploration of possible paths, meaning more options than a human considered and therefore more creativity.

Are you boring your life for pocket money.if u don’t worry for your pocket money I share my home profit system to everyone.i enjoy my life because I using this easy online jobs and earning $25986 daily wovrk for only 3 hours a day online for doing thivs easiest online home jobz.for more details visit this link… ᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵwww.joinmate2.comᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵᴵCOPY THE LINK.:) AND GOOD LUCK.:)

Re: The Limitations of Deep Learning

#189

Post by logicallee » Tue, Jul 18, 2017, 1:59 AM UTC

fnl wrote: post #183

ThomPete wrote: post #101

Earlier quoted context omitted.

One can say that the human mind consist of millions of not very special parts. It's the aggregate, the complexity of which they interact that makes it special. Once you start to connect all these seemingly non-special abilities in deep learning the "magic" starts to happen. You get something that is more than the sum of it's parts. Of course it's not DL in itself thats interesting but the potential emergent complex r…

That's just another version of the trap GP spoke about. About a decade ago everybody was expecting emergent complex behavior from all kinds of evolutionary, intelligent ("swarm") systems. Didn't happen, seen that. https://en.m.wikipedia.org/wiki/Swarm_intelligence

>Didn't happen,

...yet. See my comment here: https://news.ycombinator.com/item?id=14770230

Re: The Limitations of Deep Learning

#190

Post by thundergolfer » Tue, Jul 18, 2017, 2:03 AM UTC

webmaven wrote: post #167

jonobelotti wrote: post #152

Earlier quoted context omitted.

Brooks' 'Intelligence Without Representation' ( http://people.csail.mit.edu/brooks/papers/representation.pdf ) starts with a pretty strong argument imo against the story of 'stick-together' AGI you're describing.

Thanks for the link to this interesting paper. I think we're seeing some recapitulation of those arguments WRT 'ensembles of DL models' approaches.

I agree. Google has come out with some papers that are, to put it harshly, basic gluing together of DL models followed by loads of training on their compute resources.