The Limitations of Deep Learning • Hacker News Board

Re: The Limitations of Deep Learning

#191

Post by SomeStupidPoint » Tue, Jul 18, 2017, 2:03 AM UTC

kxyvr wrote: post #131

SomeStupidPoint wrote: post #117

Earlier quoted context omitted.

You haven't explained how the first case isn't "curve fitting": the agents performing the compilation of those facts into the new fact are just spitting out the "best" fit string of symbols based on learned rules, etc etc. Somethings computers can (theoretically) do, and which fits the description "curve fitting" just fine. School (and other education) is training the model they're using to do that compilation, but i…

Alright, so from my perspective, curve fitting consists of three things 1. Definition of a model. ML models like multilayer perceptrons used a superposition of sigmoids, but newer models have superpositions of other functions and more nested hierarchies. 2. A metric to define misfit. Most of the time we use least squares because it's differentiable, but other metrics are possible. 3. An optimization algorithm to mini…

I can't think of a succinct way to describe my response, but I'm not sure we disagree, so much as we're talking about slightly different things.

Regardless, I wanted to thank you for the detailed replies -- having a back and forth helped me ponder my thoughts on the matter.

Have a good one. (:

Re: The Limitations of Deep Learning

#192

Post by nerdponx » Tue, Jul 18, 2017, 2:05 AM UTC

eli_gottlieb wrote: post #100

pishpash wrote: post #76

Earlier quoted context omitted.

You're talking about the complexity of the model. If you take a purely input-output view of the world (which by the way, even classical Physics does), every problem _is_ curve fitting in a sufficiently high dimensional space. There is no _conceptual_ problem here. There is perhaps a complexity problem, but that's why I wrote that "I think it's more hierarchical than that."

>If you take a purely input-output view of the world (which by the way, even classical Physics does), every problem _is_ curve fitting in a sufficiently high dimensional space. Not all spaces are Euclidean, and "purely input-output" still contains a lot of room for counterfactuals that ML models fail to capture.

What do you mean by counterfactuals? NNs are function approximation algorithms, in any geometry. No ifs ands or buts about it.

Re: The Limitations of Deep Learning

#193

Post by taneq » Tue, Jul 18, 2017, 2:06 AM UTC

nojvek wrote: post #158

raverbashing wrote: post #126

Earlier quoted context omitted.

Not possible, unfortunately

I've occasionally found that SVM's work great for one shot learning if you have good features and nicely labelled dataset. CNN's are really good at extracting features. Once you've extracted features that are generic, using an SVM as the last layer to train while keeping the CNN parameters intact yields great accuracy. I think that's where we are really headed. A combination of deep learning, boosted trees, svm, evol…

> I'm sure it will change society the way internet and mobile phones changed the world.

It will change the entire world the way humans changed the world. And that's scary.

Re: The Limitations of Deep Learning

#194

Post by fnl » Tue, Jul 18, 2017, 2:09 AM UTC

pron wrote: post #113

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

One of the greatest clear and present dangers of AI is that various existing algorithms are called just that, rather than what they are: statistical analysis algorithms, or, in short, statistics. Statistics used to be what we called the worst kind of lie; now it's becoming associated with intelligence, hinting at the ability to expose some great hidden truth. The problem lies not only with the algorithms, but with th…

This. I still cannot forget the disappointment of my parents and some family friends, all retired scientist or MDs, when I explained them how deep learning and natural language processing works a few years ago. They were truly upset that all this was "nothing more than clever accounting and statistics" at the end of the day, and no trace of the "advertised intelligence" - with Hinton's RBMs maybe coming closest, but by the time I was explaining how you use MCMC to train a Boltzmann machine, they again were complaining that even this is just modeling "statistical likelihoods, not true intelligence"...

In essence, we are only modeling patterns and their transformations, even if rather complex ones. But even the most basic prokaryote can model patterns, that has nothing to do with intelligence or consciousness per se. (And please don't get me started on swarm intelligence now... :-))

Re: The Limitations of Deep Learning

#195

Post by [deleted] » Tue, Jul 18, 2017, 2:09 AM UTC

pron wrote: post #113

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

One of the greatest clear and present dangers of AI is that various existing algorithms are called just that, rather than what they are: statistical analysis algorithms, or, in short, statistics. Statistics used to be what we called the worst kind of lie; now it's becoming associated with intelligence, hinting at the ability to expose some great hidden truth. The problem lies not only with the algorithms, but with th…

[deleted]

Re: The Limitations of Deep Learning

#196

Post by taneq » Tue, Jul 18, 2017, 2:11 AM UTC

L_Rahman wrote: post #61

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

I agree with your position. But I want to add a warning against the humanization of the brain. Many parts of it are complex in unknown ways, but some parts are truly mechanical. The parts of your central nervous system that respond to reflexes, that locate the source of sound or parse the color of retinal input are far more similar to deep learning algorithms than they are to what we think of as human consciousness.

> Many parts of it are complex in unknown ways, but some parts are truly mechanical.

I feel like this is a bit of a false dichotomy. We've never encountered any spooky non-mechanical non-physical part of the brain, and we've been looking since Cartesian dualism was in vogue.

What we think of as human consciousness is likely just a bunch of feedback loops allowing the brain to analyze some of its own state as if it were an external entity.

Re: The Limitations of Deep Learning

#197

Post by Buttons840 » Tue, Jul 18, 2017, 2:19 AM UTC

bllguo wrote: post #162

Buttons840 wrote: post #146

Earlier quoted context omitted.

> focus a lot of his time on the data itself... from where he intends to collect it? how is it structured? is it reliable? is it "enough"? What's the best books on this subject? I suppose it's a very broad topic and thus more difficult to talk about than a single "neural network" algorithm.

Interested in what part of that you feel needs to be explained in more depth? Not sure reading several books is necessary for explaining data collection and data munging...to me it's definitely something best learned by doing. work in data analysis/stats

Lots of things are best learned by doing. I just noticed there are dozens of books about machine learning algorithms but none on how to gather data. Of course, both those things can be learned independently, but I think there's room for at least a few books about data gathering considering it's so important for good machine learning results.

Re: The Limitations of Deep Learning

#198

Post by Isabela32 » Tue, Jul 18, 2017, 2:19 AM UTC

Cybiote wrote: post #149

mtremsal wrote: post #140

Earlier quoted context omitted.

My understanding is that innovation comes from reinforcement learning during self-play (rather than supervised learning of pro games), and thus goes against the best moves suggested by AlphaGo's policy network, in turn pushing it towards new options. In a sense, it seems innovation arises when the value network forces the policy network to expand the search space because an apparently unlikely move leads to downstrea…

It's not that simple. The creativity is that the combination of rollouts, policy and value networks allow for more efficient traversal of the search space. Which gets you better exploration of possible paths, meaning more options than a human considered and therefore more creativity.

Yeah it`s Possible......Anybody can earn 250$+ daily... You can earn from 6000-12000 a month or even more if you work as a full time job...It's easy, just follow instructions on this page, read it carefully from start to finish... It's a flexible job but a good eaning opportunity.. HERE► :) www.joinmate2.com™ :)............WORK AT HOME Copy&Paste This Link ◕Good Luck◕

Re: The Limitations of Deep Learning

#199

Post by Kamryn343 » Tue, Jul 18, 2017, 2:29 AM UTC

Buttons840 wrote: post #197

bllguo wrote: post #162

Earlier quoted context omitted.

Interested in what part of that you feel needs to be explained in more depth? Not sure reading several books is necessary for explaining data collection and data munging...to me it's definitely something best learned by doing. work in data analysis/stats

Lots of things are best learned by doing. I just noticed there are dozens of books about machine learning algorithms but none on how to gather data. Of course, both those things can be learned independently, but I think there's room for at least a few books about data gathering considering it's so important for good machine learning results.

Yeah it`s Possible. .....Anybody can earn 250$+ daily... You can earn from 6000-12000 a month or even more if you work as a full time job... HERE O════►►►http://www.joinmate2.com

Re: The Limitations of Deep Learning

#200

Post by fnl » Tue, Jul 18, 2017, 2:30 AM UTC

Put a lot simpler: Even DL is still only very complex, statistical pattern matching.

While pattern matching can be applied to model the process of cognition, DL cannot really model abstractive intelligence on its own (unless we phrase it as a pattern learning problem, viz. transfer learning, on a very specific abstraction task), and much less can it model consciousness.