Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Limitations of Deep Learning

blog.keras.io

Post reply on HN
Print view View original

221–230 of 296 posts

Re: The Limitations of Deep Learning

#224

Post by yorwba »

yters wrote: post #92
paganel wrote: post #58

Earlier quoted context omitted.

> The quality of the algo and I assume the deep learning model lies in the quality (breadth and depth) of the data, and how honest with himself the person choose to model it. I've only dabbled with machine-learning here and there for the past 10 years or so, but if there's one thing I've learned so far is that the data behind your ML code (and the way it is structured) is responsible for almost all the success or fai…

The data processing inequality says processing data does not increase its information content.

But processing does increase the "obviousness" of the information content.

E.g. projecting the data onto independent dimensions doesn't change the information it contains, but it highlights that those dimensions are indeed independent. Decomposing a multimodal distribution into a mixture of unimodal distribution gives more insight than just viewing it as a bunch of data mushed together. And so on.

I think there should be a branch of information theory that quantifies the obviousness of information and how it is changed by various data processing methods.

Re: The Limitations of Deep Learning

#225

Post by yorwba »

robotresearcher wrote: post #212
philipkglass wrote: post #178

Earlier quoted context omitted.

I was disagreeing with someone who wrote that we understand how neurons work and that perceptrons model them "quite well." They do not model biological neurons well at all. I agree that biological fidelity is not important for building useful ANNs.

I presented (a vulgar summary of) McCulloch's hypothesis, not my own. And since I didn't use the words "quite well", you are not entitled to put them in quotes.

philipkglass was referring to curiousgal's comment upthread: https://news.ycombinator.com/item?id=14790965

Re: The Limitations of Deep Learning

#226

Post by robotresearcher »

yorwba wrote: post #225
robotresearcher wrote: post #212

Earlier quoted context omitted.

I presented (a vulgar summary of) McCulloch's hypothesis, not my own. And since I didn't use the words "quite well", you are not entitled to put them in quotes.

philipkglass was referring to curiousgal's comment upthread: https://news.ycombinator.com/item?id=14790965

OK, thanks. Too late to edit. Adjust flames accordingly. Of course a perceptron is not an accurate model of a biological neuron. But as a reduction to a minimal model it's still pretty darn interesting.

Re: The Limitations of Deep Learning

#227

Post by kxyvr »

SomeStupidPoint wrote: post #191
kxyvr wrote: post #131

Earlier quoted context omitted.

Alright, so from my perspective, curve fitting consists of three things 1. Definition of a model. ML models like multilayer perceptrons used a superposition of sigmoids, but newer models have superpositions of other functions and more nested hierarchies. 2. A metric to define misfit. Most of the time we use least squares because it's differentiable, but other metrics are possible. 3. An optimization algorithm to mini…

I can't think of a succinct way to describe my response, but I'm not sure we disagree, so much as we're talking about slightly different things. Regardless, I wanted to thank you for the detailed replies -- having a back and forth helped me ponder my thoughts on the matter. Have a good one. (:

Thanks for chatting!
Post reply on HN