Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Limitations of Deep Learning

blog.keras.io

Post reply on HN
Print view View original

241–250 of 296 posts

Re: The Limitations of Deep Learning

#241

Post by nullc »

19eightyfour wrote: post #239

If this article is correct about limitations, couldn't one simply include a Turing machine model into the process to train algorithms? Some ideas: - The vectors are Turing tapes, or - Each point in a tape is a DNN, or - The "tape" is actually a "tree" each point in the tape is actually a branch point of a tree with probabilities going each way, and the DNN model can "prune this tree" to refine the set of "spanning tr…

You can bolt all kinds of funny structures into some DNN system, but if the system doesn't have well behaved gradients (or if it isn't even differentiable) it won't train.

Re: The Limitations of Deep Learning

#242

Post by [deleted] »

pishpash wrote: post #59
reader5000 wrote: post #4

Recurrent models do not simply map from one vector space to another and could very much be interpreted as reasoning about their environment. Of course they are significantly more difficult to train and backprop through time seems a bit of a hack.

Sure they do. The spaces are just augmented with timestep related dimensions.

[deleted]

Re: The Limitations of Deep Learning

#243

Post by erikb »

backpropaganda wrote: post #240
erikb wrote: post #231

Earlier quoted context omitted.

Unlikely. If it's mostly memorisation it couldn't learn from playing itself. And what you describe is how AI beats chess. The problem with that is that it is a quite inhuman way to play. But AlphaGo plays quite humanly.

1. Imagine infinite compute capability. Exhaustively play all possible games, and use that to figure out best moves at any state. This is essentially what Alphago did, but using translation variance to reduce the search space. 2. There is no contradiction here. We just have to accept that human-like play can emerge from memorization.

Can you explain what from your point of view is the difference between AlphaGo and a Chess AI? Because to me it sounds like the one should have resulted as an evolution from the other if it would be that simple.

Re: The Limitations of Deep Learning

#244

Post by honestoHeminway »

denzil_correa wrote: post #79
paganel wrote: post #58

Earlier quoted context omitted.

> The quality of the algo and I assume the deep learning model lies in the quality (breadth and depth) of the data, and how honest with himself the person choose to model it. I've only dabbled with machine-learning here and there for the past 10 years or so, but if there's one thing I've learned so far is that the data behind your ML code (and the way it is structured) is responsible for almost all the success or fai…

> I've learned so far is that the data behind your ML code (and the way it is structured) is responsible for almost all the success or failure of any given ML algorithm Data is indeed a necessary condition but certainly not sufficient. You require a good marriage between engineering features and data to have a good success rate. Learning curves [0] are a good way to understand if your ML algorithm requires more data…

Much of the programming with ML has moved towards cleaning, extrapolating and generating the data.

But this type of programing is - miracles- bugfree. We never hear of data-conversion gone wrong, data corrupted or data-mining withou conclusive results here. Obviously such bugs lack the glamour of security bugs.

Re: The Limitations of Deep Learning

#245

Post by posterboy »

therajiv wrote: post #5

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models. I think a lot of the issue stems from layman explanations of neural net…

I always counter, the intelligence is not in the machine, but the builder. Antropomorphism is in line with that, because it projects the human qualities onto the machine, because, in a broad sense, they are modelled after those. Egoistic as we are, that's the only way to understand anything, to remove the shizm between animate and inanimate objects. Just like a fishing rod is just the extension of an arm.

Re: The Limitations of Deep Learning

#246

Post by Aracely45 »

sjg007 wrote: post #177
eanzenberg wrote: post #147

Earlier quoted context omitted.

We can debate, but I don't think another AI winter will happen again in my lifetime. AI work is just earning way too much money for its funding to get cut, and a lot of funding is currently private too.

I'd be surprised to see inductive learning anytime soon. But I definitely see the next generation of AI systems, robots and their implementation across industry. But that will rapidly fill out and then we will still be left with self determination.

My last pay check was $9500 working 12 hours a week online. My sisters friend has been averaging 15k for months now and she works about 20 hours a week. I can't believe how easy it was once I tried it out. This is what I do, ====ⓌⓌⓌ.joinmate2.ⒸⓄⓂ

Re: The Limitations of Deep Learning

#247

Post by raverbashing »

nojvek wrote: post #158
raverbashing wrote: post #126

Earlier quoted context omitted.

Not possible, unfortunately

I've occasionally found that SVM's work great for one shot learning if you have good features and nicely labelled dataset. CNN's are really good at extracting features. Once you've extracted features that are generic, using an SVM as the last layer to train while keeping the CNN parameters intact yields great accuracy. I think that's where we are really headed. A combination of deep learning, boosted trees, svm, evol…

That's a good comment, and yes, SVM are very powerful in itself, they might not be "deep learning" but they're more powerful than linear learning and good for a lot of cases (as a last layer, as you mentioned, it's a good use case)

Yes, we'll have GAs building CNN architectures, or a mix of several techniques, I'm enthusiastic for what the future holds

Re: The Limitations of Deep Learning

#248

Post by raverbashing »

saosebastiao wrote: post #218
raverbashing wrote: post #126

Earlier quoted context omitted.

Not possible, unfortunately

Kaggle has already proven hundreds of times over that deep learning is not a silver bullet.

Thanks, I'm familiar with Kaggle and how most of the time a Random Forest (or XGBoost, or something like Vowpal Wabbit) will solve your problem

Re: The Limitations of Deep Learning

#249

Post by posterboy »

ThomPete wrote: post #235
fnl wrote: post #183

Earlier quoted context omitted.

That's just another version of the trap GP spoke about. About a decade ago everybody was expecting emergent complex behavior from all kinds of evolutionary, intelligent ("swarm") systems. Didn't happen, seen that. https://en.m.wikipedia.org/wiki/Swarm_intelligence

About a decade ago winning GO or self-driving cars were seen as pipedreams many decades away. Yet here we are. The author is making the mistake of thinking that just because he can show some areas were we aren't as far as we thought he has made an argument against AI. Thats not how it works. We don't get to decide what is the right metrics. All we can see is that we keep making progress sometimes large leaps sometime…

The fallacy there is glorifying consciousness. Full consciousness as in omniscence is an unachievable ideal. If we prescribe consciousness to ourselves, depending on the individual theory of conscious thought, that's likely faulty in some respect already.

Re: The Limitations of Deep Learning

#250

Post by raverbashing »

fnl wrote: post #185
raverbashing wrote: post #126

Earlier quoted context omitted.

Not possible, unfortunately

True - until some clever guy proves us all wrong and finds ways to train some multidimensional/complex/deep/... kernel/forest/swarm/... that can learn those nonlinearities that currently only deep nets can be trained to detect (essentially, due to their relative simplicity, I'd say) :-)

I don't think we'll see a deep svm, but if we see one I think we'll have something very powerful

Same for a deep decision tree (forest?). Or maybe a combination of several techniques, etc

Post reply on HN