I'm pretty familiar with neural networks, and skimming that article it appears to describe something that is a neural network. Is 'Deep Learning' new terminology for 'Neural Network', or does it describe a subset of ways of using them?
Deep Learning From The Bottom Up
11–18 of 18 posts
Re: Deep Learning From The Bottom Up
#12I just skimmed the post as I don't have time to fully read it right now, but I'll point out a couple of problems that you can run into with neural nets and associated approaches. One issue that can be a back breaker depending on your application is that, to produce a generalizable model, nets tend to need much more training data than the alternatives. There are ways to work around this, though. The bigger problem to…
Sure, please post the link to the paper, it sounds interesting.
Re: Deep Learning From The Bottom Up
#13I'm pretty familiar with neural networks, and skimming that article it appears to describe something that is a neural network. Is 'Deep Learning' new terminology for 'Neural Network', or does it describe a subset of ways of using them?
I have the same question - seems like the only difference is scale.
Re: Deep Learning From The Bottom Up
#14To address some of the comments being presented here, neural nets despite being harder to train can be debugged visually. A few tips for those of you who use neural nets: Debug the weights with histograms. Track the gradient and make sure the magnitude is not too large and its normally distributed. Keep track of your gradient changes when using either gradient descent or conjugate gradient. Plot your filters, visuali…
Nice to see you HN, Adam =)
We just opened up the roadmap for contributions (click the "view source" with a logged in account). Feel free to add any of these notes where you think they'd fit in nicely -- don't worry about messing anything up, we have version control for a reason. Also, please email me if you run into any problems/confusion.
Re: Deep Learning From The Bottom Up
#15I'm pretty familiar with neural networks, and skimming that article it appears to describe something that is a neural network. Is 'Deep Learning' new terminology for 'Neural Network', or does it describe a subset of ways of using them?
Deep learning models are neural networks, but their recent popularization is due to a new method of building them incrementally, adding generative hidden layers trained as autoencoders to extract representative features, until the final discriminative layer. The resulting model is still a neural network, which can be finetuned by gradient methods, though conventionally training the same model from scratch with a random initialization would not have worked.
Re: Deep Learning From The Bottom Up
#16I'm pretty familiar with neural networks, and skimming that article it appears to describe something that is a neural network. Is 'Deep Learning' new terminology for 'Neural Network', or does it describe a subset of ways of using them?
Deep learning models are neural networks, but their recent popularization is due to a new method of building them incrementally, adding generative hidden layers trained as autoencoders to extract representative features, until the final discriminative layer. The resulting model is still a neural network, which can be finetuned by gradient methods, though conventionally training the same model from scratch with a rand…
Ah, I see. Thanks.
Re: Deep Learning From The Bottom Up
#17To address some of the comments being presented here, neural nets despite being harder to train can be debugged visually. A few tips for those of you who use neural nets: Debug the weights with histograms. Track the gradient and make sure the magnitude is not too large and its normally distributed. Keep track of your gradient changes when using either gradient descent or conjugate gradient. Plot your filters, visuali…
Nice to see you HN, Adam =) We just opened up the roadmap for contributions (click the "view source" with a logged in account). Feel free to add any of these notes where you think they'd fit in nicely -- don't worry about messing anything up, we have version control for a reason. Also, please email me if you run into any problems/confusion.
Will do! Like we discussed before, great initiative!
Re: Deep Learning From The Bottom Up
#18To address some of the comments being presented here, neural nets despite being harder to train can be debugged visually. A few tips for those of you who use neural nets: Debug the weights with histograms. Track the gradient and make sure the magnitude is not too large and its normally distributed. Keep track of your gradient changes when using either gradient descent or conjugate gradient. Plot your filters, visuali…
What do you think about Google Convnet platform?
I got it running, and played with supplied configurations, however it seems that it hasn't been updated in a while, for example, there's no dropout.
Then there's also Caffe CNN from Berkeley Vision group. Not sure what are the differences between the two.
Which one would you recommend as a learning tool, speed-wise, and as a possible starting point for customization?