Neurogenesis Deep Learning
arxiv.org
Neurogenesis Deep Learning
1–10 of 105 posts
Re: Neurogenesis Deep Learning
#2Basically trying to achieve a certain level of plasticity in deep neural nets by getting inspiration from https://en.wikipedia.org/wiki/Adult_neurogenesis
Re: Neurogenesis Deep Learning
#3Neurogensis? How about neural death as a way to prune large neural networks into more compact ones--now that is a research idea!
Re: Neurogenesis Deep Learning
#4Neurogensis? How about neural death as a way to prune large neural networks into more compact ones--now that is a research idea!
Looks like these researchers are trying to make a network more adaptive, I think that deleting nodes would only make them worse at the current task they're being trained on as well as worse on the tasks they're being adapted to.
You could train a model using neurogenesis to increase its accuracy, and then use distillation to train a smaller network to comparable accuracy.
But these are two very different, but complementary, problems.
Re: Neurogenesis Deep Learning
#5Neurogensis? How about neural death as a way to prune large neural networks into more compact ones--now that is a research idea!
This idea is at least partially in use with regularisation and dropout. The difference at least with dropout is that the "killed" neurons are then massaged back into the network in order become useful again.
Re: Neurogenesis Deep Learning
#6Neurogensis? How about neural death as a way to prune large neural networks into more compact ones--now that is a research idea!
I believe several papers have examined efficiently pruning neural networks, but neural death would be better branding ;) (https://arxiv.org/pdf/1506.02626v3.pdf https://arxiv.org/pdf/1510.00149v5.pdf)
Re: Neurogenesis Deep Learning
#7Very wishful thinking on my part, but I think we're far closer to a general intelligence than most expect.
Re: Neurogenesis Deep Learning
#8Slightly off topic, but I hate how publications are written. It seems like authors are purposely using big words and sentences that are often 5-6 lines long in order to make it seem more clever.
I find myself often having to reread a sentence in order to understand it.
These algorithms are often very simple and can be easily explained. Don't over complicate them.
Re: Neurogenesis Deep Learning
#9Very wishful thinking on my part, but I think we're far closer to a general intelligence than most expect.
Out of curiosity: how far away do you think we are?
Re: Neurogenesis Deep Learning
#10It never ceases to amaze me that the best steps towards achieving AI is to look at how we perceive that a Neuron works and simulate it.
And the thing is, we aren't exactly sure why exactly that is.. it's amazing.
Sometimes the best thing we can do is imitate nature