I wonder if the progress in artificial neural network technology is going to be correlated to discoveries on real neural networks.
I'm still wondering about the natural equivalent of back-propagation.
Study reveals how, when a synapse strengthens, its neighbors weaken
11–20 of 46 posts
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#12[deleted]
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#13The naive conclusion is to avoid learning in any form in order to keep all the synapses in optimal state. Rather disappointing.
Regardless of one’s general level of intelligence, there is that age-old adage of a trade-off between book smarts and street smarts, or any other niche skill. Forgetting, and the resulting ignorance, is part of learning. Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies.
Still, any society has certain forms of learning with outsized benefits that make them worthwhile.
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#14self pruning mechanism ?
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#15The naive conclusion is to avoid learning in any form in order to keep all the synapses in optimal state. Rather disappointing.
Regardless of one’s general level of intelligence, there is that age-old adage of a trade-off between book smarts and street smarts, or any other niche skill. Forgetting, and the resulting ignorance, is part of learning. Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies. Still, any society has certain forms of learning with outsized benefits that make them wor…
I remember reading a paper a little while ago that stated that experts are better at their craft because they're better at ignoring unimportant and irrelevant input.
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#16So basically L2 regularization.
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#17The naive conclusion is to avoid learning in any form in order to keep all the synapses in optimal state. Rather disappointing.
Maximizing total synapse strength would as useful as maximizing the number of 1-bits on your hard drive. It's not what you want :)
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#18In deep learning this is called local response normalization (LRN).
ConvNets use LRN where most active neurons inhibit other neurons at the same location in neighboring feature maps. http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#19I wonder if the progress in artificial neural network technology is going to be correlated to discoveries on real neural networks.
I'm still wondering about the natural equivalent of back-propagation.
Re: Study reveals how, when a synapse strengthens, its neighbors weaken
#20So it's similar to "lateral inhibition", but for synapses.