31–37 of 37 posts
I find it disappointing that the paper makes no mention of Numenta, TBToI or HTM. How is what they are proposing not already included in Numenta's work (informally, of course)? Plus, Numenta's work seems to go much further confronting biological plausibility head-on.
The book On Intelligence by Jeff Hawkins was a fantastic read on HTM and similar concepts. (https://amzn.to/2JyQDF3)
function optimization in deep learning sense has nothing to do with neuroscience, I hope they don't think of fitting this model to brain processes just because it's popular
In a range of domains, in particular higher level brain areas, DL models trained on imagine are already the best predictive models of brain function. If they are better than all other models at describing the data, why would we say they have nothing to do with neuroscience?
I'm not sure, but I think it's trying to say that deep learning as it stands is modeled on one aspect of a model of the brain, by developing out the 3 aspects they identify and having them act in unison would be potentially a good thing, disclaimer, I am neither a neuroscientist nor deep learning expert!
What I understood is that they're saying deep learning relies on understanding neural processing in 3 parts: objective functions (activation functions maybe?), learning rules (I guess like back-prop/gradient descent?) and architecture (I assume network structure)? So it sounds like they want to use this componentization of neural processing to try to understand biological neural networks better.
Earlier quoted context omitted.
In a range of domains, in particular higher level brain areas, DL models trained on imagine are already the best predictive models of brain function. If they are better than all other models at describing the data, why would we say they have nothing to do with neuroscience?
DL models are also the best way to predict the behavior of three-body systems in physics. Would you say DL models tell us something about physics?
The fact is, there have been neuroscientists working with neural network models with greater and lesser complexity than DNNs for decades. They've been utilized to great profit outside of neuroscience lately, but that doesn't make them not an abstraction of some aspects of cortical computation.
We don't quite understand how brains could perform or approximate backprop yet, but it's the only training algorithm that has been remotely successful at training networks deep enough to do human-like visual recognition. So many people take that as a big clue as to what we should be looking for in the brain to explain its great performance and ability to learn, rather than a reason to disqualify DNNs entirely.
There's plenty of modeling work going on with more traditional biophysical models, such as those that include spiking, interneuron compartments, attractor dynamics, etc. This is just an attempt to also come at the problem from the other direction, starting from something that we know works well (for vision) and trying to figure out how to ground it in biophysical reality.
I see a lot of criticism here saying things like "DNNs have nothing to to with brains, they weren't designed to work like brains, and any resemblance is surely just an artifact of training them to do brain-like things." The fact is, there have been neuroscientists working with neural network models with greater and lesser complexity than DNNs for decades. They've been utilized to great profit outside of neuroscience…
https://singularityhub.com/2019/10/03/deep-learning-networks...