I have a metric ton of respect for Jeff and his team. However, I do think that they did not go far enough in figuring out the computational capabilities of individual pyramidal neurons, which IMO is the foundation to figuring out how the neocortex works. I wrote a short book about that once - if anyone's interested, check it out, it's free and meant to be readable for AI/neuroscience enthusiasts: http://www.corticalc…
Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
71–80 of 96 posts
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#72Earlier quoted context omitted.
They publish occasional DL papers, but results are not impressive, so mainstream DL people ignore them.
The don't do deep learning. Their papers are based on graphical models. Their RCN model can be said to be inspired by Numentas earlier HTM model that dileep developed there, but is very different in fundamental ways.
https://arxiv.org/abs/1611.02252
https://arxiv.org/abs/1611.02767
https://arxiv.org/abs/1706.04313
But it seems like lately they focus more on RL for robotics.
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#73Earlier quoted context omitted.
It's important to recognize that the cortex is a relatively late addition to the structure of the brain. Brains have been learning about the world and how to behave appropriately long before some mammals developed such a large region here. It's much more likely that the core functionality to be an autonomous agent in the world is encoded in the limbic system (e.g. basal ganglia, hypothalamus, mid brain and brain stem…
Certainly those regions are doing something in humans; and certainly those regions had those functions in species without a cortex. But the implication that these regions' presence in the human brain implies they're the ultimate arbiter of the same functionality there, isn't guaranteed to be true. Brains really do use "subsumption" (in the https://en.wikipedia.org/wiki/Subsumption_architecture sense) to accomplish va…
Looks like a kind of "deep" architecture:
"It does this by decomposing the complete behavior into sub-behaviors. These sub-behaviors are organized into a hierarchy of layers. Each layer implements a particular level of behavioral competence, and higher levels are able to subsume lower levels (= integrate/combine lower levels to a more comprehensive whole) in order to create viable behavior. For example, a robot's lowest layer could be "avoid an object". The second layer would be "wander around", which runs beneath the third layer "explore the world". Because a robot must have the ability to "avoid objects" in order to "wander around" effectively, the subsumption architecture creates a system in which the higher layers utilize the lower-level competencies. "
One can imagine how backpropagation style training on "explore the world" or other high-level scenarios would result in formation of "avoid objects" kernels at the lower levels similar to how image recognition deep nets produce Gabor like kernels at the lower levels. We do have some theorems on optimality of such deep networks for image recognition and i wouldn't be surprised if similar optimality was present for behavioral deep networks. Also, it seems that like in case of image deep nets, the deep architecture for behavior naturally allows for transfer learning by reusing the lower layers too - like one would reuse the low/mid layers of image deep net, one would naturally reuse the lower "avoid objects" layers (and may be some more complex aggregate behaviors from "mid-levels") for other tasks.
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#74I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#75Learning a lot about the neocortex. Any recommendations on how to learn more? @7:45 It’s the size of a dinner napkin, 2.5mm think, uniform, and similar in other animals. @9:50 if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. @34:30 Thousand brains theory of intelligence.
> if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. FYI, that is super hand-wavey and covers over a lot about how the path of the information from the cones/rods gets into V1. The chain of neurons that pass infomation from your eyes to V1 is well studied [0]. Interruptions in that path cause a lot of sightedness issues and are not fun diseases to h…
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#76I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…
I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#77I have a metric ton of respect for Jeff and his team. However, I do think that they did not go far enough in figuring out the computational capabilities of individual pyramidal neurons, which IMO is the foundation to figuring out how the neocortex works. I wrote a short book about that once - if anyone's interested, check it out, it's free and meant to be readable for AI/neuroscience enthusiasts: http://www.corticalc…
Thanks for sharing! Curious to learn more about this. Have you posted it in the HTM discourse forum? I would be interested in what the community thinks of it!
https://discourse.numenta.org/t/a-book-about-dendritic-compu...
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#78I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…
I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#79I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…
I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?
Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
#80Learning a lot about the neocortex. Any recommendations on how to learn more? @7:45 It’s the size of a dinner napkin, 2.5mm think, uniform, and similar in other animals. @9:50 if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. @34:30 Thousand brains theory of intelligence.