Live data from Hacker News

Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

lexfridman.com

71–80 of 96 posts

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#71

I have a metric ton of respect for Jeff and his team. However, I do think that they did not go far enough in figuring out the computational capabilities of individual pyramidal neurons, which IMO is the foundation to figuring out how the neocortex works. I wrote a short book about that once - if anyone's interested, check it out, it's free and meant to be readable for AI/neuroscience enthusiasts: http://www.corticalc…

Thanks for sharing! Curious to learn more about this. Have you posted it in the HTM discourse forum? I would be interested in what the community thinks of it!

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#72
post #30

Earlier quoted context omitted.

They publish occasional DL papers, but results are not impressive, so mainstream DL people ignore them.

The don't do deep learning. Their papers are based on graphical models. Their RCN model can be said to be inspired by Numentas earlier HTM model that dileep developed there, but is very different in fundamental ways.

Yeah they do, here are 3 papers on convnets:

https://arxiv.org/abs/1611.02252

https://arxiv.org/abs/1611.02767

https://arxiv.org/abs/1706.04313

But it seems like lately they focus more on RL for robotics.

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#73
post #50
post #47

Earlier quoted context omitted.

It's important to recognize that the cortex is a relatively late addition to the structure of the brain. Brains have been learning about the world and how to behave appropriately long before some mammals developed such a large region here. It's much more likely that the core functionality to be an autonomous agent in the world is encoded in the limbic system (e.g. basal ganglia, hypothalamus, mid brain and brain stem…

Certainly those regions are doing something in humans; and certainly those regions had those functions in species without a cortex. But the implication that these regions' presence in the human brain implies they're the ultimate arbiter of the same functionality there, isn't guaranteed to be true. Brains really do use "subsumption" (in the https://en.wikipedia.org/wiki/Subsumption_architecture sense) to accomplish va…

>https://en.wikipedia.org/wiki/Subsumption_architecture

Looks like a kind of "deep" architecture:

"It does this by decomposing the complete behavior into sub-behaviors. These sub-behaviors are organized into a hierarchy of layers. Each layer implements a particular level of behavioral competence, and higher levels are able to subsume lower levels (= integrate/combine lower levels to a more comprehensive whole) in order to create viable behavior. For example, a robot's lowest layer could be "avoid an object". The second layer would be "wander around", which runs beneath the third layer "explore the world". Because a robot must have the ability to "avoid objects" in order to "wander around" effectively, the subsumption architecture creates a system in which the higher layers utilize the lower-level competencies. "

One can imagine how backpropagation style training on "explore the world" or other high-level scenarios would result in formation of "avoid objects" kernels at the lower levels similar to how image recognition deep nets produce Gabor like kernels at the lower levels. We do have some theorems on optimality of such deep networks for image recognition and i wouldn't be surprised if similar optimality was present for behavioral deep networks. Also, it seems that like in case of image deep nets, the deep architecture for behavior naturally allows for transfer learning by reusing the lower layers too - like one would reuse the low/mid layers of image deep net, one would naturally reuse the lower "avoid objects" layers (and may be some more complex aggregate behaviors from "mid-levels") for other tasks.

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#74
post #31

I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…

I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#75
post #57
post #3

Learning a lot about the neocortex. Any recommendations on how to learn more? @7:45 It’s the size of a dinner napkin, 2.5mm think, uniform, and similar in other animals. @9:50 if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. @34:30 Thousand brains theory of intelligence.

> if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. FYI, that is super hand-wavey and covers over a lot about how the path of the information from the cones/rods gets into V1. The chain of neurons that pass infomation from your eyes to V1 is well studied [0]. Interruptions in that path cause a lot of sightedness issues and are not fun diseases to h…

Incidentally, Blindsight is the name of a science fiction novel which I really like. The name is inspired by that phenomenom, but it covers a lot of ground, from neurological processes to how aliens would look like (and how they would think), chinese room, etc.

https://rifters.com/real/Blindsight.htm

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#76
post #31

I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…

I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?

Most A.I. researchers believe the brain is a computer.

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#77

I have a metric ton of respect for Jeff and his team. However, I do think that they did not go far enough in figuring out the computational capabilities of individual pyramidal neurons, which IMO is the foundation to figuring out how the neocortex works. I wrote a short book about that once - if anyone's interested, check it out, it's free and meant to be readable for AI/neuroscience enthusiasts: http://www.corticalc…

Thanks for sharing! Curious to learn more about this. Have you posted it in the HTM discourse forum? I would be interested in what the community thinks of it!

Yes, there is a discussion thread here:

https://discourse.numenta.org/t/a-book-about-dendritic-compu...

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#78
post #31

I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…

I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?

In computer science, once you understand a data structure, its algorithm is often obvious [1]. Maybe the brain will be like that too, i.e. once you get its structure, you get its algorithm for free.

[1] https://news.ycombinator.com/item?id=20047607

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#79
post #31

I think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically…

I know the brain isn't a computer, but why do we think examining the structure is going to reveal an algorithm for intelligence? Isn't that like taking an iPhone, cutting it into slices and studying it with the hopes of finding how GarageBand works?

But before we can exam the algorithms on an iphone we need to understand the hardware, and cutting it open and looking at the structure is definitely a step on that path.

Re: Jeff Hawkins: Thousand Brains Theory of Intelligence [video]

#80
post #3

Learning a lot about the neocortex. Any recommendations on how to learn more? @7:45 It’s the size of a dinner napkin, 2.5mm think, uniform, and similar in other animals. @9:50 if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. @34:30 Thousand brains theory of intelligence.

Reading Jeff's book is a great intro. Then you can read some intro books like one of C. Koch's - very accessible. Brain theory is actually pretty readable until you get down to the biochemistry of a neuron, which is wildly complex. But, they still don't know how it all works, so it is fairly incomplete to read about (unlike much of physics, chemistry, or even many areas of biology)
Post reply on HN