Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
1–10 of 37 posts
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#2Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#3This paper makes use of hierarchical temporal memory.
I recommend reading this discussion: https://cs.stackexchange.com/questions/13089/some-criticisms...
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#4Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?
Most ANNs are not similar enough to biological NNs to attempt to recreate these structures.
A feed forward network is just matrix multiplication with a nonlinearity. It could, at best, be described as biologically inspired.
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#5So the implication in the title is almost certainly false. There are columns and layers in areas of cortex that are responsible for processing sensory modalities that are not inherently 3d in nature. If I had to guess (from my vague recollection of the literature) the current thinking is that laminar structures are a nice way to build things developmentally, and columns arise in part from development but also because of *otopic mapping, which is to say that many sensory modalities have a reasonable mapping into two dimensions that have correlations in their inputs based on the destance their sensors are from each other. Obviously not the whole story, but laminar and columnar structures are very likely not 'just' about building 3d representations.
All that said, I could buy an argument that cortex has settled on a 3d representation because, even though the exact encoding of muscle movements is most certainly not in x,y,z or other 3 vector representations, a large portion of the motor actions that we make are indeed in 3d space (a good counter example being language which has representations in cortex and acts on a very high dimensional space of our vocal and facial muscles).
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#6Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?
The short answer is no, the architectures don't transfer. At least not today. Biological neurons are much more complex, and therefore behave quite differently from neurons in neural networks.
To put things in perspective, we've fully mapped the connectome (map of neuron connections) of the simplest animal, C. Elegans, which has only about 300 neurons, yet we still can't simulate this organism's behavior computationally.
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#7For some context, this paper is from Numenta, Jeff Hawkins' company. Hawkins was the founder of Palm/Handspring back in the 90s. Afterwards he became quite interested in neuroscience and neural networks, and pulled a Wolfram in his 2004 book "On Intelligence" with claims that he had a unified theory of how the brain works.
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#8Former neuroscientist here.
I have no idea why this paper is at the top of HN, other than that the former founder of Palm is the lead author.
Setting aside that it hasn't been published or accepted, the only remotely novel thing seems to be they suggest that location information is represented in each column, and this occurs everywhere in the neocortex.
Well, what does the actual biology do? On the one hand, it's well-known that the trillions of long-range connections mean you can find neurons almost everywhere that will fire in response to place/location stimuli.
On the other hand, the entire medical history of restricted lesions and targeted deficits (Broca's area, Wernicke's area, patient HM's hippocampal anterograde amnesia, visual scotomas, etc etc etc) indicate some brain areas really are specialized for some things more than others.
The brain has some redundancy, but it's such an expensive organ (2% of body weight, but 20% of your energy budget) that I would not bet on such widespread replication if avoidable.
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#9For some context, this paper is from Numenta, Jeff Hawkins' company. Hawkins was the founder of Palm/Handspring back in the 90s. Afterwards he became quite interested in neuroscience and neural networks, and pulled a Wolfram in his 2004 book "On Intelligence" with claims that he had a unified theory of how the brain works. https://en.wikipedia.org/wiki/Jeff_Hawkins
As a former neuroscientist, the majority of people who claim to have grand unified theories of the brain are those who haven't studied it enough to know how far we still have to go.
Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure
#10Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?
And another question on top of that: is there any biological theory that corresponds to backpropagation?