Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

biorxiv.org

Post reply on HN
Print view View original

21–30 of 37 posts

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#21

Post by MrQuincle »

svara wrote: post #14
amelius wrote: post #10

Earlier quoted context omitted.

And another question on top of that: is there any biological theory that corresponds to backpropagation?

A really interesting question. It would seem far fetched at first that backpropagation could be used by the brain, because it's unclear what the mechanism for transmitting the error at every synapse backwards, to the synapses in the previous layer, would look like physically. This is a really interesting lecture given by Geoffrey Hinton ( https://www.youtube.com/watch?v=VIRCybGgHts ) where he discusses the various is…

You should check the neuromorphic community, really interesting work/people. They go from analog electronics to diehard neuroscience. I loved their Telluride summer school.

E.g. http://journal.frontiersin.org/article/10.3389/fnins.2016.00...

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#22

Post by neuronexmachina »

KingMob wrote: post #9
neuronexmachina wrote: post #7

For some context, this paper is from Numenta, Jeff Hawkins' company. Hawkins was the founder of Palm/Handspring back in the 90s. Afterwards he became quite interested in neuroscience and neural networks, and pulled a Wolfram in his 2004 book "On Intelligence" with claims that he had a unified theory of how the brain works. https://en.wikipedia.org/wiki/Jeff_Hawkins

As a former neuroscientist, the majority of people who claim to have grand unified theories of the brain are those who haven't studied it enough to know how far we still have to go.

Agreed, hence my comparison to Wolfram. Like Wolfram's "A New Kind of Science," he wrote a nice book that was a pretty decent layman's summary of the field, but includes conclusions that aren't anywhere near as revolutionary as he thinks they are.

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#24

Post by evincarofautumn »

aerlinger wrote: post #6
bmcooley wrote: post #2

Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?

The short answer is no, the architectures don't transfer. At least not today. Biological neurons are much more complex, and therefore behave quite differently from neurons in neural networks. To put things in perspective, we've fully mapped the connectome (map of neuron connections) of the simplest animal, C. Elegans, which has only about 300 neurons, yet we still can't simulate this organism's behavior computational…

Last I knew, OpenWorm was making good progress on the nervous system simulations, though. IIRC they’ve demonstrated swimming, retracting when bumping into walls, and food-seeking.

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#25

Post by sebastianmarkow »

aerlinger wrote: post #6
bmcooley wrote: post #2

Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?

The short answer is no, the architectures don't transfer. At least not today. Biological neurons are much more complex, and therefore behave quite differently from neurons in neural networks. To put things in perspective, we've fully mapped the connectome (map of neuron connections) of the simplest animal, C. Elegans, which has only about 300 neurons, yet we still can't simulate this organism's behavior computational…

Feed-forward CNNs pretty much describe V1 to V5 in primates. There are multiple papers on these architectures in the last decade.

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#26

Post by sebastianmarkow »

KingMob wrote: post #8

Former neuroscientist here. I have no idea why this paper is at the top of HN, other than that the former founder of Palm is the lead author. Setting aside that it hasn't been published or accepted, the only remotely novel thing seems to be they suggest that location information is represented in each column, and this occurs everywhere in the neocortex. Well, what does the actual biology do? On the one hand, it's wel…

This video gives a little more context: https://youtu.be/fhnMUc36opI

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#27

Post by sebastianmarkow »

sabalaba wrote: post #4
bmcooley wrote: post #2

Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?

Most ANNs are not similar enough to biological NNs to attempt to recreate these structures. A feed forward network is just matrix multiplication with a nonlinearity. It could, at best, be described as biologically inspired.

Also SNNs seem to be an active field of research. Namely research center Jülich (human brain project) is conducting experiments on modeling & simulating biological neuron populations.

https://en.wikipedia.org/wiki/Spiking_neural_network

http://www.nest-simulator.org

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#28

Post by sebastianmarkow »

KingMob wrote: post #9
neuronexmachina wrote: post #7

For some context, this paper is from Numenta, Jeff Hawkins' company. Hawkins was the founder of Palm/Handspring back in the 90s. Afterwards he became quite interested in neuroscience and neural networks, and pulled a Wolfram in his 2004 book "On Intelligence" with claims that he had a unified theory of how the brain works. https://en.wikipedia.org/wiki/Jeff_Hawkins

As a former neuroscientist, the majority of people who claim to have grand unified theories of the brain are those who haven't studied it enough to know how far we still have to go.

Hawkings was the PI at the Redwood Neuroscience Institute. Which is now part of Berkeley. He is not an outsider (15+ years of research). You could argue though, that Numenta is more focused on bringing these methods to market, then solely doing research. I get that this doesn't always resonate well with people in academia.

Re: Why Does the Neocortex Have Layers and Columns, Learning the World's 3D Structure

#29

Post by sebastianmarkow »

deepsun wrote: post #17
KingMob wrote: post #9

Earlier quoted context omitted.

As a former neuroscientist, the majority of people who claim to have grand unified theories of the brain are those who haven't studied it enough to know how far we still have to go.

Nevertheless I appreciate his efforts and just bought his book, because having an idea is way different from putting own money in it. He puts his own money to neuroscience, and does not compete with real scientists for government grants. Other millionaires may buy yachts, or just spread "novel" ideas (e.g. Hyperloop idea by St. Elon) without investing their own money.

The book being from 2004 is kind of outdated. Nevertheless fun to read.
Post reply on HN