Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How deep is the brain? The shallow brain hypothesis

nature.com

Post reply on HN
Print view View original

41–50 of 183 posts

Re: How deep is the brain? The shallow brain hypothesis

#41

Post by __loam »

robbrown451 wrote: post #13
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News. There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain,…

No I think these comments are quite necessary. People need to stop making these comparisons because they have absolutely no grounding in how brains actually work. There are bad ideas that should be dismissed.

Re: How deep is the brain? The shallow brain hypothesis

#42

Post by andromaton »

jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

Books and articles I was reading in the 80s (eg Minsky and Papert, Byte magazine) were referring to Rosenblatt and retinas.

Re: How deep is the brain? The shallow brain hypothesis

#43

Post by jacobsimon »

jakobson14 wrote: post #33
dartos wrote: post #29

Earlier quoted context omitted.

You can use that argument for anything you disagree with. Do you have a source or anything?

Have a read through the first paper describing a convolutional neural network, from 1998: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf There's absolutely no mention of biological inspiration whatsoever. At the same time, one can point to a long and rich history of convolutional filters being used in signal processing. And then there's the name, Convolutional Neural Network. The entire concept of a CNN is frame…

Surely you are trolling me now. There is a very clear biological inspiration mentioned in this paper: they literally define a CNN as having “receptive fields” and then they cite the same Hubel & Wiesel research mentioned before multiple times. LeCun mentions their research in papers even earlier in the 80s as well, during which they were awarded the Nobel prize for their research on the visual system. Of course there is also a lot of computational and mathematical research that was ongoing simultaneously, but to say that there is “no inspiration whatsoever” is pretty far from the truth.

Re: How deep is the brain? The shallow brain hypothesis

#44

Post by robbrown451 »

__loam wrote: post #41
robbrown451 wrote: post #13

Earlier quoted context omitted.

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News. There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain,…

No I think these comments are quite necessary. People need to stop making these comparisons because they have absolutely no grounding in how brains actually work. There are bad ideas that should be dismissed.

You're saying the study has no grounding in how brains work? I'd think a more reasonable conclusion would be that the neuroscientists involved have no grounding in how artificial neural networks work.

It seems the whole point is to bring in additional details of how brains work, that the think may be relevant to artificial NNs.

Re: How deep is the brain? The shallow brain hypothesis

#45

Post by visitor4711 »

SubiculumCode wrote: post #36
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

If you read this article, I think most would understand that it is primarily aimed at other neuroscientists, and only using ML structures an an analogy only, and I think a somewhat useful one to boot. The real point of the article was to propose a general hierarchy for how information flows in the brain, to emphasize the importance of subcortical brain even in higher order cognition, and proposes how simultaneous pro…

fully agree

Re: How deep is the brain? The shallow brain hypothesis

#46

Post by fastball »

jakobson14 wrote: post #33
dartos wrote: post #29

Earlier quoted context omitted.

You can use that argument for anything you disagree with. Do you have a source or anything?

Have a read through the first paper describing a convolutional neural network, from 1998: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf There's absolutely no mention of biological inspiration whatsoever. At the same time, one can point to a long and rich history of convolutional filters being used in signal processing. And then there's the name, Convolutional Neural Network. The entire concept of a CNN is frame…

That is definitely not the first paper describing a CNN. That is not even the first paper by Le Cun describing CNNs (he was already on them as early as 1989[1]).

Regardless, Le Cun is not the first to describe CNNs, merely one of the first to use them for OCR (specifically for hand-written text).

The first neural network arch to use convolutions instead of matmuls was this[2], from the year of our lord 1988. This in turn is based on Fukushima's "neocognitron"[3] (1980), which is based on the visual cortex of felines (from work done by Hubel and Wiesel in the 50s/60s).

I guess it is not super surprising you might be confused – Le Cun seems a bit more reticent than average to cite the work he's building on top of, and when he does it is frequently in reference to his own prior work. So if that is where you're getting your picture of artificial neural network history, your skewed perception makes sense.

[1] https://ieeexplore.ieee.org/abstract/document/41400

[2] https://proceedings.neurips.cc/paper/1987/file/98f1370821019...

[3] https://www.cs.princeton.edu/courses/archive/spr08/cos598B/R...

Re: How deep is the brain? The shallow brain hypothesis

#47

Post by spindle »

dilawar wrote: post #23
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

> previously: comparing brains and "electronic computers") Before that: comparing brain with hydraulic machines. There has been tendency to compare brain with most complex machine known to us at that particular time. "Descartes was impressed by the hydraulic figures in the royal gardens, and developed a hydraulic theory of the action of the brain. We have since had telephone theories, electrical field theories, and n…

And also comparing brains to clockwork.

Re: How deep is the brain? The shallow brain hypothesis

#48

Post by vkou »

crustacean111 wrote: post #15
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

CNNs actually are biologically inspired. The receptive field in a CNN mimics the way that cortical neurons only respond to stimuli in a restricted region of the visual field. Different cortical neurons have receptive fields that partially overlap to cover the whole visual field [1]. [1] - https://en.wikipedia.org/wiki/Convolutional_neural_network

Sure, and airplanes are inspired by birds. That doesn't mean that detailed studies of the Boeing 747 are going to unlock a lot of hitherto unknown mysteries of heron behaviour.

Re: How deep is the brain? The shallow brain hypothesis

#49

Post by PaulDavisThe1st »

quickthrower2 wrote: post #26
jakobson14 wrote: post #6

Earlier quoted context omitted.

The brain isn't a faster computer. An infinitely-fast computer wouldn't meaningfully change the "expensive training vs fast, static inference" workflow that neural networks have always been developed around (except in the most brute force-y "retrain on the entire world, every single nanosecond" sense).

I think we agree? I am talking to the efficiency of the brain. Not processing speed. Efficiency of the brain to do things advantageous to the selfish genes I guess. The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that. Silicon comes from an alien land, and is emulating. Even with the best algorithms there has to be a li…

> The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that.

This echoes an extremely naive view of evolution.

There are many phenotypes in the living world which have evolved but for which there is no reason to believe that the phenotype is either (a) supremely efficient and/or (b) under selection pressure (the two are obviously related).

Evolution has no tautology. Brains do not evolve to be supremely efficient, just like humans do not evolve to be supremely efficient.

What exists today is that which has survived, for whatever reason. It's not even possible to say something as apparently simplistic as "the only purpose evolution respects is leaving behind more copies" because that ignores (a) group selection (b) changing ecosystems that favor plasticity in the long run.

Re: How deep is the brain? The shallow brain hypothesis

#50

Post by ramraj07 »

robbrown451 wrote: post #13
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News. There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain,…

No one disagrees we might be able to discern insights if we understand how our brain is wired. The problem is the current state of neuroscience is so flawed in its approach it’s not looking like they’re of any use. They don’t even understand how a 900 neuron worms system works but are more than happy to tap half a billion dollars from unsuspecting politicians saying they’ll map the human connectome. Go read the brain initiative proposal [1] to see how out of touch with reality the scientists in this field are. I agree with OP that sharp criticism of the entire field is fully warranted.

1. https://braininitiative.nih.gov/sites/default/files/document...

Post reply on HN