Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How deep is the brain? The shallow brain hypothesis

nature.com

Post reply on HN
Print view View original

61–70 of 183 posts

Re: How deep is the brain? The shallow brain hypothesis

#61

Post by andbberger »

fastball wrote: post #52
__loam wrote: post #41

Earlier quoted context omitted.

No I think these comments are quite necessary. People need to stop making these comparisons because they have absolutely no grounding in how brains actually work. There are bad ideas that should be dismissed.

Neural networks are absolutely based on a very simplified model of how brains work. Specific NN architectures are in turn based on specific parts of the brain (e.g. Convolution Neural Networks are based on the visual cortices of cats/frogs).

nah, they're arbitrary function approximators that caught a lucky break. CNNs rose to prominence because natural scene statistics are translation invariant and convolutions can be efficiently computed on GPUs. and now that we have whole warehouses of GPUs, the current mood in DL is to stop building the symmetries of your dataset into the model (which is insane btw) and use brute force.

the tenuous connection DL once had to neuroscience (perceptrons) is a distant memory

Re: How deep is the brain? The shallow brain hypothesis

#62

Post by cortesoft »

quickthrower2 wrote: post #26
jakobson14 wrote: post #6

Earlier quoted context omitted.

The brain isn't a faster computer. An infinitely-fast computer wouldn't meaningfully change the "expensive training vs fast, static inference" workflow that neural networks have always been developed around (except in the most brute force-y "retrain on the entire world, every single nanosecond" sense).

I think we agree? I am talking to the efficiency of the brain. Not processing speed. Efficiency of the brain to do things advantageous to the selfish genes I guess. The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that. Silicon comes from an alien land, and is emulating. Even with the best algorithms there has to be a li…

> The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that.

Not really, evolution doesn't guarantee the brain will be supremely efficient. It just guarantees that it will be efficient ENOUGH.

Re: How deep is the brain? The shallow brain hypothesis

#63

Post by fastball »

andbberger wrote: post #61
fastball wrote: post #52

Earlier quoted context omitted.

Neural networks are absolutely based on a very simplified model of how brains work. Specific NN architectures are in turn based on specific parts of the brain (e.g. Convolution Neural Networks are based on the visual cortices of cats/frogs).

nah, they're arbitrary function approximators that caught a lucky break. CNNs rose to prominence because natural scene statistics are translation invariant and convolutions can be efficiently computed on GPUs. and now that we have whole warehouses of GPUs, the current mood in DL is to stop building the symmetries of your dataset into the model (which is insane btw) and use brute force. the tenuous connection DL once…

A fabricated re-telling of the past, given that we didn't start using GPUs for this type of compute until the turn of the millenium.

Re: How deep is the brain? The shallow brain hypothesis

#64

Post by SubiculumCode »

andbberger wrote: post #60
ramraj07 wrote: post #50

Earlier quoted context omitted.

No one disagrees we might be able to discern insights if we understand how our brain is wired. The problem is the current state of neuroscience is so flawed in its approach it’s not looking like they’re of any use. They don’t even understand how a 900 neuron worms system works but are more than happy to tap half a billion dollars from unsuspecting politicians saying they’ll map the human connectome. Go read the brain…

what are you talking about is this konrad kording's shitposting alt??? this reeks of naivety I certainly have many critiques of methods used in neuroscience rn (as a working neuroscientist) but to reduce those to the conclusion that the entire project of neuroscience is hopeless is absurd. We understand certain things quite well actually, and it's not at all obvious what "understanding" at a larger scale would look l…

As another working neuroscientist, thank you. And cheers.

Re: How deep is the brain? The shallow brain hypothesis

#65

Post by andbberger »

fastball wrote: post #63
andbberger wrote: post #61

Earlier quoted context omitted.

nah, they're arbitrary function approximators that caught a lucky break. CNNs rose to prominence because natural scene statistics are translation invariant and convolutions can be efficiently computed on GPUs. and now that we have whole warehouses of GPUs, the current mood in DL is to stop building the symmetries of your dataset into the model (which is insane btw) and use brute force. the tenuous connection DL once…

A fabricated re-telling of the past, given that we didn't start using GPUs for this type of compute until the turn of the millenium.

AlexNet was the turning point for DL.

Re: How deep is the brain? The shallow brain hypothesis

#67

Post by hliyan »

jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

"brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong"

Please don't claim things the author didn't. What I read was "ergo (artificial) neural networks may be missing a trick"

Re: How deep is the brain? The shallow brain hypothesis

#68

Post by hliyan »

hliyan wrote: post #66

"brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong" Please don't claim things the author didn't. What I read was "ergo (artificial) neural networks may be missing a trick"

Ignore. Reposted this under correct parent comment

Re: How deep is the brain? The shallow brain hypothesis

#70

Post by civilitty »

andbberger wrote: post #60
ramraj07 wrote: post #50

Earlier quoted context omitted.

No one disagrees we might be able to discern insights if we understand how our brain is wired. The problem is the current state of neuroscience is so flawed in its approach it’s not looking like they’re of any use. They don’t even understand how a 900 neuron worms system works but are more than happy to tap half a billion dollars from unsuspecting politicians saying they’ll map the human connectome. Go read the brain…

what are you talking about is this konrad kording's shitposting alt??? this reeks of naivety I certainly have many critiques of methods used in neuroscience rn (as a working neuroscientist) but to reduce those to the conclusion that the entire project of neuroscience is hopeless is absurd. We understand certain things quite well actually, and it's not at all obvious what "understanding" at a larger scale would look l…

Agreed. Reading the GP’s comment it feels like it’s from bizzaro world. It’s the computer scientists who have been claiming that neural networks resemble the human brain - they even fucking named them neural networks for christ’s sake! That could be excused as naive hubris in the 1980s, it’s utter delusion now.

A surface review of neuroplasticity literature alone should free anyone of the illusion that “neural networks” have even a passing resemblance to biological neurons, something covered in neuroscience 101 and is widely internalized by its practitioners. The BS grant writing and PR scientists have to participate in is hardly reflect of state of the art science itself.

The irony is that machine learning methods are a perfect fit for neuroscience and biology in general which generates reams of data that is largely so multidimensional that manual analysis is intractable. What we’re seeing now is the crest of the academic hype cycle which - if the history of bioinformatics is anything to go by - means that ML will take years if not decades for the field to understand and filly utilize.

Post reply on HN