Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How deep is the brain? The shallow brain hypothesis

nature.com

Post reply on HN
Print view View original

51–60 of 183 posts

Re: How deep is the brain? The shallow brain hypothesis

#51

Post by p1esk »

__loam wrote: post #41
robbrown451 wrote: post #13

Earlier quoted context omitted.

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News. There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain,…

No I think these comments are quite necessary. People need to stop making these comparisons because they have absolutely no grounding in how brains actually work. There are bad ideas that should be dismissed.

Artificial neural networks are the closest working model of a brain we have today.

Lots of graph nodes, with weighted connections, performing distributed computation (mainly hierarchical pattern matching), learning from data by gradually updating weights, using selective attention (and/or recurrence, and/or convolutional filters).

Which of the above is not happening in our brains? Which of the above is not biologically inspired?

In fact this description equally applies to both a brain and GPT4.

Re: How deep is the brain? The shallow brain hypothesis

#52

Post by fastball »

__loam wrote: post #41
robbrown451 wrote: post #13

Earlier quoted context omitted.

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News. There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain,…

No I think these comments are quite necessary. People need to stop making these comparisons because they have absolutely no grounding in how brains actually work. There are bad ideas that should be dismissed.

Neural networks are absolutely based on a very simplified model of how brains work. Specific NN architectures are in turn based on specific parts of the brain (e.g. Convolution Neural Networks are based on the visual cortices of cats/frogs).

Re: How deep is the brain? The shallow brain hypothesis

#53

Post by aeternum »

p1esk wrote: post #51
__loam wrote: post #41

Earlier quoted context omitted.

No I think these comments are quite necessary. People need to stop making these comparisons because they have absolutely no grounding in how brains actually work. There are bad ideas that should be dismissed.

Artificial neural networks are the closest working model of a brain we have today. Lots of graph nodes, with weighted connections, performing distributed computation (mainly hierarchical pattern matching), learning from data by gradually updating weights, using selective attention (and/or recurrence, and/or convolutional filters). Which of the above is not happening in our brains? Which of the above is not biological…

Many organisms have just a handful of neurons yet exhibit complex behavior that would be impossible given the weighted connections model. Not to mention single-celled organisms that exhibit ability to navigate.

The model can be the closest working model but that doesn't mean it is complete. It's very likely that cells can store memories/information independent from weights.

Re: How deep is the brain? The shallow brain hypothesis

#55

Post by fastball »

Affric wrote: post #17
4death4 wrote: post #8

Earlier quoted context omitted.

When you say “massive amount of energy” are you comparing the energy requirements to a single human or to the billions of years of solar and geothermal energy that went into producing the human species?

By that token the amount of energy for neural networks will be bound to some extent by the development of the biosphere and the creators of neural networks.

Not really? The point is that most artificial neural networks are started from basically zero (random noisy weights), where as a human neural network is jump-started with an overall neural structure that has been shaped by millions of years of evolution. Sure, it's not fair to compare the overall energy required to get there, but the point is just that a biological neural network starts with a huge headstart that is frequently forgotten when talking about efficiency.

Re: How deep is the brain? The shallow brain hypothesis

#56

Post by blovescoffee »

Salgat wrote: post #32

The brain communicates with itself, so deep layers are equivalent to sections of the brain talking to each other. The only relevance white matter depth has is with regard to how it's trained, and since it doesn't use gradient descent, it's irrelevant to neural networks in that regard.

Intercommunication does not equal layer depth.

Re: How deep is the brain? The shallow brain hypothesis

#57

Post by blovescoffee »

jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

Dude. What holy and special work do you do? There's nothing dumb or dull in searching for analogous structure between two effective machines, neither of which we understand.

Re: How deep is the brain? The shallow brain hypothesis

#58

Post by uoaei »

jakobson14 wrote: post #6
quickthrower2 wrote: post #3

Earlier quoted context omitted.

With computer-based intelligence we have the overhead of computing every bit though (probably) inefficient silicon and direct electric currents. The brain leverages the properties of chemicals, though millions of years of evolution.

The brain isn't a faster computer. An infinitely-fast computer wouldn't meaningfully change the "expensive training vs fast, static inference" workflow that neural networks have always been developed around (except in the most brute force-y "retrain on the entire world, every single nanosecond" sense).

It's an apples-to-oranges comparison. They're both fruit that grow on trees, but that's where the similarities end.

The primary difference, and likely the reason that brains are unreasonably effective, is the specifics of the architecture and internal representations (in the rigorous, information-theoretic sense) of its computational systems. It's not quite analog but it uses analog means. It's not quite digital but it does process via abstractions.

You can still reasonably call the brain a "computer" if you decide it can shed the laden history of that word and its close association with binary operations using transistors. You can do so because it uses internal structures to process inputs and emit outputs. But like I said above, it requires a generalized interpretation of the word to start to understand where and how the two fields of study may be unified.

Re: How deep is the brain? The shallow brain hypothesis

#59

Post by p1esk »

aeternum wrote: post #53
p1esk wrote: post #51

Earlier quoted context omitted.

Artificial neural networks are the closest working model of a brain we have today. Lots of graph nodes, with weighted connections, performing distributed computation (mainly hierarchical pattern matching), learning from data by gradually updating weights, using selective attention (and/or recurrence, and/or convolutional filters). Which of the above is not happening in our brains? Which of the above is not biological…

Many organisms have just a handful of neurons yet exhibit complex behavior that would be impossible given the weighted connections model. Not to mention single-celled organisms that exhibit ability to navigate. The model can be the closest working model but that doesn't mean it is complete. It's very likely that cells can store memories/information independent from weights.

We can’t do that not because our mathematical neurons are too simple. We can’t do that because we don’t know the algorithms those biological neurons are running.

Do you see the difference?

Re: How deep is the brain? The shallow brain hypothesis

#60

Post by andbberger »

ramraj07 wrote: post #50
robbrown451 wrote: post #13

Earlier quoted context omitted.

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News. There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain,…

No one disagrees we might be able to discern insights if we understand how our brain is wired. The problem is the current state of neuroscience is so flawed in its approach it’s not looking like they’re of any use. They don’t even understand how a 900 neuron worms system works but are more than happy to tap half a billion dollars from unsuspecting politicians saying they’ll map the human connectome. Go read the brain…

what are you talking about is this konrad kording's shitposting alt??? this reeks of naivety

I certainly have many critiques of methods used in neuroscience rn (as a working neuroscientist) but to reduce those to the conclusion that the entire project of neuroscience is hopeless is absurd. We understand certain things quite well actually, and it's not at all obvious what "understanding" at a larger scale would look like. It is very possible that the brain is irreducibly complex, and that the model you would need to construct to describe it would itself be so complex as to be useless in providing insight. Considering that the brain is by far the most complex object in the universe I think we're doing pretty well.

Furthermore, there are quite a lot of disagreements about the utility of connectomics. Outside of the extremists (Sebastian Seung and his ilk) no one thinks that connectomics is going to be the key that brings earth shattering insight. It's just another tool. There is a complete connectome for part of the drosophila brain already (privately funded btw), which is in daily use in many fly labs. It tells you what other neurons are connected to. Incredibly useful. Not earth shattering.

also you might want to measure the neuroscience funding you deem wasteful up against the tens of billions NASA is spending to send humans (and not robots) back to the moon for "the spirit of adventure". cold war's over. robots will do just fine for the moon.

Post reply on HN