How deep is the brain? The shallow brain hypothesis
1–10 of 183 posts
Re: How deep is the brain? The shallow brain hypothesis
#2It’s indeed odd that current dnn’s require massive amount of energy to retrain and lack any kind of practical continuous adaptation and learning.
Re: How deep is the brain? The shallow brain hypothesis
#3It’s indeed odd that current dnn’s require massive amount of energy to retrain and lack any kind of practical continuous adaptation and learning.
With computer-based intelligence we have the overhead of computing every bit though (probably) inefficient silicon and direct electric currents. The brain leverages the properties of chemicals, though millions of years of evolution.
Re: How deep is the brain? The shallow brain hypothesis
#4If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers")
Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong."
Neurologists seem to have a really hard time comprehending that researchers working on neural networks aren't as clueless about computers as neurology is about the brain. They also vastly overestimate how much engineers working on neural networks even care about how biological brains work.
Virtually every attempt at making neural networks mimic biological neurons has been a miserable failure. Neural networks, despite their name, don't work anything like biological neurons and their development is guided by a combination of
A) practical experimentation and refinement, and
B) real, actual understanding about how they work.
The concept of resnets didn't come from biology. It came from observations about the flow of gradients between nodes in the computational graph. The concept of CNNs didn't come from biology, it came from old knowledge of convolutional filters. The current form and function of neural networks is grounded in repeated practical experimentation, not an attempt to mimic the slabs of meat that we place on pedestals. Neural networks are deep because it turns out hierarchical feature detectors work really well, and it doesn't really matter if the brain doesn't do things that way.
And then you have the nitwits searching the brain for transformer networks. Might as well look for mercury delay line memory while you're at it. Quantum entanglement too.
Re: How deep is the brain? The shallow brain hypothesis
#5It’s indeed odd that current dnn’s require massive amount of energy to retrain and lack any kind of practical continuous adaptation and learning.
Yes, it's odd that sled dogs make terrible housepets. /s
Neural networks fundamentally aren't designed to be otherwise. The workflow that has guided their entire development for over a decade is based around expensive training and static inference.
Re: How deep is the brain? The shallow brain hypothesis
#6It’s indeed odd that current dnn’s require massive amount of energy to retrain and lack any kind of practical continuous adaptation and learning.
With computer-based intelligence we have the overhead of computing every bit though (probably) inefficient silicon and direct electric currents. The brain leverages the properties of chemicals, though millions of years of evolution.
The brain isn't a faster computer.
An infinitely-fast computer wouldn't meaningfully change the "expensive training vs fast, static inference" workflow that neural networks have always been developed around (except in the most brute force-y "retrain on the entire world, every single nanosecond" sense).
Re: How deep is the brain? The shallow brain hypothesis
#7It’s indeed odd that current dnn’s require massive amount of energy to retrain and lack any kind of practical continuous adaptation and learning.
Yes, it's odd that sled dogs make terrible housepets. /s Neural networks fundamentally aren't designed to be otherwise. The workflow that has guided their entire development for over a decade is based around expensive training and static inference.
Why then all the talk about AGI when fundamentals don’t even allow for it to emerge.
Re: How deep is the brain? The shallow brain hypothesis
#8It’s indeed odd that current dnn’s require massive amount of energy to retrain and lack any kind of practical continuous adaptation and learning.
When you say “massive amount of energy” are you comparing the energy requirements to a single human or to the billions of years of solar and geothermal energy that went into producing the human species?
Re: How deep is the brain? The shallow brain hypothesis
#9If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…
Agreed, but I do also think that order emerged from chaos. It’s an easy claim when order is defined by itself!
But in reality, we’re equipped exactly to exist, and we still wonder why in a backwards way, even with education (guilty!)
AI is the task of playing God like toddlers at recess, and LLMs the tower of babel. I still wanna play, it’s fun
Re: How deep is the brain? The shallow brain hypothesis
#10Earlier quoted context omitted.
Yes, it's odd that sled dogs make terrible housepets. /s Neural networks fundamentally aren't designed to be otherwise. The workflow that has guided their entire development for over a decade is based around expensive training and static inference.
Why then all the talk about AGI when fundamentals don’t even allow for it to emerge.
Because drumming up talk about AGI is a really great way to get funding for your startup. The tech industry sustains itself on hype.