Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How deep is the brain? The shallow brain hypothesis

nature.com

Post reply on HN
Print view View original

81–90 of 183 posts

Re: How deep is the brain? The shallow brain hypothesis

#81

Post by cycomanic »

PaulDavisThe1st wrote: post #49
quickthrower2 wrote: post #26

Earlier quoted context omitted.

I think we agree? I am talking to the efficiency of the brain. Not processing speed. Efficiency of the brain to do things advantageous to the selfish genes I guess. The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that. Silicon comes from an alien land, and is emulating. Even with the best algorithms there has to be a li…

> The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that. This echoes an extremely naive view of evolution. There are many phenotypes in the living world which have evolved but for which there is no reason to believe that the phenotype is either (a) supremely efficient and/or (b) under selection pressure (the two are obvi…

> There are many phenotypes in the living world which have evolved but for which there is no reason to believe that the phenotype is either (a) supremely efficient and/or (b) under selection pressure (the two are obviously related).

> Evolution has no tautology. Brains do not evolve to be supremely efficient, just like humans do not evolve to be supremely efficient.

> What exists today is that which has survived, for whatever reason. It's not even possible to say something as apparently simplistic as "the only purpose evolution respects is leaving behind more copies" because that ignores (a) group selection (b) changing ecosystems that favor plasticity in the long run.

A primary example of this are our legs, they would be much more efficient if the knees pointed backwards. They are not the most efficient design, but simply good enough.

Re: How deep is the brain? The shallow brain hypothesis

#83

Post by nathias »

jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

Metaphores and analogies are important tools of thinking, even in science, some bear fruits some lead to errors, but we can't know in advance.

Re: How deep is the brain? The shallow brain hypothesis

#84

Post by radarsat1 »

jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

> every time some neurologist tried to compare brains to neural networks

Value of this comment aside, it kind of makes me chuckle how casually it (and other comments in this thread) just drops the word "artificial" from neural networks here, specifically when comparing with neurology. The irony is funny. Like, somehow we've forgotten why we call them that in the first place, exactly when talking about the thing that inspired the approach.

Re: How deep is the brain? The shallow brain hypothesis

#86

Post by GranularRecipe »

MagicMoonlight wrote: post #77

If it was shallow then it wouldn’t take 25 years for a human brain to fully train. The fact that some parts of it need that much data mean they must be way up the hierarchy.

The reason for deep learning is that shallow networks are very hard (or impossible) to train. In that sense, long time of training is evidence for shallow networks.

Re: How deep is the brain? The shallow brain hypothesis

#87

Post by IshKebab »

GranularRecipe wrote: post #86
MagicMoonlight wrote: post #77

If it was shallow then it wouldn’t take 25 years for a human brain to fully train. The fact that some parts of it need that much data mean they must be way up the hierarchy.

The reason for deep learning is that shallow networks are very hard (or impossible) to train. In that sense, long time of training is evidence for shallow networks.

No it's because shallow networks can't express complex functions. If you think about it the shallowest network is pretty much a lookup table. They can theoretically model any function, but the number of parameters needed means in practice they can't. Deep networks can learn much more complex functions for the same number of parameters.

Re: How deep is the brain? The shallow brain hypothesis

#88

Post by mjan22640 »

bjourne wrote: post #37
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

First, I wonder how you got access to the article? It is behind a paywall and not yet uploaded to the sites I usually find paywalled articles on. Second, there is no need to compare brains to neural networks because brains are neural networks. Neurons form vertices and axons edges connecting the aforementioned. What you are perhaps thinking of are artificial neural networks - most of which are very dissimilar to brai…

A neuron is analogous to a 3d integrated circuit rather to a transistor. A molecule acts like a transistor https://medium.com/the-physics-arxiv-blog/the-origin-of-life...

Changes in mechanical pressure, electric field, other molecules attachment, photon absorption, can control the conductivity.

Organic semiconductors designed to fit like lego bricks to naturally build the desired structure are IMHO the way to go to produce 3d circuits, rather than layered silicone litography.

Re: How deep is the brain? The shallow brain hypothesis

#89

Post by cma »

quickthrower2 wrote: post #26
jakobson14 wrote: post #6

Earlier quoted context omitted.

The brain isn't a faster computer. An infinitely-fast computer wouldn't meaningfully change the "expensive training vs fast, static inference" workflow that neural networks have always been developed around (except in the most brute force-y "retrain on the entire world, every single nanosecond" sense).

I think we agree? I am talking to the efficiency of the brain. Not processing speed. Efficiency of the brain to do things advantageous to the selfish genes I guess. The brain is supremely efficient at what the brain has evolved to do. It is almost tautological! Because if it wasn't, it wouldn't have evolved to that. Silicon comes from an alien land, and is emulating. Even with the best algorithms there has to be a li…

You're talking about something orthogonal, how efficient it is. He's talking about something different:

https://en.m.wikipedia.org/wiki/Catastrophic_interference

Which practically requires full retraining at every step to integrate new knowledge. I think we have some partial solutions like learning to select between finetunings, but not if the task needs to crosscut between them.

The human brain doesn't seem to suffer with catastrophic interference to nearly the same degree, independent of its computational efficiency, though there are possibly related things like developmental stages that if they are delayed may never be able to take place.

Re: How deep is the brain? The shallow brain hypothesis

#90

Post by mjburgess »

rerdavies wrote: post #74
jakobson14 wrote: post #25

Earlier quoted context omitted.

That's later backfill, a retroactive change to give a manufactured "biological" origin story. Whether they're real or not, researchers love a good "we took this from nature, isn't nature wonderful!" explanation. The C in CNN isn't "Convolution" for no reason. It came from work with convolutional filters (yay Sobel kernels!) which at it's height became filter banks and gabor filters and so on before neural networks pr…

> The C in CNN isn't "Convolution" for no reason. The first N in CNN is "Neural" for a reason.

Can you explain that reason?

Decision trees are called 'trees' for, more or less, the same reason.

ie., the diagrammed shape of a decision tree looks a little like the branches of a real one.

likewise, in the 50s where diagramming the earliest networks they were aiming to immitate a similar real-world structure.

Better that they had called them 'Variable Activation Networks' or some such, and none of this superstition would have started

Post reply on HN