Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How deep is the brain? The shallow brain hypothesis

nature.com

Post reply on HN
Print view View original

101–110 of 183 posts

Re: How deep is the brain? The shallow brain hypothesis

#101

Post by ben_w »

NoToP wrote: post #97
jakobson14 wrote: post #4

If I had a nickel for every time some neurologist tried to compare brains to neural networks. It's a surefire way to tell someone is either desperate for grant money or has been smoking crack. (previously: comparing brains and "electronic computers") Their entire article hinges on the complaint "brain seems shallow and neural networks are deep, ergo neural networks are doing it wrong." Neurologists seem to have a rea…

I disagree profoundly. There are things the brain does we have not yet been able to reproduce with a neural network, or to the extent we have seemingly with excessive resources of training and network size. Therefore there is some salient feature of neurology which has been overlooked. I don't think it is necessary to mimic biology down to the exact function of real neurons, but there must in fact be something we are…

Possibly, but it may also be that we're training them wrong.

"Book smart, not street smart" (to use a catchphrase) would apply perfectly to GPT models: brain the size of a rodent's, with 50,000 year's experience of reading Reddit, Wikipedia, and StackOverflow, but no "real life" experiences of its own.

Re: How deep is the brain? The shallow brain hypothesis

#102

Post by albertzeyer »

jakobson14 wrote: post #33
dartos wrote: post #29

Earlier quoted context omitted.

You can use that argument for anything you disagree with. Do you have a source or anything?

Have a read through the first paper describing a convolutional neural network, from 1998: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf There's absolutely no mention of biological inspiration whatsoever. At the same time, one can point to a long and rich history of convolutional filters being used in signal processing. And then there's the name, Convolutional Neural Network. The entire concept of a CNN is frame…

Some time around mid 1995 until basically now, it became out of fashion to explain your motivations of some new modeling as inspired from biology, as that was often handwaving with only little understanding of the actual neuroscience. So that is why people stopped writing that in papers. Just let the actual performance numbers speak for themselves. Either you get good performance, then it doesn't really matter where this was inspired from, or it does not work well, then it also does not matter where this was inspired from. In machine learning, it mostly matters whether it works well or not.

Re: How deep is the brain? The shallow brain hypothesis

#103

Post by ben_w »

mjburgess wrote: post #91
dilawar wrote: post #23

Earlier quoted context omitted.

> previously: comparing brains and "electronic computers") Before that: comparing brain with hydraulic machines. There has been tendency to compare brain with most complex machine known to us at that particular time. "Descartes was impressed by the hydraulic figures in the royal gardens, and developed a hydraulic theory of the action of the brain. We have since had telephone theories, electrical field theories, and n…

I cannot agree enough with Karl here. What is the brain? An organic system with deep roots in the organic body, with deep causal connections with its environment. There's little sense in ignoring the whole basic mode of operation, physics, chemistry and biology of the brain in order to analogise it to another system without any of those properties. This, at best, provides a set of inspirations for engineers -- it doe…

I mildly disagree (although your final conclusion is correct: it indeed does nothing for science).

The deepest fundamental structures in the brain[0] are quantum fields, which are also the deepest fundamental structures in everything else.

There is no known quantum field of "soul" or "intelligence".

The right abstraction is higher, and could still be a whole lot of things; but as maths can be implemented in logic, which can be implemented in electronics or clockwork or hydraulics, it doesn't matter what analogy is used — and my mild disagreement here is that such inspiration has been useful and gotten us this far.

[0] that we know of

Re: How deep is the brain? The shallow brain hypothesis

#104

Post by ben_w »

jacobsimon wrote: post #79
vkou wrote: post #48

Earlier quoted context omitted.

Sure, and airplanes are inspired by birds. That doesn't mean that detailed studies of the Boeing 747 are going to unlock a lot of hitherto unknown mysteries of heron behaviour.

I mean, I know you’re just providing an analogy, but people are still studying the physics of bird flight and we’re nowhere close to building machines yet that can maneuver the way birds can. https://www.quantamagazine.org/geometric-analysis-reveals-ho...

I could believe "we have more to learn", but not "we're nowhere close":

https://youtu.be/w6VLzKACnS8?si=DZgOPuBRG4Vt98su

Re: How deep is the brain? The shallow brain hypothesis

#106

Post by ben_w »

mjan22640 wrote: post #88
bjourne wrote: post #37

Earlier quoted context omitted.

First, I wonder how you got access to the article? It is behind a paywall and not yet uploaded to the sites I usually find paywalled articles on. Second, there is no need to compare brains to neural networks because brains are neural networks. Neurons form vertices and axons edges connecting the aforementioned. What you are perhaps thinking of are artificial neural networks - most of which are very dissimilar to brai…

A neuron is analogous to a 3d integrated circuit rather to a transistor. A molecule acts like a transistor https://medium.com/the-physics-arxiv-blog/the-origin-of-life... Changes in mechanical pressure, electric field, other molecules attachment, photon absorption, can control the conductivity. Organic semiconductors designed to fit like lego bricks to naturally build the desired structure are IMHO the way to go to p…

> silicone

I've seen this particular mistake a lot recently. New and exciting auto-corrupt from the latest version of iOS?

Given that our brains rewire themselves live, which ANNs can only do by being excessively connected and updating weights to/from zero, silicone (I'm thinking mainly the oil form) may be a better inspiration than lego.

https://en.wikipedia.org/wiki/Silicone

Re: How deep is the brain? The shallow brain hypothesis

#107

Post by mjburgess »

ben_w wrote: post #103
mjburgess wrote: post #91

Earlier quoted context omitted.

I cannot agree enough with Karl here. What is the brain? An organic system with deep roots in the organic body, with deep causal connections with its environment. There's little sense in ignoring the whole basic mode of operation, physics, chemistry and biology of the brain in order to analogise it to another system without any of those properties. This, at best, provides a set of inspirations for engineers -- it doe…

I mildly disagree (although your final conclusion is correct: it indeed does nothing for science). The deepest fundamental structures in the brain[0] are quantum fields, which are also the deepest fundamental structures in everything else. There is no known quantum field of "soul" or "intelligence". The right abstraction is higher, and could still be a whole lot of things; but as maths can be implemented in logic, wh…

The process of evolution acts on organic systems, it doesn't act on quantum fields.

I appreciate there's some (imv strange) sense of 'intelligence' where 'finding the right puzzle piece' counts. I cannot fathom why we care about such a notion, and it seems to have almost nothing to do with what we do care about re 'intelligence'.

We care about that thing animals do, that thing which some do better than others. That thing which evolution brought about for (rapid) adaptive fitness to one's environment.

'Everything else is stamp collecting'

We already have a perfectly good understanding of puzzles and their solutions -- animals are their inventors

Intelligence isnt in the solution to a puzzle it's in its design, and especially, in what one does when one cannot solve it -- ie., how one adapts

The csci view of 'intelligence' is an act of self-aggrandising, it turns out to be: csci!

This is none-sense.

Re: How deep is the brain? The shallow brain hypothesis

#108

Post by ben_w »

sheeshkebab wrote: post #7
jakobson14 wrote: post #5

Earlier quoted context omitted.

Yes, it's odd that sled dogs make terrible housepets. /s Neural networks fundamentally aren't designed to be otherwise. The workflow that has guided their entire development for over a decade is based around expensive training and static inference.

Why then all the talk about AGI when fundamentals don’t even allow for it to emerge.

Because "AGI" is very poorly defined, and ChatGPT is very "general" (compared to everything before it) and matches some (but not all) definitions of "intelligent".

Re: How deep is the brain? The shallow brain hypothesis

#109

Post by ben_w »

jakobson14 wrote: post #10
sheeshkebab wrote: post #7

Earlier quoted context omitted.

Why then all the talk about AGI when fundamentals don’t even allow for it to emerge.

Because drumming up talk about AGI is a really great way to get funding for your startup. The tech industry sustains itself on hype.

Not only but also.

Re: How deep is the brain? The shallow brain hypothesis

#110

Post by ben_w »

mjburgess wrote: post #107
ben_w wrote: post #103

Earlier quoted context omitted.

I mildly disagree (although your final conclusion is correct: it indeed does nothing for science). The deepest fundamental structures in the brain[0] are quantum fields, which are also the deepest fundamental structures in everything else. There is no known quantum field of "soul" or "intelligence". The right abstraction is higher, and could still be a whole lot of things; but as maths can be implemented in logic, wh…

The process of evolution acts on organic systems, it doesn't act on quantum fields. I appreciate there's some (imv strange) sense of 'intelligence' where 'finding the right puzzle piece' counts. I cannot fathom why we care about such a notion, and it seems to have almost nothing to do with what we do care about re 'intelligence'. We care about that thing animals do, that thing which some do better than others. That t…

We can simulate evolution in a computer, and this is used as a form of AI directly.

That said, the way you're using biological evolution in your comment sounds as much like a strange analogy as all of the others: we may have some genetically programmed responses to snakes (bad) and potential mates (good), but we can also say that a loss of hydraulic pressure in our brain is a stroke, and use electrical signals to both read from and write to the brain.

What we evolved to think, while interesting from a social perspective, seems to me like the least interesting part of our brains from an AI perspective — it's the bit that looks like a hard-coded computer program, not learning, on the scale of a human life and seen from within.

Post reply on HN