Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study urges caution when comparing neural networks to the brain

news.mit.edu

Post reply on HN
Print view View original

71–80 of 141 posts

Re: Study urges caution when comparing neural networks to the brain

#71

Post by marginalia_nu »

lalos wrote: post #28

Here's my guess: neurons tap into quantum mechanics but we are too primitive to understand that for now. The brain was initially modeled as humors/fluids back when we developed aqueducts, then telegraph came into the scene and it was modeled as electrical impulses and now computers/ML are popular therefore we see it as a neural network. Next step is quantum.

Neurons almost certainly use quantum processes, but so do most transistors. The brain is too too warm for large-scale quantum effects though. You're not going to find phase coherence at that scale in such an environment, which is pretty much the prerequisite for quantum effects (that is fairly well understood).

Re: Study urges caution when comparing neural networks to the brain

#72

Post by vharuck »

bawolff wrote: post #38

The aspect of ai that makes me think something related is going on, is how artifacts look in image generation systems like stable diffusion. Often these systems will have really bizzare artificats, people with 3 arms, etc. However at the same time when you glance at the output without looking carefully you will sometimes miss these artifacts even though they should be absolutely glaring.

But wouldn't the people creating these models and deciding whether to publish them prefer ones with these "understandable" mistakes? There might have been other ones that had equal potential as far the evaluation measure goes, but humans had been involved all along the way and said, "Yeah, that picture looks like a person made it. We should keep developing this model."

Re: Study urges caution when comparing neural networks to the brain

#73

Post by marginalia_nu »

tarboreus wrote: post #37
Teever wrote: post #31

Earlier quoted context omitted.

But why is the next step quantum? And why is this the final step?

Because we don't understand quantum physics, and we don't understand the brain. I don't think we know if it's the final step. There could be wizard jelly or something at the bottom.

Quantum physics is fairly well understood. Perhaps not among laymen, but that's mostly due to pedagogical challenges, which is why a lot of the discourse seems to be stuck approaching it as though we were living nearly 100 years into the past.

Re: Study urges caution when comparing neural networks to the brain

#74

Post by elcomet »

marmada wrote: post #65
ben_w wrote: post #47

Earlier quoted context omitted.

> But our bugs aren't sacred and special. I'd agree about sacred, but I have a hunch they may indeed be special… or at least useful. Current AI requires far more examples than we do to learn from, and I suspect all our biases are how evolution managed to do that.

Humans are trained on petabytes of data. From birth, we ingest sights, sounds, smells etc. Imagine a movie of every second of your life. And an audio track of every second of your life. Etc. Etc. Humans get a lot of data.

And you didn't even count the data from million of years of evolution. The brain doesn't come as a blank slate when you're born.

Re: Study urges caution when comparing neural networks to the brain

#75

Post by ben_w »

marmada wrote: post #65
ben_w wrote: post #47

Earlier quoted context omitted.

> But our bugs aren't sacred and special. I'd agree about sacred, but I have a hunch they may indeed be special… or at least useful. Current AI requires far more examples than we do to learn from, and I suspect all our biases are how evolution managed to do that.

Humans are trained on petabytes of data. From birth, we ingest sights, sounds, smells etc. Imagine a movie of every second of your life. And an audio track of every second of your life. Etc. Etc. Humans get a lot of data.

Humans get a lot of data.

AI gets more examples.

Tesla autopilot has a movie of every second it's active, for every car in the fleet that uses it. It has how many lifetimes of driving data now? And yet, it's… merely ok, nothing special, even when compared to all humans including those oblivious of the fact they shouldn't be behind a wheel.

Re: Study urges caution when comparing neural networks to the brain

#76

Post by ben_w »

elcomet wrote: post #74
marmada wrote: post #65

Earlier quoted context omitted.

Humans are trained on petabytes of data. From birth, we ingest sights, sounds, smells etc. Imagine a movie of every second of your life. And an audio track of every second of your life. Etc. Etc. Humans get a lot of data.

And you didn't even count the data from million of years of evolution. The brain doesn't come as a blank slate when you're born.

That's literally what I was saying when I wrote "our biases are how evolution managed to do that".

Re: Study urges caution when comparing neural networks to the brain

#77

Post by l33tman »

ffwd wrote: post #67
bawolff wrote: post #38

The aspect of ai that makes me think something related is going on, is how artifacts look in image generation systems like stable diffusion. Often these systems will have really bizzare artificats, people with 3 arms, etc. However at the same time when you glance at the output without looking carefully you will sometimes miss these artifacts even though they should be absolutely glaring.

Not sure if I'm missing a subtle nuance in your point but to me those "artifacts" are completely expected. Those artifacts like 3 arms are the patterns / outputs in the model, but since it doesn't have a fundamental understanding of the patterns/objects like arms, it just blends many images of arms together and create things like 3 arms. Also why there are so many eyes, arms, legs and other things in other generative…

"It just spits out the training set in random configurations (ish)." is a pretty gross misrepresentation and oversimplification of how such a model works, akin to saying a human artist only spits out whatever they saw earlier in their life in random configurations, or saying that SD only spits out pixel values it has seen before, or combinations of pixel values that form edges, etc.

FWIW I don't think there is anything particularly wrong in the model architectures or training data that in some fundamental way makes it impossible to always get 2 arms. After all, lots of other tricky things are almost always correct. I suspect it's a question of training time and model size mostly (not trivial of course as it's still expensive to re-train to check modified architectures etc). It's also a matter of diffusion sampling iterations and choice of sampler at inference time, for the case of SD.

Re: Study urges caution when comparing neural networks to the brain

#79

Post by tabtab »

retrac wrote: post #69
lalos wrote: post #28

Here's my guess: neurons tap into quantum mechanics but we are too primitive to understand that for now. The brain was initially modeled as humors/fluids back when we developed aqueducts, then telegraph came into the scene and it was modeled as electrical impulses and now computers/ML are popular therefore we see it as a neural network. Next step is quantum.

It's certainly not proven, but there are many hints in that direction, and the hints keep piling up. Recent research [1] on how the classic anaesthetics work (a great mystery!) suggests they operate by inhibiting the entanglement of pairs of electrons in small molecules which split into free radicals, the electrons then physically separated but still-entangled. It seems it is at least possible, that there is speed-of…

We don't currently fully know how anesthetics work largely because we don't really know how the human brain works on a large scale. We'd have to solve that before seriously proposing quantum effects. In other words, it's too early to rule out classic physics and chemistry as the brain's primary mechanism. (Although solving how it works could first solving quantum mysteries, but Occam's razor is classic rules in my opinion.)

Re: Study urges caution when comparing neural networks to the brain

#80

Post by Retric »

dekhn wrote: post #50
Retric wrote: post #42

Earlier quoted context omitted.

If the Brian is using some physics we don’t understand that’s something new not Quantum Mechanics. QM a specific theory of how the world operates, if something else is involved it doesn’t fall under that theory it’s [insert new theory’s name here]. I really don’t get why everyone wants the Brian to operate on some new QM effect other than peoples perception that a 100 year old theory is somehow cutting edge, spooky,…

I think you're probably missing a number of the important details. In the Penrose/Hammerof model, they're explicitly saying that humans are observed to generate problem solutions that could not have been generated by a purely classical computing process, therefore, the brain must exploit some specific quantum phenomenon. When you talk about QM a a theory of how the world operates, there are wide ranges of QM. Everyth…

The Brian using entanglement would completely destroy modern physics as we know it, the effect on biology would be tiny by comparison.

Your other points are based on such fundamental misunderstanding that it’s hard to respond. Saying something isn’t the output of classical computing processes while undemonstrated, is then used to justify saying they must therefore use Quantum Phenomenon. But logically not everything that is either classical or Quantum so even that logical inference is unjustified. Logically it’s like saying well it’s not a soda so it must be a rock.

PS: If people where observed to solve problems that can’t be solved by classical computer processing that would be a really big deal. As in show up on Nightly News, and win people Nobel prizes big. Needless to say it hasn’t happened.

Post reply on HN