Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inferring neural activity before plasticity for learning beyond backpropagation

nature.com

Post reply on HN
Print view View original

11–20 of 50 posts

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#11

Post by zby »

johnea wrote: post #8

Was a study really necessary for this? Do "AI" fanbois really think LLMs work like a biological brain? This only reinforces the old maxim: Artificial intelligence will never be a match for natural stupidity

I did not read the article - but I guess it all depends on the level of abstraction we are talking about. There is a very abstract level where you can say that AI learns like a biological brain and there is a level where you would say that a particular human brain learns in a different way than another particular human brain.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#12

Post by skissane »

nickpsecurity wrote: post #7

Some are surprised that anyone would make this point, either the title or the research. It might be a response to the many, many claims in articles that neural networks work like the brain. Even using terms like neurons and synapses. With those claims getting widespread, people also start building theories on top of them that make AI’s more like humans. Then, we won’t need humans or they’ll be extinct or something. M…

> It might be a response to the many, many claims in articles that neural networks work like the brain. Even using terms like neurons and synapses.

Artificial neural networks originated as simplified models of how the brain actually works. So they really do "work like the brain" in the sense of taking inspiration from certain rudiments of its workings. The problem is "like" can mean anything from "almost the same as" to "in a vaguely resembling or reminiscent way". The claim that artificial neural networks "work like the brain" is false under the first reading of "like" but true under the second.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#13

Post by anon291 »

johnea wrote: post #8

Was a study really necessary for this? Do "AI" fanbois really think LLMs work like a biological brain? This only reinforces the old maxim: Artificial intelligence will never be a match for natural stupidity

> Do "AI" fanbois really think LLMs work like a biological brain?

If you read the article you'd know two things: (1) the article explicitly calls out Hopfield networks as being more bio-similar (Hopfield networks are intricately connected to attention layers) and (2) the overall architecture (the inference pass) of the networks studied here remain unmodified. Only the training mechanism changes.

As for a direct addressing of the claim... if the article is on point, then 'learning' has a much more encompassing physical manifestation than was previously thought. Really any system that self optimizes would be seen as bio-similar. In both mechanisms, there's a process to drive the system to 'convergence'. The issue is how fast that convergence is, not the end result.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#14

Post by anon291 »

isaacimagine wrote: post #4

Wait, my brain doesn't do backprop over a pile of linear algebra after having the internet rammed through it? No way that's crazy /s tl;dr: paper proposes a principle called 'prospective configuration' to explain how the brain does credit assignment and learns, as opposed to backprop. Backprop can lead to 'catastrophic interference' where learning new things abalates old associations, which doesn't match observed bio…

This is like expressing surprise that a photon doesn't perform relativistic calculations on its mini chalkboard.

A simulation of a thing is not thing itself, but it is illuminating.

> pile of linear algebra

The entirety of physics is -- as you say -- a 'pile of linear algebra' and 'backprop' (differential linear algebra...)

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#15

Post by yongjik »

The title of the paper is: "Inferring neural activity before plasticity as a foundation for learning beyond backpropagation"

The current HN title ("Brain learning differs fundamentally from artificial intelligence systems") seems very heavily editorialized.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#16

Post by anon291 »

nickpsecurity wrote: post #7

Some are surprised that anyone would make this point, either the title or the research. It might be a response to the many, many claims in articles that neural networks work like the brain. Even using terms like neurons and synapses. With those claims getting widespread, people also start building theories on top of them that make AI’s more like humans. Then, we won’t need humans or they’ll be extinct or something. M…

> Even using terms like neurons and synapses. With those claims getting widespread, people also start building theories on top of them that make AI’s more like humans.

Except the networks studied here for prospective configuration are ... neural networks. No changes to the architecture have been proposed, only a new learning algorithm.

If anything, this article lends credence to the idea that ANNs do -- at some level -- simulate the same kind of thing that goes on in the brain. That is to say that the article posits that some set of weights would replicate the brain pretty closely. The issue is how to find those weights. Backprop is one of many known -- and used -- algorithms . It is liked because the mechanism is well understood (function minimization using calculus). There have been many other ways suggested to train ANNs (genetic algorithms, annealing, etc). This one suggests an energy based approach, which is also not novel.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#17

Post by robotresearcher »

The post headline is distracting people and making a poor discussion. The paper describes a learning mechanism that had advantages over backprop, and may be closer to what we see in brains.

The contribution of the paper, and its actual title is about the proposed mechanism.

All the comments amounting to ‘no shit, sherlock’, are about the mangled headline, not the paper.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#18

Post by skissane »

isaacimagine wrote: post #4

Wait, my brain doesn't do backprop over a pile of linear algebra after having the internet rammed through it? No way that's crazy /s tl;dr: paper proposes a principle called 'prospective configuration' to explain how the brain does credit assignment and learns, as opposed to backprop. Backprop can lead to 'catastrophic interference' where learning new things abalates old associations, which doesn't match observed bio…

> Backprop can lead to 'catastrophic interference' where learning new things abalates old associations, which doesn't match observed biological processes.

Most people find that if you move away from a topic and into a new one your knowledge of it starts to decay over time. 20+ years ago I had a job as a Perl and VB6 developer, I think most of my knowledge of those languages has been evacuated to make way for all the other technologies I've learned since (and 20 years of life experiences). Isn't that an example of "learning new things ablates old associations"?

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#19

Post by lukeinator42 »

It has been clear for a long time (e.g. Marvin Minsky's early research) that:

1. both ANNs and the brain need to solve the credit assignment problem 2. backprop works well for ANNs but probably isn't how the problem is solved in the brain

This paper is really interesting, but is more a novel theory about how the brain solves the credit assignment problem. The HN title makes it sound like differences between the brain and ANNs were previously unknown and is misleading IMO.

Re: Inferring neural activity before plasticity for learning beyond backpropagation

#20

Post by blackeyeblitzar »

The comments here saying this was obvious or something else more negative are disappointing. Neural networks are named for neurons in biological brains. There is a lot of inspiration in deep learning that comes from biology. So the association is there. Pretending you’re superior for knowing the two are still different, contributes nothing. Doing so in more specific ways, or attempting to further understand the differences between deep learning and biology through research, is useful.
Post reply on HN