Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computational Model Reveals How the Brain Manages Short-Term Memories

neurosciencenews.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#11

Post by lamename »

earthboundkid wrote: post #3

> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent ne…

As a neuroscientist, I can tell you with high confidence that many if not most neuroscientists would agree with you.

You might also be interested in "Could a Neuroscientist Understand a Microprocessor?"

https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#12

Post by bra-ket »

subroutine wrote: post #9
bra-ket wrote: post #7

here's a free version https://scihubtw.tw/https://www.nature.com/articles/s41593-0... Looks like another paper on how everything in the brain can be tied to RNN or GPT2 or some other trendy ML model. 'Cure cancer with deep learning' style. Before something like RNN can be used in a model of biological memory it has to respect the constraints of biological memory. Taking an arbitrary model and twisting the arm of data…

I agree with this sentiment. To be fair though, the authors did attempt to generate a biologically plausible RNN, which they published in a previous article... https://www.pnas.org/content/pnas/116/45/22811.full.pdf I'm not fully sold on this model, but the article is fairly dense and I dont fully understand it. (Sejnowski, the senior author, was on my dissertation committee; he fell asleep during my defense lol)

Oh god, what he calls constraints are the "spiking nature of biological neurons", then he happily proceeds, just like you'd guess, to create this abomination of 'spiking RNN' and train this pretty much plain vanilla RNN via backpropagation, completely disregarding the energy spent, learning mechanism, and actual biological plausibility or relevance of this horrendous architecture as memory model.

The first author knows just enough to be dangerous.

>Sejnowski fell asleep

it surely looks like it

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#13

Post by amirkdv »

lamename wrote: post #11
earthboundkid wrote: post #3

> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent ne…

As a neuroscientist, I can tell you with high confidence that many if not most neuroscientists would agree with you. You might also be interested in "Could a Neuroscientist Understand a Microprocessor?" https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

I can't recommend this paper enough to anyone interested in neuroscience.

It's a thoughtfully executed set of experiments that IMHO should be part of every computational neuroscientist's training.

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#14

Post by trevyn »

h-1 wrote: post #6

Anyone can point out that autistic, schizophrenic or very old people lack certain inhibitions. But it's downright fascinating to think it may be explained by dulled inhibitory neurons or hypersensitive "regular" ones. That could be an easy fix!

>That could be an easy fix!

Famous last words?

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#15

Post by whymauri »

Glad to see more work in the 'stable memories' camp win out against the 'the brain is too stochastic and therefore unstable' theories. As a complement to this work on short-term memory, here's one of my favorite papers on long-term memories!

https://science.sciencemag.org/content/365/6455/821.abstract

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#16

Post by goldenshale »

subroutine wrote: post #8
babesh wrote: post #2

What do these inhibitory neurons do to manage short-term memories?

This is explained in the Nature article (summary on page 133) and shown as a diagrams in figure 6... > In summary, these findings imply that robust inhibition of oppositely tuned inhibitory subpopulations is critical for memory maintenance in our RNN model. For example, a positive cue stimulus activates the inhibitory and excitatory subgroups selective for that stimulus and deactivates the negative stimulus subgroups…

My intuitive understanding of these findings is that short term memory works by maintaining a consistent firing pattern in a particular region of the prefrontal cortex. By default a given firing pattern might activate other patterns because neurons tend to be a part of multiple potential pattern groups, and so left unchecked it could result in an unstable, random walk type of sequence. To stay in the attractor of one memory though, inhibitory neurons are used to turn off everything that isn't a part of the memory, thereby allowing it to remain in a more steady state.

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#17

Post by subroutine »

goldenshale wrote: post #16
subroutine wrote: post #8

Earlier quoted context omitted.

This is explained in the Nature article (summary on page 133) and shown as a diagrams in figure 6... > In summary, these findings imply that robust inhibition of oppositely tuned inhibitory subpopulations is critical for memory maintenance in our RNN model. For example, a positive cue stimulus activates the inhibitory and excitatory subgroups selective for that stimulus and deactivates the negative stimulus subgroups…

My intuitive understanding of these findings is that short term memory works by maintaining a consistent firing pattern in a particular region of the prefrontal cortex. By default a given firing pattern might activate other patterns because neurons tend to be a part of multiple potential pattern groups, and so left unchecked it could result in an unstable, random walk type of sequence. To stay in the attractor of one…

To further clarify, I believe the authors are advocating a WM model where an excitatory subnet (activated by a Positive cue) sustains its firing (in the absence of that cue; the Delay period) by activating inhibitors of its own inhibitors. (these inhibitory subnets, like goldenshale mentions, also seem to inhibit excitatory subnets, diffusely).

model diagram: https://i.ibb.co/nrh1pVJ/figs.png

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#18

Post by l33tman »

earthboundkid wrote: post #3

> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent ne…

I guess neuroscience mostly compares to the condensed matter / semiconductor field, in the computer analogy. They know everything about the transistors but guess at why they're connected like they are. And some in their respective fields will also be very interested in the high-level connections although maybe with less depth..

Unfortunately, in the neuroscience area, there is not yet any analogy to a computer programmer.

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#19

Post by seesawtron »

earthboundkid wrote: post #3

> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent ne…

> Neuroscience is so laughably primitive.

Its hard to comprehend how complex of a machinery the brain is when you are "just" a computer scientist. I often see people from the AI field eager to do a short rotation in a neuroscience lab to "learn how the brain works" and then go back to their company and develop a novel algorithm to make a paradigm shift in AI field - it's enthusiastic but laughably primitive.

The technology available to understand this complex machine is primitive when you compare it to electronics or AI tech. This is not because of the lack of funding and talent being invested in the field but because of the highly complex and efficient nature of the brain that has evolved over millions of years but only started to be investigated anatomically for less than a couple of centuries. Arguably, the same problem exists in the AI field too when it comes to understanding the black-box models. We don't even entirely comprehend how the artifical neural network based models work in many use-cases; there are ideas floating around to make sense of it which use the same strategy of "plug in/out and play" to understand what parameter affects what in your model which to neuroscientists seems (yet again) laughably primitive.

Re: Computational Model Reveals How the Brain Manages Short-Term Memories

#20

Post by esc_colon_q »

seesawtron wrote: post #19
earthboundkid wrote: post #3

> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent ne…

> Neuroscience is so laughably primitive. Its hard to comprehend how complex of a machinery the brain is when you are "just" a computer scientist. I often see people from the AI field eager to do a short rotation in a neuroscience lab to "learn how the brain works" and then go back to their company and develop a novel algorithm to make a paradigm shift in AI field - it's enthusiastic but laughably primitive. The tech…

And people from neuro come into AI for short rotations pretty regularly to do astoundingly naive work there which amounts to nothing at all.

Maybe both fields could stop flinging shit and realize that they are both missing a key piece of the puzzle...

Post reply on HN