Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A sleep-like consolidation mechanism for LLMs

arxiv.org

Post reply on HN
Print view View original

31–40 of 155 posts

Re: A sleep-like consolidation mechanism for LLMs

#31

Post by DonHopkins »

famouswaffles wrote: post #12
pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

Anthropomorphization is not inherently wrong, and in some instances, it actually lets you reason better about about complex behavior than whatever convoluted (and often wrong, especially in the case of giant neural networks) mechanistic description one might conjure. Here the analogy isn't without reason.

Is it "Anthropicmorphization" when Claud treats human beings like LLMs?

Re: A sleep-like consolidation mechanism for LLMs

#32

Post by verisimi »

pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

... and anyway, maybe it was hungry? Or getting the sniffles?

Re: A sleep-like consolidation mechanism for LLMs

#33

Post by adastra22 »

pcrh wrote: post #20
djeastm wrote: post #14

Earlier quoted context omitted.

They provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough…

The function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?

There is a lot that is known about sleep. We don't know everything and there are large gaps in our knowledge, but there is also a lot that we do know. And this research explicitly tried to emulate the things we know that sleep does do. Calling it "sleep" is warranted, imho.

Re: A sleep-like consolidation mechanism for LLMs

#34

Post by DonHopkins »

rahen wrote: post #23

That's an idea I had a few months ago: after going through a compaction once the KV cache is nearing capacity, accumulate this knowledge into a dataset to fine-tune a LoRA during offline hours. This would create a three-layer memory system: - Stable long-term memory (initial base weights) - Mid-term memory built from the compactions and replay buffers - Short-term memory (KV cache) Sleeping would just be a fancy term…

It's a network of computers with GPUs, so there's no reason it can't sleep at the same time it's awake. Just a continuous "sleeping" process going on in the background, incrementally updating the model. No need for the "thinking" process to be "unconscious" while the "sleeping" process runs. Anthropomorphism confuses everything. There's no such thing as "offline hours" because the Earth is a sphere and the United States is not the center of the universe.

Re: A sleep-like consolidation mechanism for LLMs

#35

Post by order-matters »

djeastm wrote: post #14
pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

They provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough…

but isnt sleep an already defined technical term for significantly reducing power consumption while preserving its state until woken up?

i feel like its confusing to reuse the word for a process that aims to deliberately change state of the machine / process

Re: A sleep-like consolidation mechanism for LLMs

#36

Post by jack_pp »

chermi wrote: post #28
rahen wrote: post #23

That's an idea I had a few months ago: after going through a compaction once the KV cache is nearing capacity, accumulate this knowledge into a dataset to fine-tune a LoRA during offline hours. This would create a three-layer memory system: - Stable long-term memory (initial base weights) - Mid-term memory built from the compactions and replay buffers - Short-term memory (KV cache) Sleeping would just be a fancy term…

Wouldn't that just accelerate collapse? How much do you trust the outputs of the llm to provide trustworthy and valuable new information? I mean I understand distillation works. But that's much more structured and thoughtful than my sessions at least.

We can trust the feedback we give it based on the output it provides.

Re: A sleep-like consolidation mechanism for LLMs

#37

Post by ben_w »

pcrh wrote: post #20
djeastm wrote: post #14

Earlier quoted context omitted.

They provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough…

The function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?

> If an animal can't sleep it will eventually die.

Very few animals fail to eventually die even with as much sleep as they want.

But before death, there is a loss of cognitive function from sleep deprivation, and we observe this too with AI whose context windows get too full.

While we don't know very much about sleep, my understanding is that we do have a long list of things that we do during it, we just don't really understand if sleep is necessary for each of them or simply a convenient opportunity for it.

There's lots of things biology does in response to easy-to-detect proxy signals instead of the real thing they care about: Our sensation of needing to breathe more is based on too much carbonic acid in our blood, not lack of oxygen, which is why in general nobody is allowed in an elevator with a liquid nitrogen dewar; Our natural distaste for incest is based on who we grew up with, not our actual DNA; Get too cold and some people suddenly feel warm and want to (and some do) take all their clothes off even though that would just make them hypothermic even faster.

Being asleep may trigger the things we need to get done, but that doesn't mean sleep is *fundamentally* necessary for the things we need to get done. It could be just that it happens to be the way our biochemistry is wired, and we may find some other way to trigger those things.

The quotation given by djeastm would by my guess for what a dream is, and why we have them. But we don't spend all our time asleep, dreaming. And I'd be the first to say that my guess isn't worth much, as I'm not a brain scientist.

Re: A sleep-like consolidation mechanism for LLMs

#38

Post by palmotea »

pcrh wrote: post #20
djeastm wrote: post #14

Earlier quoted context omitted.

They provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough…

The function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?

> The function of sleep in animals is largely obscure.

Also, there's different kinds/stages of sleep, which probably perform different functions.

For instance, REM may do something like the GP describes, consolidating memories and processing learning. Deep sleep may do something else (I vaguely recall some stage of sleep is used by neurons to clear certain waste products).

Re: A sleep-like consolidation mechanism for LLMs

#39

Post by rahen »

chermi wrote: post #28
rahen wrote: post #23

That's an idea I had a few months ago: after going through a compaction once the KV cache is nearing capacity, accumulate this knowledge into a dataset to fine-tune a LoRA during offline hours. This would create a three-layer memory system: - Stable long-term memory (initial base weights) - Mid-term memory built from the compactions and replay buffers - Short-term memory (KV cache) Sleeping would just be a fancy term…

Wouldn't that just accelerate collapse? How much do you trust the outputs of the llm to provide trustworthy and valuable new information? I mean I understand distillation works. But that's much more structured and thoughtful than my sessions at least.

I was thinking of curated replay buffers, which would act like "dreams". To prevent collapse, the offline dataset would mix the new mid-term data with a baseline of anchor data (the original training distribution) so the model doesn't drift.

Also, we wouldn't train on the whole session. A separate critic module, like a reward model, would filter the KV cache to extract the high-value information, like a garbage collector before the LoRA.

That's just an idea though. Right now most research focuses on changing the architecture itself (TITAN, HOPE...) instead.

Post reply on HN