Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A sleep-like consolidation mechanism for LLMs

arxiv.org

Post reply on HN
Print view View original

21–30 of 155 posts

Re: A sleep-like consolidation mechanism for LLMs

#21

Post by colechristensen »

delis-thumbs-7e wrote: post #19
colechristensen wrote: post #13

Earlier quoted context omitted.

No, they're actually training weights based on context before compaction. Context is context, this is splitting the model into persistent weights and malleable ones which are periodically updated.

Wouldn’t that be extremely computationaly expensive considering how resource incentive training is?

No, training a state of the art model involves training on the order of 10 trillion tokens.

We're talking about a step that updates weights based on say between 10k and 1M tokens.

Re: A sleep-like consolidation mechanism for LLMs

#22

Post by delis-thumbs-7e »

colechristensen wrote: post #21
delis-thumbs-7e wrote: post #19

Earlier quoted context omitted.

Wouldn’t that be extremely computationaly expensive considering how resource incentive training is?

No, training a state of the art model involves training on the order of 10 trillion tokens. We're talking about a step that updates weights based on say between 10k and 1M tokens.

I learned something. Thank you!

Re: A sleep-like consolidation mechanism for LLMs

#23

Post by rahen »

That's an idea I had a few months ago: after going through a compaction once the KV cache is nearing capacity, accumulate this knowledge into a dataset to fine-tune a LoRA during offline hours.

This would create a three-layer memory system:

- Stable long-term memory (initial base weights)

- Mid-term memory built from the compactions and replay buffers

- Short-term memory (KV cache)

Sleeping would just be a fancy term for consolidating and transferring information from one memory layer to another during offline hours. Maybe that's also what the brain does while sleeping.

Re: A sleep-like consolidation mechanism for LLMs

#24

Post by micromacrofoot »

To reach a more brain-like behavior LLMs need to integrate your inputs into their model dynamically, essentially retraining real-time based on the most salient input. Human brains do this selectively all the time and it's part of our plasticity.

Biologically humans do similar compression, so introducing a similar concept to an LLM also feels reasonable. Hardware isn't fast/cheap enough to do this on an ongoing basis, similar to how it's too expensive for our brains to do this while we're moving through the world.

All we have now most of the time in LLMs is "working memory" we're missing a lot of the functionality that allows for episodic memory and selective plasticity.

The more you read about how human brains work, the more you realize that we may have figured out a piece with LLMs, but it's certainly nothing approaching AGI. People insisting so are blowing smoke for investor hype or don't understand a big piece of the concepts involved.

Re: A sleep-like consolidation mechanism for LLMs

#25

Post by simonw »

pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

> When I reboot a computer, is that equivalent to a nap?

I mean, you do put your computer into "sleep" mode and then "wake" it.

Analogies are useful. I think we need to learn how to continue to benefit from them despite the risk of anthropomorphication.

Re: A sleep-like consolidation mechanism for LLMs

#26

Post by [deleted] »

pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

[deleted]

Re: A sleep-like consolidation mechanism for LLMs

#27

Post by bayarearefugee »

pcrh wrote: post #20
djeastm wrote: post #14

Earlier quoted context omitted.

They provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough…

The function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?

> Does their LLM "die" if it can't perform the function described?

If you don't periodically clean the context, an LLM effectively goes insane in terms of outputs.

If the LLM were fully controlling a physical system (like a robot body) that contained it the resulting insanity of an ever-growing, never cleaned context would likely result in some sort of death-like event.

Re: A sleep-like consolidation mechanism for LLMs

#28

Post by chermi »

rahen wrote: post #23

That's an idea I had a few months ago: after going through a compaction once the KV cache is nearing capacity, accumulate this knowledge into a dataset to fine-tune a LoRA during offline hours. This would create a three-layer memory system: - Stable long-term memory (initial base weights) - Mid-term memory built from the compactions and replay buffers - Short-term memory (KV cache) Sleeping would just be a fancy term…

Wouldn't that just accelerate collapse? How much do you trust the outputs of the llm to provide trustworthy and valuable new information? I mean I understand distillation works. But that's much more structured and thoughtful than my sessions at least.

Re: A sleep-like consolidation mechanism for LLMs

#29

Post by SR2Z »

cowlby wrote: post #8
pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

The analogy is helpful, but yes we should be able to “intelligently design” something better than sleep analogues since we’re not constrained by evolution like in humans.

Evolution constrains the evolution of human beings, but it's also excellent at discovering elegant designs that work very reliably at a low cost.

Maybe someday we'll understand the way our minds work well enough to design from first principles but until then we've only got one template for how a thinking machine should look.

Re: A sleep-like consolidation mechanism for LLMs

#30

Post by Windchaser »

pcrh wrote: post #20
djeastm wrote: post #14

Earlier quoted context omitted.

They provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough…

The function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?

I don't think it's necessarily correct to think of sleep in terms of "it is necessary for animals or they will die". It might be more useful to think of it as "it was so useful that animals who slept outcompeted all the animals who didn't".

Meaning: it might just provide a big advantage.

I don't want to overextend and assume that any advantage extends to LLMs. That rest-and-recuperate advantage might also extend to LLM-based AIs. Or maybe not, and the rest-and-recuperate is mainly useful for biology-based organisms. But there is some logic to it.

> The function of sleep in animals is largely obscure.

In my understanding, it's well-understood that sleep is used to consolidate and store long-term memories (amongst other functions, like cell and muscle repair). They've found this memory-consolidation-during-sleep even in relatively simple animals like bees.

Post reply on HN