A sleep-like consolidation mechanism for LLMs

Post by pfdietz » Tue, May 26, 2026, 5:39 PM UTC

famouswaffles wrote: post #12

pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

Anthropomorphization is not inherently wrong, and in some instances, it actually lets you reason better about about complex behavior than whatever convoluted (and often wrong, especially in the case of giant neural networks) mechanistic description one might conjure. Here the analogy isn't without reason.

Post by selfsimilar » Tue, May 26, 2026, 5:41 PM UTC

sillysaurusx wrote: post #65

burnte wrote: post #57

Earlier quoted context omitted.

HIV doesn't kill you, but it creates circumstances where other things will. Sleep is the same. You may not die from lack of sleep, but you die from the things it can cause. Effectively there's no difference.

I’m shocked by how careless everyone here is about their definitions, and their science. Sleep isn’t the same as HIV. It’s in fact so hard to kill something with a lack of sleep that it’s never once been observed in vertebrates outside of one specific rat study, and that rat study couldn’t conclusively identify sleep as the cause of death. For something so incredibly difficult to do (die from lack of sleep) it’s fran…

Post by hajile » Tue, May 26, 2026, 5:44 PM UTC

sillysaurusx wrote: post #43

pcrh wrote: post #20

Earlier quoted context omitted.

The function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?

> If an animal can't sleep it will eventually die. That turns out to be un-settled science. No human has ever died from lack of sleep. People point to “fatal familial insomnia” as a counterexample. But they die to the disease, not the lack of sleep. In a series of controlled experiments, rats and fruit flies did die from lack of sleep. But no one has yet proven that it holds true for vertebrates except for rats. In o…

Post by CuriouslyC » Tue, May 26, 2026, 5:54 PM UTC

pcrh wrote: post #4

I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities. Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?

Post by micromacrofoot » Tue, May 26, 2026, 5:58 PM UTC

logicchains wrote: post #71

micromacrofoot wrote: post #24

To reach a more brain-like behavior LLMs need to integrate your inputs into their model dynamically, essentially retraining real-time based on the most salient input. Human brains do this selectively all the time and it's part of our plasticity. Biologically humans do similar compression, so introducing a similar concept to an LLM also feels reasonable. Hardware isn't fast/cheap enough to do this on an ongoing basis,…

>To reach a more brain-like behavior LLMs need to integrate your inputs into their model dynamically, essentially retraining real-time based on the most salient input. That's already possible with LLMs. The challenge is that 1. it would allow permanently jail-breaking models and 2. there'd be no way for them to efficiently transfer what they'd learned to a new model generation.