Language Models Need Sleep: Can LLMs Learn to Forget Less?
I recently read this paper titled “Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories.” This paper introduces a very nice concept for dealing with a problem we often face with language models: they are very good at using information in the moment, but continually transferring that information into stable long-term knowledge is much harder.
Currently, Large Language Models take user input in a sequential manner. As the amount of information grows, the model has to keep reasoning over an increasingly large context. We can take an analogy from how humans learn. For example, when we are in a long lecture, we often get overwhelmed by all the knowledge flowing into our brain. To deal with it, we usually take breaks, revise our notes, and revisit what we learned. Slowly, the information becomes more familiar and we start to form a stronger understanding of the topic.
Hence, the authors of the paper tried to implement a similar analogy for language models, and came up with a “Sleep” paradigm.
Sleep Architecture
The key idea is to separate learning into two phases: Wake and Sleep.
During the wake phase, the model interacts with new information and forms a short-term, fragile memory of it. This knowledge is useful immediately, but it is not necessarily stored permanently in the model’s parameters.
When the model enters sleep, this temporary knowledge is consolidated into more stable long-term parameters. The paper calls this process Memory Consolidation, and its main mechanism is Knowledge Seeding: knowledge from a smaller or short-term “self” is distilled into a larger network with more capacity to preserve it.
Sleep also contains a second stage called Dreaming. Here, the model generates synthetic examples based on what it has learned and uses them as a curriculum for further training. In other words, instead of simply replaying the original information, the model can generate its own practice material and use it to rehearse and refine its knowledge.
There is also an important idea of replay. Just as humans may revise older topics while learning something new, the model revisits previously learned information during sleep. This is important for continual learning because otherwise learning something new can overwrite older knowledge.
My Small Experiment
After reading the paper, I wanted to build a much smaller version of the same idea to understand how it works.
Instead of implementing the complete architecture from the paper, I created a toy experiment using DistilGPT-2 with LoRA adapters. The goal was to reproduce the basic learning cycle rather than reproduce every component of the original research.
I maintain two copies of the model:
Fast memory: temporary and optimized for quickly learning the new fact.
Slow memory: persistent and responsible for retaining knowledge across learning cycles.
The process looks roughly like this:
New information -> Fast learning -> Consolidation -> Dreaming + Replay -> Slow memory
When a new fact arrives, I first copy the current slow model into a fast model. The fast model is then fine-tuned on the new fact. I also verify that it actually learned the fact before allowing it to act as a teacher.
Next comes the sleep phase.
The fast model generates examples related to the new information. These generated examples, together with the original ground-truth information, are used to train the slow model. This is my simplified version of the paper’s memory-consolidation idea.
Then comes dreaming.
The slow model generates several possible answers about the new fact. I keep the generations that contain the correct information and use them as additional rehearsal data. I also replay previously learned facts so that the model does not focus only on the newest information.
Finally, the fast model is discarded.
This gives us a cycle like:
Wake -> Learn -> Sleep -> Consolidate -> Dream -> Replay -> Forget the fast memory
Then the next piece of information arrives, and the process starts again using the updated slow memory.
Results
Why is this interesting?
The interesting part is that we are no longer treating learning as simply:
Fact 1 -> Fact 2 -> Fact 3 -> Fact 4
Instead, we are introducing a periodic consolidation process:
Learn new information -> pause -> reorganize/rehearse knowledge -> continue learning
This is particularly interesting for continual learning. If the model keeps receiving new information indefinitely, we want the new information to become part of its long-term knowledge without destroying what it learned previously.
My experiment is obviously much smaller than the original paper. It uses only a few synthetic facts and a small model, so it should not be interpreted as evidence that the approach will automatically scale to large real-world knowledge streams.
But it provides a useful way to understand the core intuition behind the paper:
Learning does not have to happen only when the model is actively interacting with the world. A separate consolidation phase can be used to turn short-term experience into longer-term knowledge.
That raises a much bigger question:
What happens if we let a language model learn continuously, but also give it time to “sleep” or maybe effectively reorganize its memory?
That is the experiment I want to explore next.