News · 2026-08-15

Compressed memory stretched a 7,000-token model to 1.75 million

A long-context framework trained on 7,000-token inputs kept working at 1.75 million tokens, a 250-fold extrapolation, by refusing to read most of them. The method, from a team led by Zhuoen Chen, compresses each chunk of a long document into memory blocks, uses a learned gate to pick only the relevant blocks, and trains the compressor and the reasoning model together with reinforcement learning. It reports up to half the peak GPU memory and six times the inference speed of MemAgent, a strong existing baseline.

Key facts

• Context extrapolation from 7,000 tokens to 1.75 million, roughly 250x beyond the training length.
• Up to a 2x reduction in peak GPU memory and a 6x inference speedup over MemAgent.
• Competitive accuracy on multi-hop reasoning benchmarks including RULER-HQA.
• Posted 9 February 2026 as arXiv:2602.08382, 26 pages. Hugging Face paper page, where the authors say code and models will be released.

There are three standard ways to make a language model handle a document longer than it was built for, and all three have a known failure. You can enlarge the context window, which is expensive because attention cost grows with the square of the length, and which degrades as models forget material in the middle. You can use retrieval-augmented generation, fetching only the passages that seem relevant, which is cheap but hands the model disconnected fragments with the connective tissue stripped out. Or you can summarize as you go, which loses whatever the summarizer did not think was important.

The authors name all three problems directly in their opening: "quadratic computational costs, information forgetting, and the context fragmentation inherent in retrieval-augmented generation." Their answer borrows its shape from how people read a long report. Nobody holds a 400-page document in working memory. You read it in passes, keep compressed impressions of each section, and when someone asks a question you pull back the two or three sections that matter and reason over those in detail.

Mechanically that becomes four parts. A segmenter splits the long input into chunks. A learned compressor encodes each chunk into compressed memory representations rather than raw tokens. A gating module, trained separately as a classifier, decides which memory blocks are relevant to the current question. A reasoning module then iterates over the selected blocks, maintaining an evolving working memory as it goes.

The part that distinguishes this from the many bolt-on memory layers of the past two years is the training. The compressor and the reasoner are optimized jointly, end to end, with reinforcement learning. That means the compressor is not trained to write good summaries in the abstract; it is trained to write whatever representation makes the downstream reasoner answer correctly. Anyone who has watched a summarization pipeline discard exactly the detail the next stage needed will recognize why that coupling matters.

The efficiency numbers follow from the architecture rather than from a clever kernel. Because the reasoner only ever sees a handful of compressed blocks, the amount of material it attends to stays roughly constant no matter how long the source document is. That is what makes the 250-fold extrapolation possible at all: the system was never trained to attend across 1.75 million tokens, and it never has to.

The caveats are real and the authors do not oversell them. Extrapolating to 1.75 million tokens is a claim that the architecture keeps functioning at that length, not that its reasoning quality holds up there, and multi-hop question-answering benchmarks are forgiving of a system that finds the right two chunks and ignores the rest. The efficiency comparison is against MemAgent, another memory-based system, rather than against a frontier model reading the full document. And the code is not out. The arXiv comments say code and models will be released, and the Hugging Face submitter comment says the same, but until a repository exists nobody outside the group has reproduced a 6x speedup.

Where it fits: the long-context field spent 2024 and 2025 competing on window size, and the numbers stopped being informative once every lab claimed a million tokens. The interesting work has moved to what a model does with a window it cannot afford to fill. This paper, the SkillsVote work on governing what enters an agent's skill library, and SkillZip's compression of skill files are all versions of the same shift: the constraint is no longer how much a model can read, it is what deserves to be read.

Primary source, verified: read the paper → (arXiv 2602.08382)

Ground Truth. (2026, August 15). Compressed memory stretched a 7,000-token model to 1.75 million. Ground Truth. https://groundtruth.day/news/compressed-memory-stretched-7000-tokens-to-1-75-million.html

@misc{groundtruth:compressed-memory-stretched-7000-tokens-to-1-75-million,
  title  = {Compressed memory stretched a 7,000-token model to 1.75 million},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/news/compressed-memory-stretched-7000-tokens-to-1-75-million.html}
}

Topics: long-context · memory · reinforcement-learning · efficiency · research