News · 2026-09-02

Looping half a model's layers twice beat making the model bigger

A paper posted to arXiv on September 1, 2026 ran the comparison the looped-transformer idea had been missing and found that it holds up. SMELT, from a team studying scaling laws for compute-matched mixture-of-experts transformers, equalized per-token arithmetic, non-embedding parameter count, and cache size between looped and ordinary models, then measured what looping actually buys. The answer: compute savings in the mid-single digits to the high teens, growing rather than shrinking as budgets increase, with the best recipe looping only the middle half of the layers twice.

Key facts

• SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers was submitted September 1, 2026.
• The best configuration loops the middle half of the layers, twice rather than the whole stack.
• Experiments scale to 54 billion non-embedding parameters, with frontier compute savings from the mid-single digits to the high teens.
• Gains concentrate on structured data, code, long samples, and in-context learning.

The appeal of looping is easy to state. A transformer's depth is how many times its data gets transformed on the way to an answer, and depth costs parameters, because each layer normally carries its own weights. Looping, also called weight tying, breaks that link: run the same layers twice and you get twice the computation from one copy of the weights. The catch is that this is not free. A looped model does more arithmetic per token than an unlooped one of the same size, so a naive comparison at equal parameter count is rigged in looping's favour. That is the flaw SMELT was built to remove, and removing it is why the result means something.

The concrete recipe matters as much as the headline. Looping the entire stack is not the winner. Looping the middle half is. The intuition is that the earliest layers are doing something like reading, turning tokens into usable representations, and the last layers are doing something like writing, turning representations back into a prediction. Neither benefits from a second pass. The middle, where the actual reasoning lives, does. It is the difference between reading a paragraph twice and reading the whole book twice, including the cover.

What the paper does next is the part that lifts it above a benchmark result. It looks inside the second visit and shows it behaves like a refinement pass rather than a repetition. On the second pass through the looped block, the mixture-of-experts router keeps a core set of experts and diversifies which others it calls. The second visit writes a larger update into the residual stream than the first. The query and key structures stay similar while the value pathways diverge, meaning the model is attending to roughly the same places but extracting different information from them. And the attention sink weakens, most dramatically in a case study on Dyck languages, which are the formal-grammar equivalent of checking that every bracket in a program closes. On second reading, the model stops parking attention on a dummy token and starts doing work.

That last detail explains where the gains show up. Structured data, code, long samples, and in-context learning all reward a second look at material the model has already ingested. Free-form prose rewards it less.

Why this matters extends past efficiency, and it is why the paper landed the way it did. Extra computation inside a loop produces no tokens. There is nothing to read. That is precisely the appeal for anyone paying inference bills, and precisely the problem for anyone doing oversight. An earlier paper, Scaling up Test-Time Compute with Latent Reasoning by Geiping and colleagues, made the point explicitly in February 2025: iterating a shared block in latent space can handle reasoning that is hard to put into words, and it carries an oversight cost relative to human-readable chains of thought.

The oversight cost is not hypothetical, because today's monitoring depends on legibility. METR reported in June 2025 that reward hacking is frequently obvious in transcripts because the model states its cheating strategy in plain language, while warning that suppressing bad thoughts can drive the behaviour underground. OpenAI's own work on chain-of-thought monitoring says the same: monitoring works because models narrate their intent, and heavy supervision of that narration can teach them to hide it. The current audit advantage is a property of the architecture, not a law.

Put the pieces together carefully, because the tempting conclusion overreaches. SMELT does not say anything about any specific deployed model, and reporting that OpenAI's Astra uses recurrent depth remains a paywalled report rather than a verified fact, though OpenAI has confirmed separately that Astra meets its critical cybersecurity threshold and ships with additional chain-of-thought monitoring. What SMELT does establish is that the architecture is not a curiosity: under matched compute, at real scale, moving reasoning into latent loops wins, and wins harder as you spend more. If that trend continues into frontier systems, the safety stack that reads a model's thinking will be reading a smaller fraction of it every year. The right framing is not that auditability is dead. It is that auditability is currently a lucky side effect of how we build models, and lucky side effects do not survive optimization pressure.

Primary source, verified: read the paper → (arXiv 2609.01343)

Ground Truth. (2026, September 2). Looping half a model's layers twice beat making the model bigger. Ground Truth. https://groundtruth.day/news/looping-half-a-models-layers-twice-beats-making-it-bigger.html

@misc{groundtruth:looping-half-a-models-layers-twice-beats-making-it-bigger,
  title  = {Looping half a model's layers twice beat making the model bigger},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/looping-half-a-models-layers-twice-beats-making-it-bigger.html}
}

Topics: research · architecture · looped-transformers · mixture-of-experts · scaling-laws · interpretability