News · 2026-07-18

LongStraw Makes Million-Token RL Training Possible on 8 GPUs - But Its Code Doesn't Run Yet

A systems paper called LongStraw demonstrates that reinforcement-learning post-training can execute on shared prompts of more than 2 million tokens using only 8 GPUs, a scale that has been out of reach for RL training even as inference systems routinely handle million-token contexts. Posted to arXiv (2607.14952) and currently the #1 paper of the day on Hugging Face, the work does this by scoring the giant shared prompt once with no gradient tracking at all, then replaying only the short, trainable response separately. The authors are careful to call this a demonstration of execution capacity, not a proof that every part of the training math is complete and correct.

Key facts

• On 8 H20 GPUs, LongStraw's grouped scoring plus response backward pass reaches 2.1 million positions for response groups of both 2 and 8, adding only about 0.21 GB of extra peak memory for the larger group. (arXiv 2607.14952)
• A stress test pushes the same pipeline to 4.46 million positions, and a 32-GPU run validates the full end-to-end path for a 2.1-million-token prompt across all 78 layers of GLM-5.2.
• The paper is #1 Paper of the Day on the Hugging Face paper page, with 172 upvotes.
• The accompanying code, hosted at the LongStraw GitHub repo, is explicitly marked not runnable and has drawn only about 7 stars so far.

For the last couple of years, the AI industry has been on two separate tracks. Inference - just running a trained model to answer questions - has scaled to enormous context windows, with some systems now reading and reasoning over a million tokens or more in a single pass. Training, and specifically the reinforcement-learning step where a model practices generating responses and gets rewarded for good ones, has lagged far behind, often capped around 256,000 tokens. Models trained at that shorter length are then simply hoped to generalize to the longer contexts they'll face once deployed - a real gap between how a model is taught and how it's actually used.

The reason RL training falls so far behind inference is memory, not model design. To reinforcement-learn on a response, a training run traditionally has to hold the entire prompt plus a full gradient graph - the bookkeeping needed to later compute how every part of the output should be adjusted - in GPU memory at once. A team publishing the work under the GitHub organization MindLab Research, and instantiating their method with GRPO (a widely used reinforcement-learning post-training recipe built around comparing groups of sampled responses, explained further in our group relative policy optimization lesson), found a way to avoid carrying that full graph.

LongStraw's trick is to split the work in two. First, it reads the entire shared prompt - the long context every response in a batch has in common - exactly once, and does so without building any gradient graph at all, since the prompt itself is never what gets updated. It keeps only the specific internal state the model will need to make sense of what comes next. Then, for each short response the model generated, it replays that response one at a time, this time with full gradient tracking switched on, so the training signal only ever has to flow through the short piece of text actually being learned from. Response groups are processed one after another rather than all at once, with their gradients accumulating into the same small set of trainable parameters, which keeps the size of the live "under construction" graph independent of how many responses are in a group.

It's a bit like being handed a hundred-page brief before writing a short memo. You read the whole brief once, quickly, taking only the notes you'll actually need - you're not tracking exactly how every sentence of the brief influenced your thinking, because the brief itself isn't what you're being graded on. Then you go back and carefully draft your memo, word by word, keeping careful track of your own reasoning only for that much shorter piece of writing. LongStraw trades some extra time re-reading for a training process that never has to hold the whole brief and its reasoning trail in memory simultaneously.

If it holds up, this matters because it removes a real bottleneck standing between "models that can read very long documents" and "models that can be reinforcement-learned to actually reason well over those documents," rather than just generalizing to them after the fact from shorter training. The honest caveat is significant, though: LongStraw's authors are upfront that these results establish execution capacity - that the pipeline runs, fits in memory, and scales - rather than complete, verified training correctness, since some distributed forward and gradient-composition paths remain incomplete and the captured prompt state is detached. The public repo backs that framing up bluntly, shipping as review_only_not_runnable under a README that itself is titled "MinT-2M: 2M-Context Response-Only GRPO." A visible comment on the paper's Hugging Face discussion also flags a real limitation: real-world agent trajectories aren't always a clean prompt-then-generation shape, since tool outputs can arrive mid-trajectory, which muddies the clean shared-prompt assumption LongStraw is built on. LongStraw is a genuine capacity result and a promising direction for closing the training-context gap - it just isn't, yet, a system anyone can download and run.

Primary source, verified: read the paper → (arXiv 2607.14952)

Ground Truth. (2026, July 18). LongStraw Makes Million-Token RL Training Possible on 8 GPUs - But Its Code Doesn't Run Yet. Ground Truth. https://groundtruth.day/news/longstraw-million-token-rl-post-training.html

@misc{groundtruth:longstraw-million-token-rl-post-training,
  title  = {LongStraw Makes Million-Token RL Training Possible on 8 GPUs - But Its Code Doesn't Run Yet},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/longstraw-million-token-rl-post-training.html}
}

Topics: rl-post-training · long-context · efficiency · grpo · systems