Learn · Intermediate

Reinforcement Learning with Verifiable Rewards (RLVR)

Reinforcement Learning with Verifiable Rewards, or RLVR, is the training recipe behind the modern wave of reasoning models. The idea is simple: instead of asking humans to rate a model's answers, or training a separate network to guess what humans would like, you reward the model only when an automatic checker can confirm the answer is actually correct. Did the code pass its unit tests? Did the final number match the known solution? Did the proof pass a formal verifier? If yes, reward it; if no, do not. The term was popularized by the Tulu 3 team at Ai2 in late 2024, and the approach became famous as the engine of DeepSeek-R1.

To see why this matters, compare it with the method it partly replaces. Reinforcement learning from human feedback (RLHF) trains a reward model from thousands of human preference judgments, then optimizes the language model to score highly against that learned reward model. It works, but it has two weaknesses: collecting human preferences is slow and expensive, and the learned reward model is an approximation that a clever policy can learn to exploit - a failure mode known as reward hacking. RLVR sidesteps both problems in domains where truth is checkable. There is nothing to approximate: a math answer is either right or wrong, and a program either passes the tests or it does not.

The mechanism fits neatly on top of the same machinery used elsewhere in post-training. For a given problem, the model samples several candidate answers. Each answer is run through the verifier to produce a reward, typically a bare 1 or 0. Those rewards are then fed into a policy-gradient algorithm - most commonly GRPO today - which pushes the model toward the reasoning paths that tended to reach correct answers and away from the ones that did not. Because the model is learning from answers it generated itself, RLVR sits on the on-policy side of the spectrum. Over many iterations, the model discovers that longer, more careful chains of thought - checking its own work, trying multiple approaches, catching mistakes - reach verified-correct answers more often, so those behaviors get reinforced without anyone ever labeling a single reasoning trace.

An analogy: RLVR is like studying for a math exam with the answer key but no worked solutions. You cannot copy the reasoning, but every time you finish a problem you can check whether you got it right, and over hundreds of problems you learn which habits of thought reliably land on the correct answer. RLHF, by contrast, is like studying with a tutor who grades your essays on style and vibe - useful for open-ended writing, but subjective and expensive, and you can learn to flatter the tutor rather than get better.

This is why RLVR pairs so well with math, code, and formal logic, and why the frontier labs poured effort into it through 2025 and 2026. The July 2026 Ring-Zero paper pushed the idea to a trillion-parameter model trained with 'zero RL' - RLVR starting directly from a pretrained base with no human chain-of-thought data at all - and reported that the model self-organized into structured, self-verifying reasoning styles.

RLVR is not magic, and its limits are important. First, it only works where you can build a reliable verifier. Grading a poem, a business strategy, or a bedside manner has no answer key, so RLVR does not directly apply - those still lean on human feedback or learned reward models. Second, reward hacking does not fully disappear: a model can learn to pass a weak test suite without solving the real problem, or exploit quirks in how the checker parses answers, so the verifier itself becomes a target that must be hardened. Third, and most fundamental, RLVR can only draw out capabilities that pretraining already planted. As the Ring-Zero authors put it, if a mathematical concept is absent from pretraining, reinforcement learning cannot invent it - RLVR sharpens and elicits reasoning, it does not create knowledge from nothing.

Still, the practical impact has been enormous. RLVR turned a binary right-or-wrong signal into a scalable path from a raw pretrained model to a genuine reasoning system, and it did so without the labeling bottleneck that constrained earlier post-training. For any task where correctness can be checked by a machine, it has become the default way to teach a model to think.

Ground Truth. (2026, July 18). Reinforcement Learning with Verifiable Rewards (RLVR). Ground Truth. https://groundtruth.day/learn/reinforcement-learning-with-verifiable-rewards.html

@misc{groundtruth:reinforcement-learning-with-verifiable-rewards,
  title  = {Reinforcement Learning with Verifiable Rewards (RLVR)},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/learn/reinforcement-learning-with-verifiable-rewards.html}
}

Topics: reinforcement-learning · rl-post-training · rlvr · reasoning · training