Learn · Beginner

Capability Thresholds and Responsible Scaling Policies

A capability threshold is a specific dangerous ability that an AI lab names in advance and commits to treat as a trigger: once a model demonstrates it, a defined set of safeguards becomes mandatory. Responsible scaling policies and preparedness frameworks are the documents that hold these thresholds. Their entire purpose is to make the hard decision before the moment of temptation -- to decide what would be too dangerous to ship while nobody has a launch date riding on the answer.

The problem they exist to solve is ordinary organizational psychology, not anything exotic about AI. A company that decides, on the eve of a launch, whether its own model is too dangerous to release is a company grading its own homework with money on the line. Every incentive points one direction. The response, borrowed from finance and from clinical trial design, is precommitment: write the rule down early, in public, in specific enough language that violating it would be visible.

How a threshold is structured

A well-formed threshold has three parts. First, a named capability -- not "the model is dangerous" but something like "can provide meaningful uplift to someone attempting to create a biological weapon" or "can autonomously discover and chain novel software vulnerabilities." Second, an evaluation that tests for it, which is where most of the real difficulty lives. Third, a consequence that follows automatically: a security level, a deployment restriction, an access program, or a halt.

Anthropic's Responsible Scaling Policy organizes this around AI Safety Levels, numbered ascending, each attaching a security and deployment standard to a capability tier. OpenAI's Preparedness Framework uses tracked risk categories with severity levels, of which High and Critical carry obligations. The structural difference between those two levels is the most important detail in the whole framework and is usually skipped in coverage: High capability requires safeguards before you deploy. Critical requires safeguards during development. That is a claim that some capability is dangerous enough to need containment before any customer sees it -- while the model is still being trained and evaluated inside the lab.

This is not abstract. On September 1, 2026, OpenAI designated its Astra model as meeting the Critical cybersecurity threshold, the first time it had placed any model at that level, after expert testers used the model to find previously unknown vulnerabilities and chain novel zero-days into a working exploit. Anthropic, on the same day, published a release where the same underlying model ships under two names with two different safeguard levels -- the restricted version for everyone, the permissive version only for vetted organizations. Both are capability thresholds being operated in public.

The mental model, and where it strains

The closest analogy is a building code. A code does not predict which building will burn; it specifies that above a certain occupancy you install sprinklers, and the specification exists before anyone breaks ground. Nobody negotiates fire safety with the developer during construction. Capability thresholds try to be the same thing for model releases.

The analogy breaks in one important place, and it is worth being blunt about it. Building codes are written by regulators and enforced by inspectors who do not work for the developer. Responsible scaling policies are written by the labs, graded by the labs, and revised by the labs. When OpenAI says Astra meets the Critical threshold, the threshold is OpenAI's definition, the evaluation is OpenAI's evaluation, and the safeguards are OpenAI's choice. That is not nothing -- a public precommitment is genuinely harder to walk back than a private one, and it gives internal safety teams a document to point at. But it is self-governance, and it should be read as such.

The deeper technical problem is that measuring a capability is much harder than naming one. A model's apparent ability depends enormously on the scaffolding around it: the tools it can call, how many attempts it gets, how the prompt is structured. A model that looks harmless in a chat box can look formidable inside a well-built agent harness, which is why the most common objection to capability designations is that the harness did the work. Evaluations also face the reverse problem -- models that behave differently when they detect they are being tested, covered in our lesson on evaluation awareness -- and the general fragility described in how AI gets benchmarked. A threshold is only as good as the test that decides whether it has been crossed, and these tests are young.

Why it still matters

Two things make thresholds worth taking seriously despite the self-grading. The first is that they force specificity. It is very hard to write "we will restrict access if the model can meaningfully help synthesize a pathogen" and then not notice when your model can. The second is that they create a public record. When OpenAI moved from "we cannot rule out Critical capability" in early August 2026 to a formal Critical designation three weeks later, that shift was legible precisely because the earlier hedge was on the record. Frameworks that get revised quietly to accommodate the model you want to ship are a real failure mode -- and the only defense against it is that the previous version was published.

If you take one thing from this: a capability threshold is not a safety guarantee. It is a commitment device, and its value comes entirely from being specific, public, and older than the decision it governs.

Ground Truth. (2026, September 1). Capability Thresholds and Responsible Scaling Policies. Ground Truth. https://groundtruth.day/learn/capability-thresholds-and-responsible-scaling.html

@misc{groundtruth:capability-thresholds-and-responsible-scaling,
  title  = {Capability Thresholds and Responsible Scaling Policies},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/learn/capability-thresholds-and-responsible-scaling.html}
}

Topics: ai-safety · governance · capability-thresholds · evaluation · policy · frontier-models