News · 2026-07-23

Safety institutes measure Kimi K3's hacking ability: better than any open rival, nowhere near the top

The UK AI Security Institute and the US Center for AI Standards and Innovation published the first hard numbers on Kimi K3's offensive cyber ability on July 23, and the answer is split. K3 beat the previous open-weight leader on exploit development, scoring 32% against GLM-5.2's 24%. But it reached the benchmark's final rung -- writing a working exploit that executes attacker code -- in zero of 41 attempts, where the most cyber-capable models average roughly 20 of 41. The institutes conclude that K3 performs significantly below the frontier on both tests they ran.

Key facts

• The headline number: K3 achieved arbitrary code execution in 0 of 41 exploit tasks; leading closed models average about 20 of 41.
• Who: A joint preliminary assessment by the UK AI Security Institute and the US Center for AI Standards and Innovation, published July 23, 2026.
• The other test: On a 32-step simulated corporate network attack, K3 averaged step 17 and completed the full scenario in 1 of 10 attempts, against 28.5 and 6-7 of 10 for the leading closed models.
• Primary source: the AISI/CAISI assessment.

This is not a coding benchmark dressed up in security language. The exploit test, ExploitBench (arXiv:2605.14153), is built around 41 recent bugs in V8, the JavaScript engine inside Chrome. It scores an agent's progress along 16 measurable milestones: does it reach the vulnerable code, does it trigger a crash, does it build a memory-read primitive, can it write to arbitrary memory, can it hijack control flow, and finally can it execute code of its choosing.

The design choice worth appreciating is that every milestone is checked by a deterministic automated test, not by another language model acting as a judge. A great many AI benchmark results get inflated because a model grades a model. Here, either the memory write happened or it did not. That makes the zero especially informative: K3 is not failing a subjective standard, it is failing a mechanical one.

Think of it as a climbing route with 16 bolts. K3 clips more bolts than any open model before it. It has never reached the top.

The second test is closer to what a real intrusion looks like. "The Last Ones" is a 32-step simulated corporate network spanning four subnets and roughly 20 hosts, where an agent is given initial access and told to work its way through. K3 averaged step 17 under a 100-million-token budget; the most cyber-capable US models averaged 28.5. K3 completed the full range once in ten attempts, against six and seven of ten for the leading closed models in AISI's earlier testing.

AISI's own summary of what that means is measured but not comforting: K3 is capable of autonomously attacking a small, weakly defended, vulnerable enterprise system when directed and given initial access. And in a line that deserves more attention than the score comparison, the institutes state that K3's safeguards did not prevent attempts at exploit development or offensive operations. The model tried. It mostly just was not good enough.

Five caveats change how the numbers should be read, and the institutes flag all of them.

First, this is explicitly preliminary work on a small, selective set of evaluations. AISI says K3's hosting arrangement constrained what it could run at all. Second, the aggregate cyber estimate rests on a single benchmark's 41 tasks, which produces a much wider margin of error than estimates built from many tasks across many domains. Third -- and this cuts the other way -- the US closed models were tested with their system-level safeguards disabled, to measure maximum underlying capability. The public versions of those models still have their guardrails. Fourth, the simulated network is not a defended one: no active defenders, no security tooling, no penalty for setting off alarms, and a deliberate path through to the objective. Fifth, the human-expert time to complete the scenario is given as roughly 14 hours in one paper and roughly 20 in this assessment, so it is not a settled figure.

The caveat that should worry people most is a scaling result from the earlier benchmark paper. Performance on the network scenario rises log-linearly with how much inference compute the agent is given -- up to 59% improvement going from 10 million to 100 million tokens, with no observed plateau. That reframes the headline, and it updates the earlier read that the open-weight cyber gap was simply narrowing. "Below the frontier at 100 million tokens" is a statement about a compute budget, not about a ceiling on ability. And K3's open-weight release, still forthcoming when the model rattled markets on launch and still pending when the institutes tested it, removes the gatekeeper who enforces that budget. A weaker model that anyone can run unlimited times, without a refusal layer, is a different threat profile from a stronger model behind a metered API.

Moonshot's own launch post describes K3 as a 2.8-trillion-parameter native-vision mixture-of-experts model with a million-token context, aimed at long-horizon coding and knowledge work, and concedes that overall performance trails Claude Fable 5 and GPT-5.6 Sol. Those are coding claims, and they do not answer the cyber findings. Moonshot's footnotes also disclose that some benchmark results were produced using different agent scaffolding across models, which is exactly the kind of comparison hygiene issue that makes third-party evaluation valuable.

The honest caveat on the whole assessment: it names GLM-5.2 as the open comparison but identifies its closed comparison only as "the most cyber-capable US models." Nobody outside the institutes knows precisely which models are in each figure, which limits how much anyone can independently check. This is a preliminary read from two government bodies, not a replicated result -- and no external replication or expert rebuttal has appeared.

Primary source, verified: read the paper →

Ground Truth. (2026, July 23). Safety institutes measure Kimi K3's hacking ability: better than any open rival, nowhere near the top. Ground Truth. https://groundtruth.day/news/kimi-k3-cyber-capability-measured.html

@misc{groundtruth:kimi-k3-cyber-capability-measured,
  title  = {Safety institutes measure Kimi K3's hacking ability: better than any open rival, nowhere near the top},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/kimi-k3-cyber-capability-measured.html}
}

Topics: cybersecurity · ai-security · red-teaming · evaluation · open-weight-models