News · 2026-10-19

Harness² improves agents by rewriting their surrounding software

Google Cloud AI Research’s Harness² lets an agent propose and test changes to the software surrounding a frozen language model. The public research system modifies prompts, tools, memory and execution procedures rather than training new model weights. Its paper reports gains across professional task benchmarks, while documenting nine regressions that show why apparent improvement still needs an independent check.

Key facts

• Harness² changes eight kinds of harness components around a frozen executor model.
• The repository’s first commit was October 8; the paper’s first version was submitted October 9, 2026.
• The paper documents nine regressions, including a candidate-selection mistake between outputs later scored 55 and 95.
• Primary artifacts are the repository and full paper.

A better agent need not begin with a better model. The same system can perform differently when given a clear checklist, a reliable script, a useful memory entry or a less confusing tool interface. Harness² turns those surrounding design decisions into things an automated process can change and carry forward.

An analogy is improving a workshop without replacing its workers. Better jigs, instructions and inspection procedures can change the result even when the workers’ underlying abilities stay constant. The corresponding AI distinction is between changing learned model weights and changing the harness that organizes execution. Harness² operates on the latter.

The method uses a Propose–Probe–Compose loop. It first runs a baseline on an unfamiliar task, then proposes an invariant edit and a domain-specific edit. Candidates can affect prompts, guardrails, memory, skills, tool guidance, scripts, subagents or middleware. The system executes candidates in the same task and environment, compares what happened and composes a successor configuration.

A second recursion maintains a persistent collection of adopted harnesses and lessons. This allows useful changes to influence later tasks. It also creates an evaluation problem: an appealing change can become established because it looked better in a small set of executions, even if it actually encodes a mistaken assumption.

The paper’s methods and analysis make the central limitation explicit. The improver receives no answer key or correctness label for the unseen task. It observes differences in behavior. A candidate that produces a longer report, runs more tools or appears more decisive is not necessarily the candidate that gets the answer right.

The authors’ own failure analysis is therefore unusually important. One selection picked an output retrospectively scored 55 over an alternative scored 95. Another edit narrowed task coverage. These examples show that automated review can prefer the wrong trade-off, especially when the editor and executor share assumptions or when a single run is affected by randomness.

The reported improvements are not merely prompt rewriting. Skills and scripts account for about half of the edits. In the paper’s three-domain WorkBuddy analysis, removing skills reduced the average by 5.3 points, while removing scripts reduced it by 3.7. That points toward reusable executable procedures as an important part of the system, though these remain the authors’ experiments rather than an outside replication.

Improvement also consumes additional work. Even the base setting includes a reference execution, two candidate probes and a final execution for each task. More recursion adds more editor and executor calls. A method that improves final quality might still be unsuitable when latency or spending matters more than squeezing out another successful task.

A concrete GitHub cost-accounting issue reinforces the point. The discussion identified executor rollouts missing from recorded dollar spending, which would cause a budget cap based on that ledger to omit much of the run cost. This was a time-specific implementation concern, not proof that all published comparisons are wrong. It shows why a budget must include the whole execution process.

The Hacker News thread asked for comparisons holding the model constant while varying the harness. Reddit discussion raised overfitting and questioned how new the idea was. These are useful questions rather than independent evaluations. The repository also says it is “not an officially supported Google product”; the paper’s affiliation is Google Cloud AI Research, not Google DeepMind.

The October 19 attention is renewed discovery of research released earlier in the month. Its lasting significance is practical: agent improvement increasingly includes editing the procedures that organize a model’s work. The honest caveat is that a successful-looking execution cannot certify the edit that produced it. Independent tests, cost accounting and held-out tasks remain necessary before accumulated changes deserve lasting trust.

Primary source, verified: read the paper →

Ground Truth. (2026, October 19). Harness² improves agents by rewriting their surrounding software. Ground Truth. https://groundtruth.day/news/harness2-edits-agent-software-without-retraining-models.html

@misc{groundtruth:harness2-edits-agent-software-without-retraining-models,
  title  = {Harness² improves agents by rewriting their surrounding software},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/news/harness2-edits-agent-software-without-retraining-models.html}
}

Topics: agents · open-source · evaluation · harnesses · google