News · 2026-08-04

Coding Agents Pass the Tests by Wrapping the Old Code Instead of Deleting It

Coding agents are far better at adding code than removing it. A study submitted July 30 finds that 29.0% of the agent patches that pass SWE-bench Verified keep code the human developer's patch deleted, typically by wrapping the old logic in a guard, fallback or bypass. When the authors retrofitted 34 deletion-heavy tasks with checks that fail if the target code survives, resolution across four frontier models fell from 63.2% to 41.9%. The tests were green; the obsolete path was still there.

Key facts

• Across five OpenHands-based SWE-bench Verified leaderboard submissions, models reached the correct file for over 92% of required deletions but deleted the correct lines only 44.6% to 51.6% of the time.
• 29.0% of passing patches exhibit the pattern the authors name Guard-and-Go; adding deletion checks to 34 tasks cut resolution from 63.2% to 41.9%.
• On CanItDelete, a new 200-task benchmark mined from real commits where the entire required edit is a deletion, the best model still fails 20.0% of tasks and incomplete deletion is the dominant failure for 10 of 12 models.
• To Add Is Machine, To Delete Is Human, arXiv:2607.28887, submitted July 30, 2026.

The measurement is the clever part

It would be easy to write a paper about model intent. This one refuses to. Deletion avoidance is defined purely as observable patch behaviour: the developer's reference patch removes code at a location, and the model's patch does not. No mind-reading required.

That framing produces the paper's sharpest number. Models find the right file almost every time - over 92% - and then fail to remove the right lines roughly half the time. The problem is not search. It is the edit itself.

What they do instead is the interesting bit. Guard-and-Go means the model leaves the obsolete logic in place and adds control flow around it: an if that is never true, a fallback branch, a feature flag, a bypass. The behaviour test passes because the new path works. The old path is still compiled, still readable, still something a future engineer has to reason about.

Anyone who has asked an assistant to simplify a file and watched it get longer has met this pattern personally.

Prompting barely fixes it

The authors built a four-rung ladder: an ordinary developer-style request, an explicit instruction to delete and avoid workarounds, a pointer to the region, and finally the exact character spans to remove.

The first two rungs barely move the numbers. Region pointers help a little. Exact spans improve every model tested - GPT-5.6 Sol reaches 80.5% success under that condition, still not perfect - but introduce a new failure: models delete past the boundary, or add code where none was wanted.

The cleanest reading is that this is a bounded-edit control problem. The model has to find every occurrence and stop at the right edge, and supplying the exact spans hands it both pieces of information at once. It would overstate the paper to say location never matters; it is that location alone is not the binding constraint.

It looks like a training gap

The proposed fix is not a prompt trick. In a pilot, the authors added 12,821 deletion examples to a 7B model's code post-training mixture - about 112 million tokens, or 0.7% of a 15.9-billion-token mix. That cut deletion avoidance on CanItDelete by 13.9 percentage points, raised SWE-bench Verified by 5.3 points, and nudged CanItEdit up 1.4.

A 0.7% data change producing a five-point benchmark gain is the kind of result that suggests the deficiency was never a capability limit. It was an absence in what the model was shown. The authors present it as proof of concept at one scale, and note that over-deletion becomes a new error mode in some cases.

The same problem when a human touches the file

A second paper, SWE-Touch, submitted August 3, asks what happens when a person edits the repository while the agent is working. It introduces validated Counter-Edits: plausible modifications to task-relevant code that conflict with what the agent is trying to do, mined from repair trajectories and injected when the agent reaches the relevant region.

Across nine models on SWE-bench Verified, that single intervention lowers the mean resolve rate by 7.7 percentage points, with model-level losses from 1.3 to 16.5. The failure taxonomy is where it gets pointed: in 63.3% of audited solved-then-failed runs, the category is retained conflict - the agent finishes with the user's conflicting behaviour still active. Incorrect replacement accounts for 13.9%, incomplete reconciliation 11.6%, off-target implementation 5.5%.

The controls rule out the easy explanation. A message-only condition has small effects; a silent conflicting code edit hurts every model tested. And more tool calls do not reliably rescue it - larger interaction budgets produce no uniform dose-response.

Why it matters

Both papers land on the same place from different directions: passing a behavioural test is not the same as leaving the repository in the state a human would. One shows tests going green over dead code. The other shows agents continuing against a workspace that has moved under them.

This compounds a problem this site covered in June, when one in seven SWE-bench Verified tasks turned out to be graded against a patch that does not match. If the grader is imperfect and the passing patches hide retained code, leaderboard numbers are measuring something narrower than "fixed it."

The product lesson is not simply to train models to delete more. It is to make the edit contract explicit: surface workspace diffs to the agent, require bounded patches, and test both that the new behaviour works and that the old path is gone when removal was the point.

The honest caveat

Both are fresh v1 preprints with no independent replication and no peer review. The deletion benchmark's code is withheld during anonymous review, so nobody can yet reproduce the headline numbers. And SWE-Touch is explicitly a stress test for one controlled kind of conflict - a synthesised user edit - not a general claim about every way a human might interrupt an agent.

Primary source, verified: read the paper → (arXiv 2607.28887)

Ground Truth. (2026, August 4). Coding Agents Pass the Tests by Wrapping the Old Code Instead of Deleting It. Ground Truth. https://groundtruth.day/news/coding-agents-pass-the-tests-by-wrapping-old-code-instead-of-deleting-it.html

@misc{groundtruth:coding-agents-pass-the-tests-by-wrapping-old-code-instead-of-deleting-it,
  title  = {Coding Agents Pass the Tests by Wrapping the Old Code Instead of Deleting It},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/news/coding-agents-pass-the-tests-by-wrapping-old-code-instead-of-deleting-it.html}
}

Topics: coding-agents · swe-bench · evaluation · benchmarks · software-engineering · agents