News · 2026-08-19

Agent skills work by anchoring procedure, not by adding knowledge

Agent skill files improve performance mainly by locking in the order of operations, not by teaching the model anything it did not know. In a study of 8,135 agent trial records, researchers labeled 65.7 percent of successful skill cases as procedural anchoring, meaning the skill stabilized setup steps, tool ordering, checks and error recovery. Only 4.5 percent involved injecting explicit knowledge. The same paper reports that the benefit collapses as skill libraries grow.

Key facts

• The paper is Demystifying Agent Skills: Why They Work-Until They Don't, submitted to arXiv on August 14, 2026, and currently the top-voted paper of the day on Hugging Face with 142 upvotes.
• Procedural anchoring accounts for 65.7 percent of skill cases; explicit knowledge injection accounts for 4.5 percent.
• As the candidate pool grows from 5 skills to 100, actual-use precision drops from 29.6 percent to 3.3 percent.
• The authors open-coded 240 trajectories, kept 238 valid labels, and validated the taxonomy against 714 human trajectory checks with 95.8 percent exact agreement.

An agent skill is a folder: some instructions, sometimes a script, and a note about when to use it. Over the past year the format went from a Claude Code convention to something closer to a package ecosystem, a shift we covered when agent skills quietly became a package format. The community explanation for why they work has always been vague. People say skills "give the agent context," which is true in the way that "food gives you energy" is true.

This paper is the first serious attempt to say which part of the context is doing the work, and the answer is unglamorous. Skills are not knowledge transfer. They are choreography. When a skill helps, it is usually because it told the agent to check the build before editing, or to run the test suite in a particular order, or to look at the config file first. The model already knew how to do each of those things. What it did not reliably do was do them in that order, every time.

Think of a good skill as a pilot's checklist rather than a textbook. The pilot knows how the aircraft works. The checklist exists because knowing is not the same as remembering under load, in sequence, without skipping the boring step.

The authors are careful to show that brevity alone is not the mechanism, which is the obvious rival explanation. On one benchmark slice they compare four kinds of compact text: a short plan derived from the instructions, a test-first template derived from a past workflow, raw workflow memory, and a distilled skill. Performance climbs steadily across that ladder, with the distilled skill well ahead of all three. So the win comes from abstraction and from selecting the right procedure, not merely from having fewer tokens in front of the model.

Then comes the part that should worry anyone maintaining a skill library. Retrieval is a separate, and worse, problem. When the agent has five candidate skills to choose from, it uses the right one about three in ten times. At a hundred candidates, it is closer to three in a hundred. That is a nine-fold degradation from library growth alone, before anything about skill quality enters the picture. The authors also separate retrieval from outcome, finding that pulling the correct skill is neither sufficient nor necessary to finish the task. A perfect retriever does not guarantee a solved problem.

This lands directly on top of a result we covered days earlier, when researchers argued that agent skill libraries now need a librarian and proposed admitting new skills only after execution evidence justified them. Read together, the two papers point the same way: the interesting engineering problem has moved from writing skills to curating and scoping them.

The failure modes the paper names are brittleness, context mismatch, and insufficient adaptation. A skill written for one repository layout misfires in another. A skill that assumes a package manager fails when the project uses a different one. Because the agent has to first decide whether the skill applies at all, every added skill is a new opportunity to decide wrong. That is a genuinely new failure surface, and it is created by the same mechanism that produces the benefit.

The honest caveat is scale. The quantitative advantage of a distilled skill over plain workflow memory in matched comparisons is 6.06 percentage points. Real, statistically defensible, and considerably smaller than the enthusiasm around skill repositories would suggest. The methodology is unusually careful for this corner of the literature, with human validation reaching a Cohen's kappa of 0.952, which makes the modest effect size more credible rather than less.

For practitioners the takeaway is concrete: prune aggressively, scope skills tightly to the environment they were written for, and treat a growing library as a cost rather than an asset. There is also a security dimension that this paper does not address but that follows from it, since a folder full of instructions that an agent will execute without much scrutiny is a supply-chain surface. Our lessons on agent harnesses and scaffolding and prompt injection cover the shape of that risk.

Primary source, verified: read the paper → (arXiv 2608.14036)

Ground Truth. (2026, August 19). Agent skills work by anchoring procedure, not by adding knowledge. Ground Truth. https://groundtruth.day/news/agent-skills-work-by-anchoring-procedure-not-by-adding-knowledge.html

@misc{groundtruth:agent-skills-work-by-anchoring-procedure-not-by-adding-knowledge,
  title  = {Agent skills work by anchoring procedure, not by adding knowledge},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/news/agent-skills-work-by-anchoring-procedure-not-by-adding-knowledge.html}
}

Topics: agents · agent-skills · research · context-engineering · coding-agents · retrieval