Learn · Intermediate
Sandboxing an AI agent: least privilege for a program that improvises
Sandboxing an AI agent means deciding in advance which actions it is capable of taking, instead of relying on the model to decide well in each moment. It matters because an agent's behaviour is steered by text, much of that text arrives from sources an attacker can influence, and no improvement in model quality removes that structural exposure. The security boundary has to live in the system around the model, not inside it.
Start with what makes an agent different from a chatbot. A chatbot produces text a human reads. An agent produces text that a program executes: it runs shell commands, edits files, calls APIs, sends messages, browses the web. The harness around the model turns its output into real actions. That is what makes agents useful, and it is also the entire security problem in one sentence.
The specific danger is that agents consume untrusted input as part of doing their job. An agent researching a topic reads web pages. An agent triaging a repository reads issues and pull requests filed by strangers. An agent processing email reads whatever arrives. Every one of those inputs enters the same context window as the instructions, and the model has no reliable way to tell an instruction from a quotation of an instruction. That is prompt injection, and Greshake and colleagues showed in 2023 that it works indirectly — the attacker never talks to the agent, they simply leave text where the agent will read it.
The instinctive response is to make the model better at refusing. This helps, and it is not sufficient, for a reason worth internalising: refusal is a judgement about text, made by a system whose inputs an adversary partly controls. Zou and colleagues demonstrated that adversarial suffixes can be found automatically and transfer across models, which means the attack surface is not a fixed list of bad phrasings someone can patch. Red-teaming measures how hard the model is to talk into something. Sandboxing assumes that eventually someone succeeds, and asks what happens next.
The classical answer, borrowed wholesale from decades of systems security, is least privilege: give a component exactly the capabilities its task requires, for exactly as long as it needs them, and nothing more. Applied to agents, that decomposes into a few concrete practices.
Constrain the tools, not the prompt. An agent that only ever needs to read files should not hold a tool that writes them. A capability the agent does not have cannot be misused, regardless of what any web page told it. This is the single highest-leverage control, and it is enforced in code rather than in English.
Separate trusted instructions from untrusted data by construction. The CaMeL line of work argues for building systems where retrieved content can influence values flowing through a program but not the control flow — the plan is fixed by trusted code, and untrusted text can only fill in slots. That is a stronger guarantee than asking a model to ignore instructions it finds in a document, because it does not depend on the model noticing.
Bound the blast radius. Run the agent in a container with no credentials it does not need, on a network segment that cannot reach production, with a filesystem view limited to the working directory. If the agent is compromised, the attacker inherits the agent's reach — so make that reach small.
Put a human at irreversible steps. Reading is recoverable. Sending an email, deleting data, merging a branch, moving money and publishing content are not. Gating exactly those actions costs little and converts a silent compromise into a visible prompt.
Log the actions, not just the conversation. Tool calls are the ground truth of what an agent did. Conversation transcripts show intent; action logs show effect.
A recent case illustrates why the boundary must be structural rather than behavioural. Palo Alto's Unit 42 published a reconstruction of an attacker's agent that exposed its operator's entire workspace — configuration, API keys, target lists, shell history and full session logs — because the agent started a public file server from its home directory. The agent was not attacked; it was simply given broad capability and left running, and one ordinary misconfiguration turned that capability into total exposure. It cuts both ways: an unsandboxed agent is dangerous to its target and to whoever runs it.
There is a second, newer surface worth naming. Agents increasingly carry memory — files, retrieved lessons, trust scores — that is written during operation and read later. If an attacker can influence what gets written, they influence every future decision that reads it, which is data poisoning with a persistence property. As of mid-2026, most published agent-memory systems are not evaluated against an adversary who controls the writes, which makes memory stores a good default candidate for sandboxing rather than trust.
The honest summary is that sandboxing is unglamorous and works, while making the model harder to fool is glamorous and helps but does not finish the job. If a control only holds when the model behaves correctly, it is not a control. It is a hope with a nice interface.
Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
Defeating Prompt Injections by Design
Universal and Transferable Adversarial Attacks on Aligned Language Models
Key questions
Why can't you just train an agent to refuse dangerous actions?
What does least privilege mean for an AI agent?
How is sandboxing different from red-teaming?
Cite this
APA
Ground Truth. (2026, August 2). Sandboxing an AI agent: least privilege for a program that improvises. Ground Truth. https://groundtruth.day/learn/sandboxing-ai-agents.html
BibTeX
@misc{groundtruth:sandboxing-ai-agents,
title = {Sandboxing an AI agent: least privilege for a program that improvises},
author = {{Ground Truth}},
year = {2026},
month = {aug},
url = {https://groundtruth.day/learn/sandboxing-ai-agents.html}
}