Every few months a new list of prompt-injection defences circulates, mostly variations on detecting malicious instructions in retrieved content. They help at the margin and none of them are a solution, because the underlying problem is not detection. It is that we routinely hand a model untrusted input and privileged tools in the same breath, then ask it to keep them straight.
The framing that works
Treat model output as coming from whoever authored the least trusted input in the context. If your agent read a web page, then the agent's proposed actions are as trustworthy as that web page. Once you internalise this, the mitigations stop being about wording and start being about permissions.
Three structural defences
- Separate the reading context from the acting context. One call summarises untrusted content and returns structured data. A second call, which never sees the raw text, decides what to do with that data. Injected instructions die at the boundary because they are not in the room where actions are chosen.
- Bind tool permissions to the user, not the model. The agent should be unable to call anything the requesting user could not call themselves, enforced server-side. A successful injection then achieves only what that user was already allowed to do.
- Require confirmation for anything irreversible. Not a modal that says "are you sure", but a specific, readable statement of the action and its target. Injections that survive everything else usually die here, because the user reads a sentence that makes no sense given what they asked for.
Exfiltration is the quiet risk
The attack that gets underrated is not destructive, it is extractive: text that persuades the agent to include sensitive context in a URL, an image request, or an outbound message. Defences are unglamorous and effective. Do not let the model construct arbitrary outbound URLs. Strip or proxy images in rendered output. Allow-list domains for any network egress. Log every outbound call with its full payload.
Test it like security, not like quality
Add an adversarial suite to your evaluation set with cases drawn from real attack patterns: instructions hidden in documents, in file names, in code comments, in HTML attributes, in the middle of otherwise legitimate content. Run it on every change. When one succeeds, the fix belongs in the architecture, not in the system prompt.
A system prompt is a suggestion. A permission check is a rule. Only one of them survives an attacker who is better at writing than you are.
The uncomfortable conclusion
There is no wording that makes a model reliably ignore instructions in its input, and designing as though there might be is the actual vulnerability. Assume injection succeeds, and make sure the successful attacker inherits an account with almost no power.
Join the conversation
Discussion0
Join the conversation. Sign in to leave a response or reply to comments.