Prompt injection is an attack in which adversarial instructions hidden in the text a large language model processes override the developer's intended…
Understand why an LLM that reads untrusted text can be commanded by it.
Prompt injection exists because a large language model receives the system prompt, the user's message, and any retrieved content as one text stream, with no reliable way to mark part of it as trusted instructions and the rest as inert data. Any text the model reads can act as an instruction.
Direct injection is typed by the user to defeat the app's own rules. Indirect injection is more dangerous: the payload is hidden in content the agent fetches for someone else — a web page it browses, a document it summarizes, an email it reads — so the victim never sees it.
Survey the concrete forms injection takes, up to the lethal trifecta.
Jailbreaking crafts input that makes the model ignore its safety instructions — role-play framings, fake 'developer' overrides, or encodings that slip past filters. Data exfiltration goes further: an indirect payload instructs the agent to take private data it can see and send it out, often by embedding it in a URL or an outbound message.
The scary cases combine reading untrusted content with the power to act, turning a text trick into real data loss or unwanted actions.
An agent is most exposed when it has all three of: access to private data, exposure to untrusted content, and the ability to communicate externally. With all three present, an indirect injection can read secrets and ship them out. Remove any one leg — restrict the data, sanitize the content, or block external sends — and the exfiltration path is broken.
Layer the practical defenses, knowing none alone is complete.
Because injection cannot be fully prevented, defenses are layered. The foundation is least privilege: give the agent only the tools and data it truly needs, and require human approval for irreversible or sensitive actions. If a hijacked model cannot reach a dangerous tool or the open internet, the injection has nowhere to go.
On top of that: separate untrusted content with clear delimiters or spotlighting so the model treats it as data; filter and scan inputs and outputs for known payloads and for secrets leaving the system; constrain responses to a fixed format; and keep secrets out of any context retrieved text can reach.
Stronger designs limit trust structurally. A dual-LLM pattern uses a privileged model that never sees untrusted text and an unprivileged model that processes it but cannot call sensitive tools. Action allow-lists, per-tool scopes, and blocking untrusted-triggered external requests cut the exfiltration path directly.
Design an agent that resists injection and verify it before shipping.
Design as if the model will sometimes obey an injected instruction. Break the lethal trifecta by default, gate risky tools behind human approval, log every tool call, and keep the agent's reach minimal. These limits hold even when a filter is bypassed.
Then test adversarially: run known direct and indirect payloads against your agent, try to make it exfiltrate a planted secret, and add each new bypass to a regression suite. Treat prompt injection like any security risk — continuously tested, never assumed solved.
Prompt injection overrides a model's intended instructions with adversarial text it reads. Direct attacks are typed by the user; indirect attacks hide in fetched content and are the real danger, especially data exfiltration. The lethal trifecta — private data, untrusted content, and external communication together — is the high-risk pattern. Defense is layered: least privilege, human approval, breaking the trifecta, filtering, and structural patterns, all verified by adversarial testing rather than assumed solved.
Take an agent that reads customer emails and can send replies. Identify which legs of the lethal trifecta it has, choose which leg you would remove first, and write one indirect injection payload you would add to its adversarial test suite.
What is prompt injection?
Injection exploits the fact that an LLM processes instructions and data in one stream and cannot reliably tell them apart.
What is the lethal trifecta in AI agent security?
When an agent has all three, an indirect injection can read secrets and send them out; removing any one leg breaks the exfiltration path.
How is prompt injection different from jailbreaking?
Jailbreaks are a subset focused on defeating safety guardrails; prompt injection includes indirect attacks that hijack an agent through content it processes.
How do you defend an AI agent against prompt injection?
No single fix works; defense in depth plus limiting the agent's reach contains the damage even when an injection succeeds, and adversarial testing keeps it honest.