Layer schemas, policy checks, citation validation, and review workflows.
An LLM will confidently produce wrong, unsafe, or off-policy output. A guardrail is an explicit check that sits between that output and the real world. This module frames the whole topic before we open up each layer.
A guardrail is an explicit check that decides whether an LLM output is safe to use.
Why this matters: It matters because an LLM will confidently produce wrong or unsafe output, and something must catch it.
Before reading on, predict: you ship a support bot that drafts refund emails. The model is usually right. What is the one thing that, if it goes wrong even 1% of the time, would force you to add a check before the email is sent?
A language model is a fluent guesser. It produces the most plausible-sounding continuation, not a verified-correct one. Most of the time that is fine. The problem is the tail: it will sometimes a fact, leak private data, or follow an instruction smuggled into its input — and it will do all of this with total confidence.
That confidence is exactly why you cannot wire model output straight to a real action. A is the answer: an explicit check, separate from the model, that inspects an output and returns a clear pass or fail before anything reaches the user or the outside world.
The mental model to carry through this lesson:
The bot drafts a refund email and is about to send it. Worst plausible output: it invents a refund amount the customer never qualified for. That is a that costs real money.
The boundary is the send action. So a guardrail sits there with a one-sentence pass condition: the refund amount in the draft must exactly match the amount in the order record. If it does not match, the email is blocked and routed to a human — the model never gets the final say on money.
Slide from 'ship it raw' to 'block until validated' to see how the guardrail's job grows with the stakes of the action.
Ask: if the model is wrong here, what is the worst thing that actually happens? A bad brainstorm is harmless; a wrong refund amount is money out the door.
Locate the point where the output leaves the model and touches the user or an external system. The guardrail goes right there, before the crossing.
State, in one sentence, what must be true for the output to pass. If you cannot write it, you do not yet have a guardrail — you have a hope.
Mechanically, a guardrail is just a function that runs on the model's input or output and returns a verdict. It lives in your application code, not inside the model — which is the whole point. The model is untrusted; the guardrail is the trusted layer that wraps it.
A guardrail can sit in three places: screening the input before the model sees it, validating the output before anyone sees it, and gating a risky action before it runs. This module names the three layers so you know where each specific check belongs.
Guardrails come in layers — screen the input, validate the output, gate risky actions.
Why this matters: It matters because knowing the layers tells you where to put each specific check.
Before reading on, predict from memory: in module 1 we said a guardrail returns a clear verdict, not a suggestion. What were the three possible verdicts?
(The answer: pass, fail, or repair. Hold that — every layer below produces one of those three.)
A guardrail is not one thing in one place. There are three distinct points in the flow where a check can run, and they catch different failures.
Each layer is the trusted code wrapping an untrusted step. You rarely need all three — but you need to choose deliberately, because a check in the wrong layer catches nothing.
A support assistant retrieves help-center docs, answers the user, and can open a refund ticket. Watch all three layers fire.
Each layer has blind spots. An input guardrail cannot catch a hallucination the model invents on its own. An output validator cannot un-send a payment that already fired. An action gate trusts whatever output reached it. Layers compose: input narrows what enters, output narrows what leaves, action protects what is irreversible.
More validation means fewer bad outputs slip through — but also more latency, more cost, and more good outputs wrongly rejected. This module makes that tradeoff explicit so you size each guardrail to the risk instead of defaulting to maximum strictness.
Pick how much validation to add by weighing risk against latency and false rejections.
Why this matters: It matters because over-guarding is as harmful as under-guarding — it blocks valid work.
Decision this forces: Choose a strictness level for this guardrail: log-only, a deterministic check, or a model-judge plus human gate.
It is tempting to make every check as strict as possible. Resist it. A is never free — it costs you on three axes.
So the real question is never "is this safe enough?" in the abstract. It is: how bad is the worst output here, and how much friction can I add before the feature stops being useful? Match the strictness to the stakes.
How bad is the worst output? One annoyed user, or money and trust lost at scale? This sets your ceiling on acceptable friction.
How often does the model actually get this wrong? A rare, low-harm failure may need only logging; a frequent, high-harm one needs a hard block.
Prefer a cheap deterministic rule (regex, schema, range check) over a model-based judge. Only reach for the expensive check when the cheap one cannot express the rule.
| Option | What it catches | Latency & cost | False rejections | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Log-only (observe) | Nothing is blocked; failures are recorded for later review. | Near zero — runs async, no extra wait. | None — it never blocks a good output. | Use while you are still learning how often the model fails and what real failures look like. | Low | Low |
| Deterministic check | Anything expressible as a rule: schema, ranges, banned terms, required citations. | Tiny — milliseconds, no extra model call. | Low if the rule is precise; can over-block if too broad. | Use as the default for anything you can state as an explicit rule. | Low | Medium |
| Model-judge + human gate | Fuzzy, semantic failures plus a human backstop for high-stakes actions. | High — an extra model call, plus wait time for human approval. | Can be higher; the judge itself can be wrong. | Use only for irreversible, high-harm actions where a deterministic rule cannot capture the risk. | High | High |
A deterministic validator — a JSON-schema check, a numeric range, a citation-presence test — is cheap, instant, and itself never hallucinates. A model-based judge is more flexible but adds latency, cost, and a second source of error. Prefer the deterministic check whenever the rule can be written down; layer a judge on top only for the genuinely fuzzy cases.
A validator is just code that inspects an output and returns pass, fail, or repair. This module turns the idea into the smallest runnable artifact, then shows how to verify a validator an AI assistant writes for you before you trust it.
A validator is code that inspects an output and returns pass, fail, or repair.
Why this matters: It matters because a guardrail you cannot run and test is just a wish.
Strip away the framing and a is a small function. It takes the output, applies one rule, and returns a verdict. The whole skill is deciding what that verdict triggers.
Notice: the verdict must be deterministic for a given output. That is what makes a validator testable — you can hand it a known-bad output and assert it returns fail.
Your assistant must return JSON like {"answer": "...", "citations": ["doc-3"]}. Two things can go wrong: the JSON is malformed, or the answer cites a document that was never retrieved (a ).
The validator runs two rules in order. First, parse the JSON — malformed output is a repair case, so re-prompt once. Second, check every citation id is in the set of retrieved doc ids — an ungrounded citation is a fail, so block and fall back to "I don't have a sourced answer."
def validate(answer: str, citations: list[str], retrieved_ids: set[str]) -> str: # Rule: every cited doc must have actually been retrieved. for c in citations: if c not in retrieved_ids: return "fail" # ungrounded citation -> block if not citations: # TODO: an answer with NO citations at all — what verdict? return ___ return "pass"
retrieved_ids: set[str]if c not in retrieved_idsif not citationsThe first loop catches the hallucinated citation. The TODO is the case people forget: an answer that cites nothing. For a grounded assistant, no citation means no evidence — so it should not pass.
"fail" — an answer with no citations has no evidence behind it. Returning "pass" here is the single most common hole in a grounding validator: it lets confident, unsourced claims straight through.
"The refund amount must equal the order's recorded amount." One sentence, true or false — nothing fuzzy yet.
For money, fail means block and escalate — never auto-repair. For a malformed JSON field, repair (re-prompt) may be fine. Choose deliberately.
Before the validator works, write the failing example it must catch. If your validator passes that input, it does nothing useful.
Verify it before you trust it. When an AI assistant writes a validator for you, the code usually looks right and handles the happy path. Check these four things — they are exactly where generated validators are weakest:
A guardrail that fails silently is worse than none — it manufactures false confidence. This module names the ways guardrails break in practice, the decision of what to do when one trips, and a recall check to lock it all in.
Guardrails fail by being bypassed, too brittle, or trusted blindly.
Why this matters: It matters because a guardrail that fails silently is worse than none — it creates false confidence.
Decision this forces: Choose what the system does when a guardrail trips: hard-block, repair and retry, or fall back to a safe default or human.
Before reading on, predict from memory: in module 4 a validator returned one of three verdicts. Name them — and recall which one money should never be auto-handled with.
(The answer: pass, repair, fail. Money must never be auto-repaired — a high-stakes fail blocks and escalates.)
A guardrail can give you false confidence — you believe you are protected while the check quietly does nothing. These are the three ways it happens.
The unsafe content slips around the check. A is rephrased to dodge a keyword filter; a leak hides in a field the validator never inspects. The guardrail returns pass on something it should have failed.
The check is so strict it rejects valid outputs. Users hit "I can't help with that" constantly, learn to distrust the feature, and route around it — which is its own safety failure.
The verdict is correct but nobody acts on it. The validator logs "fail" and the code continues anyway; or a model-based judge is itself wrong and no one audits it. A guardrail you do not monitor decays silently as inputs drift.
Record pass/repair/fail with the input that produced it. A guardrail with no telemetry cannot be debugged or trusted.
A zero block rate means bypassed; a sky-high one means brittle. Both are alarms, not silence.
Keep a growing set of real bad outputs and re-run them whenever you change the guardrail or the model — the cheapest defense against silent decay.
| Option | User experience | Safety | When to use | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Hard block + error | Worst — the user gets a refusal with no result. | Strongest — nothing unsafe ever ships. | Irreversible, high-harm actions. | Use for payments, deletes, and anything you cannot undo. | Low | Low |
| Repair + retry | Good — the user often never notices. | Medium — only safe if the repair is verified. | Recoverable format or style issues. | Use for malformed output you can fix and re-check, like broken JSON. | Medium | Medium |
| Fall back / escalate to human | Slower but still served — a safe degraded answer or a human. | Strong — a human or a known-safe default catches it. | Ambiguous or high-stakes but not instant. | Use when a wrong answer is costly but you can afford a delay for review. | High | Medium |
The failures above map directly to the Top 10 for LLM applications — prompt injection, sensitive-information disclosure, and excessive agency are all guardrail-shaped problems. Logging each tripped guardrail into a turns ad-hoc checks into an auditable safety story you can show a reviewer.
Reconstruct the whole pipeline from memory before you look back: Why can't you trust a raw LLM output? What are the three layers a guardrail can sit in? What does a stricter guardrail cost you? What three verdicts can a validator return? And what are the three ways a guardrail breaks in practice?
If you can answer all five from memory, you own this lesson. If one is fuzzy, that is the module to reopen.
Solo build: pick one real LLM feature you'd ship. Write its worst plausible output, choose the layer and strictness, code the smallest validator with a known-bad test that returns 'fail', and define what happens on failure. Then take the next rung — add a second layer (e.g. an action gate on top of your output validator) and a block-rate metric you'd alert on.
An LLM returns a confident-sounding answer that contains a made-up fact. Which guardrail layer is the RIGHT place to catch this before it reaches the user?
Hallucinations are a risk in what the model produces, so the output layer — which sits between the model and the user — is where that check belongs.
A user embeds hidden instructions in a document they upload, trying to hijack the model's behavior. Which layer and risk does this map to?
Prompt injection is an attack that arrives in the model's input, so the input layer is where it must be detected and blocked.
Your team is debating guardrail strictness for a low-stakes feature that auto-suggests email subject lines. Which approach best matches the stakes?
Maximum strictness adds latency, cost, and false rejections; for low-stakes actions a log-only or simple deterministic check is proportionate and avoids those costs.
You write a validator that checks whether the model's output is valid JSON. What is one specific failure the validator might silently miss — i.e., the output passes the check but is still wrong for your system?
A validator that only checks format (is it JSON?) will pass output that is well-formed but missing required keys or containing wrong value types — the rule it must catch but might miss.
A guardrail trips on 30% of real user requests — far more than expected. After investigation, the rule is too broad. This is an example of which of the three main ways guardrails fail in practice?
A guardrail that blocks too many legitimate requests is miscalibrated toward over-strictness — one of the three core failure modes alongside gaps (under-catching) and bypass (circumvention).