Add approvals, review, and correction points to agent workflows.
You map the risk/autonomy spectrum and identify which agent actions cross the threshold that demands a human checkpoint. Using a refund-processing agent as the running scenario, you classify its tool calls by impact and reversibility.
A framework for classifying agent actions by impact and reversibility to decide which ones need a human checkpoint.
Why this matters: Before you can build a safe, production-grade agent, you need a principled way to decide which actions it can take autonomously and which ones must pause for human review.
Your refund-processing agent issued a $4,200 refund to the wrong order. It followed instructions correctly and needed no oversight. Who is liable?
This is the core tension of the . More autonomy means faster execution. But it also means the agent can cause real harm before anyone notices. Accuracy alone doesn't eliminate liability. An agent can be right 99% of the time and still cause catastrophic outcomes on the 1%.
The fix isn't to remove autonomy. It's to place exactly where risk crosses a threshold. This module gives you the framework to find those thresholds in any workflow.
Slide to see how increasing agent autonomy changes the risk profile and the checkpoint requirement for a refund-processing workflow.
Any one of three conditions justifies a checkpoint. You don't need all three.
These conditions are independent. A low-dollar action can still be irreversible. A reversible action can still be high-stakes if it triggers a downstream cascade.
A refund-processing agent has five tools. Before reading the classification below, predict which ones need a gate and why.
lookup_order — read-only, fully reversible, zero financial impact. Safe to automate.check_policy — read-only retrieval of refund rules. No side effects. Safe to automate.send_email — irreversible (you can't unsend), moderate stakes (customer-facing). Gate recommended unless content is templated and low-risk.issue_refund — irreversible, high financial stakes, and often ambiguous (partial vs. full?). Gate required. This is the canonical checkpoint trigger.flag_for_fraud — triggers downstream action (account suspension), potentially irreversible in effect. Gate required — high stakes even if the flag itself is technically reversible.Notice that issue_refund hits all three conditions simultaneously — that's why it's the canonical example of a tool that must pause for a .
def run_refund_agent(order_id, customer_message): intent = classify_intent(customer_message) # "full_refund" order = lookup_order(order_id) # {amount: 4200, ...} policy = check_policy(order) # eligible: True result = issue_refund(order_id, order["amount"]) # fires immediately send_email(order["customer_email"], "Refund issued.") return result
classify_intent(customer_message)issue_refund(order_id, order["amount"])order["amount"]This loop classifies intent, checks policy, then calls issue_refund and send_email with no pause — the model's confidence is treated as authorization.
issue_refund fires for the full $4,200 immediately. The customer gets a confirmation email. There is no error — the code runs cleanly. The damage is done before anyone sees it. This is a silent failure: no exception, no log warning, just a wrong outcome at scale.
REFUND_AUTO_LIMIT = 50 # dollars; above this, pause for human review def run_refund_agent(order_id, customer_message): intent = classify_intent(customer_message) order = lookup_order(order_id) policy = check_policy(order) amount = resolve_refund_amount(intent, order) # partial or full if amount > REFUND_AUTO_LIMIT or intent["confidence"] < 0.90: return request_human_approval(order_id, amount, intent) result = issue_refund(order_id, amount) send_email(order["customer_email"], "Refund issued.") return result
REFUND_AUTO_LIMIT = 50intent["confidence"] < 0.90request_human_approval(order_id, amount, intent)resolve_refund_amount(intent, order)Two conditions now gate issue_refund: the refund amount exceeds the auto-limit, OR the model's confidence is below 90%. Either condition alone is enough to pause and route to a human.
$8 < $50 and confidence 0.95 ≥ 0.90, so both gate conditions are false. The agent calls issue_refund($8) automatically. This is the correct outcome — low-stakes, high-confidence, reversible-enough to automate.
def handle_fraud_signal(order_id, signals): score = compute_fraud_score(signals) # returns 0.0–1.0 if score > 0.80: # TODO: route to human review instead of auto-flagging # Hint 1: flag_for_fraud has downstream effects (account suspension) # Hint 2: use the same pattern as the refund gate above pass else: log_low_risk_signal(order_id, score) return {"action": "monitor", "score": score}
compute_fraud_score(signals)log_low_risk_signal(order_id, score)Stop — attempt the TODO before revealing the answer. The missing line is the crux: flag_for_fraud triggers account suspension downstream, making it irreversible in effect even if the flag itself can be cleared.
return request_human_approval(order_id, action="flag_for_fraud", context={"score": score, "signals": signals})
Changed lines vs. Stage 2: (1) action is now 'flag_for_fraud' not a dollar amount — the reviewer sees WHAT will happen, not just HOW MUCH. (2) context carries the raw signals so the reviewer can judge the evidence, not just the score. The gate fires on score > 0.80 — the irreversibility condition, not a dollar threshold.
Three failure modes surface repeatedly in production refund and approval workflows:
REFUND_AUTO_LIMIT is $500 instead of $50, hundreds of wrong refunds execute before anyone notices. No error appears — just a growing financial leak in the logs.flag_for_fraud looks low-risk until you trace what it triggers. Map the full action chain — not just the immediate call — when classifying reversibility.The next module — Interrupt Patterns — gives you four concrete mechanisms. Pre-action approval, post-action review, threshold-triggered pause, and exception escalation implement these gates reliably. The logic becomes a structured, resumable workflow rather than a one-off conditional.
You examine the four interrupt patterns — pre-action approval, post-action review, threshold-triggered pause, and exception escalation — and trace each through the refund agent's tool-call sequence with a worked code pattern.
Covers the four interrupt patterns — pre-action gate, post-action review, threshold pause, and exception escalation — and shows how to implement each in a refund agent loop.
Why this matters: Choosing the right interrupt pattern is the core engineering decision in any human-in-the-loop system; the wrong choice either blocks reviewers with noise or lets risky actions slip through unchecked.
Module 1 gave you the answer: impact and reversibility. High-impact, hard-to-undo tool calls sit at the controlled end of the . That's where a belongs.
This module answers the next question: once you know a checkpoint is needed, which kind do you use? Four distinct interrupt patterns exist. Picking the wrong one either over-burdens reviewers or lets risky actions slip through.
is a rule that decides when the agent loop pauses and hands control to a human.
Walk through the refund agent handling a $620 return request and see where each pattern fires.
lookup_order(order_id) — low impact, reversible. No interrupt; the agent runs it freely.issue_refund(amount=620) — high impact, irreversible. A fires: the loop pauses, shows the proposed call to a human, and waits for approval before the money moves.send_confirmation_email() — medium impact, reversible (email can be recalled). A queues the sent email for a supervisor to spot-check, but the agent continues immediately.issue_refund(amount=30) — same tool, but below the $100 threshold. The threshold-triggered pause does not fire; the agent handles it autonomously.lookup_order(order_id='???') — the order ID is malformed and the tool returns an error the agent can't resolve. Exception escalation fires: the agent surfaces the ambiguity to a human rather than guessing.Notice that the same tool (issue_refund) triggers different patterns depending on the amount — threshold logic lets you tune granularity without writing separate tools.
def run_agent_loop(task, tools, reviewer): while True: action = agent.plan(task) # agent picks next tool call if action.requires_approval: # pre-action gate check decision = reviewer.ask(action) if decision != "approved": return {"status": "rejected", "action": action} result = action.tool.execute() # tool runs only after approval task.update(result) if agent.is_done(task): return {"status": "done"}
action.requires_approvalreviewer.ask(action)action.tool.execute()task.update(result)This skeleton shows the minimal pre-action gate: the agent plans an action, the gate checks a flag, and the tool only executes if a human approves.
The key structural point is that execute() sits after the approval block — not inside it — so a rejection exits the loop cleanly.
It never runs. The function returns early with status 'rejected' on line 7, so execute() on line 8 is skipped entirely. This is the core guarantee of a pre-action gate: the tool cannot fire without explicit approval.
REFUND_THRESHOLD = 100 # dollars def gate_for_refund(action, reviewer): amount = action.params.get("amount", 0) if amount > REFUND_THRESHOLD: # threshold-triggered pause return reviewer.ask(action) return "approved" # below threshold: auto-approve def handle_tool_error(action, error, reviewer): if error.is_unresolvable: # exception escalation return reviewer.escalate(action, error) raise error # re-raise recoverable errors
action.params.get("amount", 0)reviewer.escalate(action, error)raise errorStage 2 adds the two remaining patterns alongside the gate from Stage 1.
Threshold logic lives in a separate function so you can tune REFUND_THRESHOLD without touching the loop. Exception escalation is a distinct path — it fires on agent confusion, not on parameter size.
$99 → auto-approved (below threshold, returns 'approved' without calling reviewer). $101 → pauses and calls reviewer.ask(). The threshold is a strict greater-than check, so exactly $100 also auto-approves.
def run_refund_action(action, reviewer, audit_log): amount = action.params.get("amount", 0) if amount > REFUND_THRESHOLD: decision = reviewer.ask(action) audit_log.record(action, decision) # always log the decision if decision != "approved": return {"status": "rejected"} # TODO: call the refund tool and return its result # Hint: use action.tool.execute() and wrap the return value # in a dict with key "status": "done" and key "result".
audit_log.record(action, decision)# TODOThis is a completion exercise: the gate, threshold check, and audit log are already wired — your job is to add the one line that actually runs the tool and return its result.
The crux is recognising that execute() must come after the approval block, and the return shape must match the rejection path so the caller stays simple.
Changed lines:
result = action.tool.execute() # ← executes the refund tool
return {"status": "done", "result": result} # ← wraps output
Why: execute() is the only way to actually invoke the tool (as shown in Stage 1). Wrapping in a dict with 'status':'done' keeps the return shape consistent with the rejection path, so callers don't need to branch on type.
Click a query to see which interrupt pattern fits that risk/frequency profile. Points closer together share similar tradeoffs.
Three failure modes repeat in production. Know what they look like before they hit yours.
When reviewing AI-generated gate code, check three things: (1) is execute() unreachable on rejection paths? (2) does every reviewer.ask() call have a timeout? (3) are threshold boundaries tested?
With four patterns in place, the next challenge emerges: what happens to agent memory during the pause? Conversation history, tool outputs, and pending parameters must survive. Module 3 on state management tackles this.
You learn how to serialize and restore agent state — conversation history, tool outputs, pending action parameters, and approval metadata — so the workflow resumes correctly after a human responds. The refund agent's mid-workflow pause is the worked example.
How to serialize and restore the four components of agent state — conversation history, tool outputs, pending action parameters, and approval metadata — so a paused workflow resumes correctly after a human responds.
Why this matters: Without durable state, a human-in-the-loop pause silently corrupts the workflow: the agent re-runs work, executes wrong parameters, or loses the reviewer's decision entirely — making your approval gate unreliable.
Answer: the . When the refund exceeds the limit, the agent halts before issuing the refund and waits for approval. That waiting period is the gap this module fills: the agent's state must survive it.
Module 2 showed you how to interrupt. This module answers what you do with the agent's memory while it sits paused — and how you restore it cleanly when the human responds.
At a , four components of agent state must be persisted so the workflow can resume correctly after a human responds.
refund_amount=340, order_id='ORD-8821') so the approved action is executed as intended.Miss any one of these and the resumed agent either re-runs work it already did, executes the wrong parameters, or loses the reviewer's decision entirely.
Storing agent state only in a Python dict or local variable is fragile. Process restart wipes it. The agent resumes with no history, no tool outputs, and no pending action record.
The core failure: the agent silently restarts from scratch. It re-classifies intent, re-calls customer lookup, and may issue a second refund. Or it stalls because the linking the human's response to the paused run no longer exists.
The fix: . Write the checkpoint record to an external store (database, queue, or Temporal) before pausing. Read it back on resume.
When the refund agent hits a $300+ approval threshold, it writes a before pausing. Each field serves a purpose.
run_id — unique identifier for this agent run; routes the human's response back to the paused workflow.conversation_history — full message list up to pause; the model reads this on resume and avoids re-reasoning.tool_results — cached outputs from already-called tools; skipped on resume to prevent duplicate API calls.pending_action — exact tool name and arguments frozen at pause; the approved action executes these, not a re-reasoned version.approval_metadata — reviewer ID, timestamp, decision, and parameter edits; written here first, then appended to audit trail on completion.expires_at — deadline for human response; enforces the so stale approvals don't execute days later.A reviewer opening this record sees full context — customer request, agent findings, and exact intended action — without re-reading chat.
import uuid, time, json def pause_for_approval(history, tool_results, pending_action, store): record = { "run_id": str(uuid.uuid4()), "conversation_history": history, "tool_results": tool_results, "pending_action": pending_action, # {"tool": "issue_refund", "args": {...}} "approval_metadata": {"status": "pending", "reviewer": None}, "expires_at": time.time() + 86400, # 24-hour window } store.put(record["run_id"], json.dumps(record)) return record["run_id"] # hand this token to the reviewer UI
store.put(key, value)json.dumps(record)"expires_at": time.time() + 86400return record["run_id"]This function freezes the agent's full state and writes it to an external store before the process yields control. The returned run_id is the — the reviewer's response must carry it back so the right paused run is resumed.
Yes — store.put() already wrote the record to the external store before the crash. The run_id is lost from the local call stack, but the reviewer UI can still look up the record by run_id. This is exactly why you write to the store BEFORE returning the token, not after.
def resume_after_approval(run_id, reviewer_decision, store): record = json.loads(store.get(run_id)) if time.time() > record["expires_at"]: raise TimeoutError(f"Checkpoint {run_id} expired — action cancelled.") record["approval_metadata"].update(reviewer_decision) # {"reviewer": "ana", "status": "approved"} store.put(run_id, json.dumps(record)) # persist the decision if record["approval_metadata"]["status"] != "approved": return {"outcome": "rejected", "run_id": run_id} action = record["pending_action"] return execute_tool(action["tool"], action["args"]) # use frozen args, not re-reasoned ones
store.get(run_id)record["approval_metadata"].update(reviewer_decision)action["args"]raise TimeoutError(...)This is the delta from Stage 1: we read the checkpoint back, validate the expiry, stamp the reviewer's decision into , and execute the frozen pending action — not a re-reasoned one. Writing the decision back to the store before executing is what makes the audit trail durable.
The refund executes but the approval_metadata is never persisted. If the process crashes between execute_tool and store.put, the audit trail shows 'pending' forever — compliance sees no record of who approved the $340 refund. Always write the decision to the store before executing the action.
def pause_for_approval_v2(history, tool_results, pending_action, store, timeout_hours=48): record = { "run_id": str(uuid.uuid4()), "conversation_history": history, "tool_results": tool_results, "pending_action": pending_action, "approval_metadata": {"status": "pending", "reviewer": None, "edited_args": None}, # TODO: add the expires_at field using timeout_hours # TODO: write the record to store and return the run_id } pass # replace with your implementation
timeout_hours * 3600"edited_args": NoneStop — attempt this before revealing. The structure is complete except for the two lines that make the checkpoint durable and time-bounded.
Hints: expires_at should use timeout_hours (not a hardcoded 86400). The store write and return follow the same pattern as Stage 1 — but this version also supports edited_args in the metadata, so a reviewer can amend the refund amount before approving.
Changed lines:
"expires_at": time.time() + timeout_hours * 3600, # uses the parameter, not a magic number
store.put(record["run_id"], json.dumps(record))
return record["run_id"]
Why edited_args matters: when the reviewer changes the refund from $340 to $200, the resume handler should check edited_args first and use those instead of the original pending_action["args"]. Without this field, reviewer edits are silently discarded and the original amount executes.
Slide through storage options for checkpoint records. Higher tiers survive more failure scenarios but add operational overhead.
You trace how a human's approve/reject/edit decision is injected back into the agent's context as a corrected instruction or amended tool call, and how the loop distinguishes a one-shot correction from an iterative review cycle. The refund agent's edit-and-retry path is the completion example.
How a human's approve/reject/edit decision is injected back into the agent's context, and how the loop terminates safely.
Why this matters: Without a correct injection mechanism and a termination guard, human corrections either get ignored or spin the agent in an infinite retry cycle — both break the HITL contract.
Decision this forces: Should a human correction restart the full agent loop, branch to a sub-task, or patch the pending action in place?
Answer: conversation history, tool outputs, pending action parameters, and . Module 3 showed you how to freeze that state; this module shows what happens when the human's decision changes it.
A human's approve/reject/edit response isn't just a gate signal — it's new information that must enter the agent's context as authoritative instruction. The question this module answers: how does that correction land, and what does the loop do next?
When a human edits a pending action, inject the corrected instruction as a new message in the agent's conversation history. Use system or tool role so the model treats it as ground truth.
The injection point matters: it goes after the assistant's last message and before the next model call. The model sees the correction as the most recent context.
Three injection shapes cover most cases:
A one-shot correction resolves in a single human turn: the reviewer edits the refund amount, the agent executes, and the loop ends. An iterative review cycle means the corrected output is itself reviewed again — useful when the human's first edit may still be wrong or incomplete.
Every iterative loop needs an explicit or it cycles forever. Common conditions: human approves, max-revision count reached, or a timeout fires.
A refund request comes in for $340. The agent proposes issue_refund(amount=340, reason='billing_error') and pauses for review (the $300 threshold from module 1).
The reviewer edits the amount to $280 — within policy — and approves. The system patches the pending call to issue_refund(amount=280, reason='billing_error'), appends a correction message to history, and executes without re-invoking the model.
Now imagine the reviewer instead rejects the refund entirely, citing 'no billing error found'. The system injects a system message: 'Refund rejected by reviewer: no billing error confirmed. Re-evaluate the customer's claim.' The model re-plans — it may now route to a human-agent handoff instead.
A max-revision guard (e.g. 3 attempts) ensures that if the model keeps proposing a refund, the loop escalates rather than cycling.
def apply_human_decision(history, pending_call, decision): if decision["action"] == "approve": return history, pending_call # execute as-is if decision["action"] == "edit": patched = {**pending_call, **decision["overrides"]} history.append({"role": "system", "content": f"Reviewer edited call: {patched}"}) return history, patched # execute patched call # reject: inject correction, signal re-plan history.append({"role": "system", "content": decision["reason"]}) return history, None # None → re-invoke model
{**pending_call, **decision["overrides"]}history.append({"role": "system", ...})return history, NoneThis function is the injection point: it takes the current history, the pending tool call, and the human's decision, and returns an updated history plus either a (possibly patched) call or None to signal a re-plan.
On edit, the correction message enters history before execution so the model's next turn sees the reviewer's reasoning as authoritative context.
{"tool": "issue_refund", "amount": 280, "reason": "billing_error"} — the amount is overwritten; tool and reason are preserved from the original call.
MAX_REVISIONS = 3 def review_loop(agent, history, get_human_decision): for attempt in range(MAX_REVISIONS): pending_call = agent.propose(history) decision = get_human_decision(pending_call) history, call = apply_human_decision(history, pending_call, decision) if call is not None: result = agent.execute(call) history.append({"role": "tool", "content": result}) return history, result # ✅ done # call is None → re-plan; loop continues # TODO: replace this line with an escalation call raise RuntimeError("Max revisions reached — escalate")
for attempt in range(MAX_REVISIONS)agent.propose(history)if call is not Nonehistory.append({"role": "tool", ...})This loop wires Stage 1's injection function into a bounded retry cycle. The TODO on the last line is your completion task: replace the RuntimeError with a real escalation call that routes the case to a human agent and records the failure in the audit trail.
Changed line: agent.escalate(history, reason='max_revisions_exceeded', route_to='human_agent')
Why: RuntimeError crashes the process silently — a real escalation call routes the case gracefully, preserves history for the next reviewer, and logs the reason. The 'reason' and 'route_to' fields are the minimum the audit trail needs. If your escalation also writes to a checkpoint record, add checkpoint_id=current_checkpoint_id.
If the correction lands as a user message instead of system, the model may treat it as a suggestion. It re-proposes the original action unchanged. Fix: always inject corrections at system role.
An edit that patches arguments but also re-invokes the model causes re-derivation. The model discards the reviewer's edit and generates different arguments. The patched call executes with wrong values — no error is raised. Fix: on edit, execute the patched call directly. Only reject should trigger a re-plan.
A reject loop without a max-revision guard runs until the process is killed. The context window fills and token costs escalate. No resolution occurs. Fix: set MAX_REVISIONS and escalate when it's hit.
The next module defines what context and actions the reviewer interface must expose. This ensures decisions are made with full information, not guesswork.
You define the interface contract a HITL system must fulfill — what context to surface, what actions to expose, and how to record the decision — and audit the refund agent's review screen against a checklist of required elements.
Defines the interface contract a human review UI must fulfill — what context to show, what actions to expose, and how to record the decision for audit.
Why this matters: Without a well-specified review screen, human oversight breaks down: reviewers guess, agents execute malformed actions, and decisions can't be reconstructed — defeating the entire HITL system.
Module 4 called this . The human's decision is written back into the agent's context as an amended instruction or corrected tool call. The resumes from that point.
That mechanism only works if the reviewer had enough information to make a confident decision. That's the gap this module closes. What must the review screen show? What must it let the reviewer do?
An is the minimum set of obligations a review UI must fulfill. A human can then make a confident, auditable decision. It has three parts: surface the right context, expose the right actions, and record the outcome.
Missing any one of these three obligations creates a gap. Either the reviewer guesses, the agent misbehaves, or the decision can't be reconstructed later.
The refund agent's current review screen shows: the proposed refund amount, an Approve button, and a Reject button. Walk through the interface contract checklist against it.
Six of seven contract elements are absent or unverifiable. A reviewer using this screen is forced to guess on risk, context, and time — exactly the conditions that produce and rubber-stamp approvals.
def submit_review_decision(checkpoint_id, reviewer_id, action, edited_params=None): record = load_checkpoint(checkpoint_id) # fetch saved agent state decision = { "reviewer": reviewer_id, "timestamp": utc_now(), "action": action, # "approve" | "reject" | "edit_approve" "edited_params": edited_params, } record["approval_metadata"] = decision # TODO: validate edited_params and inject into agent context before resuming write_audit_trail(checkpoint_id, decision) return resume_agent(record)
load_checkpoint(checkpoint_id)record["approval_metadata"] = decisionwrite_audit_trail(checkpoint_id, decision)resume_agent(record)edited_params=NoneThis fragment shows the full decision-recording pipeline: load state, attach , write the , then resume. The TODO is the crux of edit-and-approve — fill it in before revealing.
Replace the TODO with:
if edited_params:
validate_params(edited_params) # raises ValueError if malformed
record["pending_action"]["params"].update(edited_params) # correction injection
Changed lines vs. the stub: the TODO is replaced by a guard + an in-place update of record["pending_action"]["params"]. This is the crux: without validate_params(), a reviewer can submit a negative refund and the agent executes it; without the .update(), the edit is logged but the agent still runs the original amount.
Three failure patterns account for most real-world review UI breakdowns.
Drag to see how many interface contract elements are visible and what that means for decision quality and reviewer behavior.
A complete interface contract closes the gap between the agent's decision and the human's judgment. But the UI is only one layer of the system. It can't compensate for a broken . It can't fix a corrupted checkpoint or a reviewer queue that's grown too large.
Those are system-level failure modes, not UI gaps. The next module diagnoses all five: timeout with no fallback, state corruption on resume, from over-gating, silent approval bypass, and paths that dead-end. It maps a recovery path for each.
You diagnose the five common HITL failure modes — timeout with no fallback, state corruption on resume, reviewer fatigue from over-gating, silent approval bypass, and missing audit trail — and design escalation paths that prevent each. You also verify whether an AI-generated checkpoint implementation covers these cases.
Diagnose the five HITL failure modes and design escalation policies that prevent each one.
Why this matters: Even a well-designed checkpoint system breaks in predictable ways — knowing the failure modes lets you build one that holds up under real operational pressure.
Your refund agent has checkpoints, state management, and a review UI. Five structural failure modes can still bring it down.
Each mode has a distinct cause and fix. Diagnosing them separately is what makes the difference between a patch and a real escalation policy.
Here is how each failure mode surfaces concretely in the refund-processing workflow — and what the observable symptom looks like, not just the abstract cause.
PENDING_REVIEW forever. The customer's ticket ages out. Symptom: support dashboard shows the ticket open for 72 h with no action taken.issue_refund(amount=900, account_id=None). Symptom: the refund API throws a validation error; the agent retries and double-charges.timeout policy has three parts: a deadline, an escalation target, and a fallback action.
reviewer fatigue: gate only actions above a meaningful risk threshold. Not every refund — only those above the irreversibility line from module 1.
def request_review(checkpoint, timeout_hours=4): send_to_reviewer(checkpoint, queue="fraud_team") response = wait_for_response(checkpoint.id, timeout=timeout_hours * 3600) if response is None: escalate(checkpoint, target="senior_manager", fallback="reject") return {"status": "escalated", "action": "reject"} persist_audit(checkpoint.id, response) # always write before returning return {"status": "reviewed", "decision": response.decision}
wait_for_response(..., timeout=...)escalate(checkpoint, target=..., fallback=...)persist_audit(checkpoint.id, response)This pattern covers three of the five failure modes in one function: timeout with no fallback (the if response is None branch), missing audit trail (persist_audit before returning), and escalation path (the escalate() call with an explicit fallback).
Notice that persist_audit is called on BOTH paths — a common bug is only writing the audit record on the happy path.
Successful approvals are never written to the audit log. The system works operationally but leaves no record of who approved what. A compliance query returns rows only for escalated cases — approved refunds are invisible. This is the 'missing audit trail' failure mode on the happy path.
def resume_after_approval(checkpoint_id): record = load_checkpoint(checkpoint_id) # load from durable store # TODO: validate that record contains all required fields # before calling issue_refund — what check goes here? issue_refund( amount=record["amount"], account_id=record["account_id"], approved_by=record["reviewer_id"] ) mark_complete(checkpoint_id)
load_checkpoint(checkpoint_id)mark_complete(checkpoint_id)Stop — attempt this before revealing. The TODO is the crux of the 'state corruption on resume' failure mode. Hint 1: what fields does issue_refund require? Hint 2: what should happen if any field is missing or None?
# CHANGED LINES — the guard that prevents state corruption on resume:
required = ["amount", "account_id", "reviewer_id"]
missing = [f for f in required if not record.get(f)]
if missing:
raise ValueError(f"Corrupt checkpoint {checkpoint_id}: missing {missing}")
# Why: without this, a None account_id reaches issue_refund and causes a
# silent bad call or a downstream API error that's hard to trace back here.
Each point is a checkpoint configuration. Click a query to find the nearest real-world tradeoff zone. Points close together share a risk/load profile.
When an AI tool generates your checkpoint implementation, run it against this five-point checklist before production.
response is None with explicit escalation and fallback?persist_audit called before every return — not just the happy path?issue_refund) — each must pass through the gate function.AI-generated HITL code most often fails items 2 and 5. The audit write on the timeout path and unguarded alternate code paths are the two gaps LLMs most reliably miss. You now have the full toolkit to build and verify a production-ready HITL workflow.
Before looking at the summary: reconstruct from memory the five components that must be in a checkpoint record, name the four interrupt patterns in order of when they fire relative to the action, and describe how a human correction re-enters the agent loop. Write it out, then compare.
Apply what you learned to Human-in-the-Loop Agents.
An agent is about to send a bulk email to 14,000 customers with a promotional discount. The action is accurate — the model correctly identified the right segment and the right offer. Should a human gate be required, and why?
Accuracy does not eliminate liability. A bulk email is both high-impact (reaches thousands of people) and irreversible (you cannot un-send it), which independently justifies a checkpoint even when the model is correct. Option A conflates accuracy with safety — a correct action can still cause irreversible harm at scale. Option C misidentifies the criterion as 'external communication' rather than impact and reversibility. Option D is wrong because irreversibility and high stakes each individually justify a gate, independent of model uncertainty.
Consider this simplified agent loop logic:
if action.risk_score > THRESHOLD:
return escalate_to_human(action)
if action.is_exception:
return escalate_to_human(action)
A new action arrives with risk_score = 0.3 (below threshold) but it is flagged as an exception. Which interrupt pattern fires, and what is the correct behavior?
The two checks are independent conditionals evaluated in sequence. The risk score of 0.3 does not trigger the threshold gate, but the exception flag hits the second condition and escalates to a human — exception escalation is designed to catch cases that fall outside normal risk scoring. Option A is wrong because the threshold check is not the only criterion. Option B is wrong because the code reaches the second conditional even when the first is false. Option D is wrong because the two patterns are distinct mechanisms, not a combined gate — only one needs to fire to pause the agent.
An agent is paused at a checkpoint waiting for human approval. The server restarts before the reviewer responds. When the agent process comes back up, it has no record of the pending action. What is the root cause of this failure?
In-memory state is lost on any process interruption — restart, crash, or scale-down. The fix is to persist the full checkpoint record to durable storage before handing off to the human. Option A describes a UI problem, not a state problem — the action was never stored, so no UI failure is needed to explain the loss. Option C confuses a timeout policy (a separate failure mode) with the cause of data loss on restart. Option D is a partial concern about checkpoint completeness, but the core issue here is that nothing was persisted at all, not that a field was missing.
A human reviewer rejects an agent's proposed database query and types a corrected version. Describe how you would inject that correction so the model treats it as authoritative, and explain when this correction should trigger a new pre-action gate versus proceeding directly.
Injecting into message history is the mechanism that makes the model treat the correction as ground truth rather than a suggestion. The gate decision reuses the interrupt-pattern logic from module 2: the corrected action is evaluated on its own impact and reversibility, not automatically cleared just because a human touched it.
You are auditing a review UI for an agent that can approve vendor payments. The UI shows: the vendor name, the payment amount, and two buttons — Approve and Reject. Which gap most directly creates a decision error and an audit failure?
A reviewer who sees only a vendor name and amount has no basis for a confident decision — they cannot verify why the agent chose this vendor or this amount, and there is no audit trail of the evidence. This violates the minimum information requirement of the interface contract. Option B is also a real gap (Edit-and-Approve is a required UI action), but it affects correction capability, not the reviewer's ability to make an informed decision in the first place — the question asks for the gap that most directly causes a decision error. Option C invents a requirement (confidence scores are not universally mandated). Option D mischaracterizes what Reject should do — it returns control to the agent or workflow, not necessarily to a senior reviewer.