Know when planning helps agents and when it only adds latency.
Trace a reactive agent handling a direct lookup task — observe exactly where latency lives and what the loop costs with zero planning overhead. This is the baseline every planning decision must beat.
Traces a reactive agent loop on a direct lookup task to establish the latency and token-cost baseline.
Why this matters: Every planning decision in this lesson is measured against this baseline — you cannot evaluate planning overhead without knowing what the bare loop actually costs.
Your agent returned a stale answer. It fired one tool call, read one observation, and stopped. No check. No retry. No awareness the source was outdated. That is : the model reads the goal, emits an action, receives an , and decides whether the is met. Repeat until done or capped.
The cost structure is simple: one LLM call per turn. Each carries (goal + history + latest observation) tokens. Latency is dominated by the slowest tool call in the critical path, not model inference.
On a direct lookup — fetch a record, summarise a document, answer a factual question — the reactive is the cheapest architecture. Zero planning overhead. One round-trip. Predictable token spend.
Task: retrieve the current price of a product SKU from an inventory API and return it to the user.
Turn 1 — the model receives the goal and emits a single tool call: lookup_inventory(sku="B-4471"). The API responds in 340 ms. The model reads the observation and emits a final answer. Total wall time: ~700 ms (340 ms tool + ~360 ms inference). Token spend: ~620 input, ~80 output.
Now the task changes: the user asks for the price AND the restock date AND the supplier contact. The model still has no plan — it fires lookup_inventory, reads the observation, realises it needs more, fires lookup_supplier, and so on. Three sequential tool calls, each waiting on the previous observation.
Total wall time: ~2 100 ms — three serial RTTs that a planning step could have parallelised. This is the latency gap that planning must justify its own overhead against.
goal = "Get price, restock date, and supplier for SKU B-4471" history = [] while True: response = llm(goal=goal, history=history) if response.done: print(response.answer); break obs = execute_tool(response.tool_call) # blocks until API returns history.append((response.tool_call, obs)) # context grows: each turn re-sends full history
llm(goal=goal, history=history)execute_tool(response.tool_call)history.append((response.tool_call, obs))if response.doneThis loop has no parallelism and no plan — every tool call blocks on the previous before the model can decide the next action.
The comment on the last line is the non-obvious cost: the full history is re-sent on every turn, so a 3-turn loop on a 500-token base context sends 500 + 700 + 900 = 2 100 input tokens, not 3 × 500.
Turn 3 input = 500 (base) + 200 (obs from turn 1) + 200 (obs from turn 2) = 900 tokens. The history is cumulative — each turn re-sends everything prior, not just the latest observation.
Drag to see how token input and latency compound as the reactive loop deepens — before any planning overhead is added.
The model cannot issue concurrent tool calls without explicit orchestration. Three independent lookups run sequentially, tripling wall time. Latency scales linearly with subtask count — serialisation is the bottleneck, not model speed.
On loops beyond 4–5 turns, early observations compete with the goal in the model's attention. The symptom: the model re-calls a tool it already ran, or contradicts an earlier observation. This is retrieval failure inside the context window, not hallucination.
Without a hard iteration cap, a reactive loop on an ambiguous goal can run until the context limit is exhausted. The failure is silent: the model keeps emitting tool calls, cost accumulates, and the user sees nothing until the cap is hit.
if response.done; (2) history is not unbounded — confirm truncation or summarisation strategy; (3) parallelisable tool calls are not serialised by the loop structure.goal = "Summarise all open support tickets for account A-99" history = [] MAX_TURNS = 5 for turn in range(MAX_TURNS): response = llm(goal=goal, history=history) if response.done: print(response.answer); break obs = execute_tool(response.tool_call) history.append((response.tool_call, obs)) # TODO: what happens if the loop exhausts MAX_TURNS without response.done?
for turn in range(MAX_TURNS):for ... else:raise LoopBudgetExceeded(..., partial=partial)Stop here and write the missing guard before revealing. The crux is what the caller receives when the model never emits done=True within the turn budget — this is the failure mode from the previous block made concrete.
Add after the for-loop:
else:
partial = summarise(history) # distil what was gathered
raise LoopBudgetExceeded(f"Stopped at {MAX_TURNS} turns", partial=partial)
Changed lines vs. the worked example: the 'else' clause on the for-loop fires only when the loop exhausts without a 'break' — Python's for/else semantics. The key insight: surface the partial result rather than silently returning None, so the caller can decide whether to retry with a plan or accept the partial answer.
A single-turn reactive call is a degenerate loop case: one LLM call, one tool call, one observation, done. Token cost is fixed and predictable. Latency is one tool RTT plus one inference call.
A multi-turn loop compounds both: input tokens grow as O(n × avg_obs_size) across n turns. Latency is the sum of all tool RTTs on the critical path — serial by default.
This crossover is what the next module quantifies. Upfront adds one model call and a to the budget. It can unlock parallel execution and shrink per-step context. The tradeoff only makes sense once you know the reactive baseline you are beating.
Map the mechanics of upfront decomposition: the extra model call, the plan representation, and how downstream steps consume it. Understand the latency and token overhead before evaluating whether any task justifies it.
Maps the mechanics and cost profile of upfront planning: the extra model call, plan representation choices, and how downstream steps pay for both.
Why this matters: Before adding planning to any agent pipeline, you need to know exactly what you are paying — in latency, tokens, and replanning risk — so you can judge whether a task justifies it.
In a , latency is dominated by the model inference call and the tool round-trip. The loop bookkeeping itself is negligible. Each step is one model call plus one tool call. Nothing runs before the first observation arrives. That baseline is the cost floor every planning strategy must justify beating.
Upfront inserts at least one additional model call before any tool executes. This is a call. Its output is a consumed by every downstream step.
That call carries its own prompt, its own output tokens, and its own inference latency — typically 300–800 ms at moderate plan sizes. Downstream steps then pay a second cost: each step must read the plan. The plan's token count is added to every subsequent context window.
For a 5-step pipeline with a 400-token plan, that's ~2,000 extra tokens billed across the run. This is on top of the planning call itself. The overhead is fixed at plan creation and then amortized — or wasted — depending on how many steps actually execute.
A flat list encodes steps in sequence. A dependency graph encodes which steps block which others. The choice determines cost when an diverges from the plan's assumptions.
The non-obvious cost: a dependency graph inflates every downstream context window more than a flat list does. The full graph must be present for the agent to know which nodes are unblocked. For short, linear tasks, the graph's replanning advantage never materializes. You pay the serialization cost for nothing.
An agent must answer: "Summarize the competitive landscape for three SaaS pricing models." A planner decomposes this into: fetch pricing pages (×3), extract key data, compare, synthesize. That's a 5-node flat-list plan.
The planning call costs ~600 ms and emits a 320-token plan. Each of the 5 downstream steps carries that 320-token plan in its context — 1,600 extra tokens billed. One fetch fails (a paywalled page). The flat list has no dependency metadata. The agent replans from scratch: another 600 ms + 320 tokens.
Total planning overhead: ~1,200 ms wall-clock, ~1,920 extra tokens. A reactive agent handling the same failure would have retried that one step and continued. No replanning call. No context inflation. The planning overhead only breaks even if the upfront decomposition prevented at least two reactive retries elsewhere in the pipeline.
Drag to see how planning overhead amortizes (or compounds) as pipeline depth grows. Planning latency is fixed; per-step context inflation grows linearly.
A planner that emits 12 sub-tasks for a 3-step job — — bloats every downstream context window. It multiplies model calls. You won't see an error. You'll see a 4× token bill and a 3× latency spike with no quality gain. Watch for plan token counts that exceed the sum of all tool outputs.
The plan is generated from the goal prompt alone — before any observations arrive. If step 3's output contradicts a plan assumption, a flat-list agent continues executing steps 4–5 against a now-invalid premise. The final answer looks coherent but is built on stale scaffolding. No exception is raised. The output is just wrong.
If the task environment is highly dynamic, tool outputs vary widely per call. A trigger on every divergent turns the planning overhead from a fixed cost into a compounding one. Each replan resets the context and re-bills the planning tokens. A reactive agent with logic is cheaper here.
import time def run_with_planning(goal, steps_fn, plan_fn): t0 = time.perf_counter() plan = plan_fn(goal) # extra model call plan_ms = (time.perf_counter() - t0) * 1000 plan_tokens = len(plan["steps"]) * 64 # rough estimate results = [] for step in plan["steps"]: obs = steps_fn(step, context=plan) # plan in every ctx results.append(obs) return results, {"plan_ms": plan_ms, "plan_tokens": plan_tokens}
plan_fn(goal)context=planplan_tokens = len(plan["steps"]) * 64This snippet isolates the two cost components of planning: the fixed latency of the planning call itself, and the per-step token inflation from passing the plan into every downstream context.
Instrument both before deciding whether planning is justified — wall-clock cost and token cost move independently and hit different budget constraints.
Minimum total: ~500 ms (plan call) + 6 × 400 ms = ~2,900 ms. The reactive agent pays only 6 × 400 ms = 2,400 ms. Line 4 (plan = plan_fn(goal)) is the fixed overhead — it runs once, unconditionally, before any tool executes. The context=plan argument on line 9 is the per-step variable overhead (token inflation), but it doesn't add wall-clock time directly.
You now have the cost model: one fixed planning call, per-step context inflation, and a replanning multiplier that activates on observation divergence. The question this module deliberately leaves open is: under what task conditions does absorbing that overhead produce a net win?
The next module works through a multi-step research-and-synthesis task. Ambiguous goals and error-recovery needs make upfront decomposition cheaper than reactive thrashing. It shows the concrete conditions that tip the balance.
Work through a multi-step research-and-synthesis task where ambiguous goals and error-recovery needs make upfront decomposition cheaper than reactive thrashing. Pinpoint the three task signals — complexity, ambiguity, error-recovery — that flip the cost-benefit calculation.
Identifies the three task signals — complexity, ambiguity, and error-recovery surface — that make upfront decomposition cheaper than reactive thrashing.
Why this matters: Gives you a concrete cost model to decide when planning reduces total token spend rather than just adding latency overhead.
Decision this forces: Does this task's complexity or ambiguity make upfront decomposition cheaper than reactive thrashing?
Answer: an extra model call to produce the (latency + tokens) and the per-step context overhead as downstream steps consume that plan. Every planning decision must beat this baseline — the same reactive baseline Module 1 measured.
This module asks: under what task conditions does that overhead pay off? The answer turns on three signals, and getting them wrong in either direction is expensive.
: complexity means many interdependent sub-tasks. ambiguity means the goal is underspecified enough that a reactive agent will misinterpret it mid-run. error-recovery surface means failures at step N invalidate downstream steps, so reactive retry is expensive.
None of these signals alone is decisive. They compound. A complex task with a clear goal and idempotent steps may still be cheaper to run reactively. The threshold is crossed when two or more signals are present at once.
Task: "Produce a competitive landscape report on three LLM API providers — compare pricing, rate limits, and latency benchmarks — then draft an executive summary with a recommendation."
All three signals fire here. Complexity: nine distinct data-fetch sub-tasks, three providers times three dimensions, plus a synthesis step with ordering constraints. Ambiguity: "recommendation" is underspecified. The planner can surface that ambiguity and resolve it before any tool fires. Error-recovery surface: if the rate-limit fetch for provider B fails, a reactive agent retries the whole loop. A plan graph retries only that branch and checkpoints the rest.
iteration. Tokens are spent, latency is added, and convergence is still not guaranteed.
pays for itself here. One planning call resolves the ambiguity, lays out the fetch graph, and makes every subsequent step deterministic. The plan overhead, one model call plus about 300 tokens, is smaller than two reactive backtracks.
Click a query task to see which reference tasks sit nearest — tasks that cluster near a query share its planning profile. Axes: x = ambiguity (0 = fully specified, 100 = open-ended); y = error-recovery surface (0 = idempotent/cheap retry, 100 = cascading failure cost).
from the top. The model re-reads the full context, re-infers what was already done, and re-decides what to try next. At N completed steps, that re-inference costs O(N) tokens every time.
externalises the dependency graph. When step K fails, the agent reads the plan, identifies which downstream steps depend on K, marks them blocked, and retries only K. The rest of the graph is unaffected. This is the structural advantage. Failure scope is bounded at plan-construction time, not discovered reactively.
# Reactive baseline (from Module 1) reactive_cost = base_loop_tokens * expected_iterations # Planning path plan_call_tokens = 300 # one decomposition call steps = ["fetch_pricing", "fetch_rate_limits", "fetch_latency", "synthesise", "draft_summary"] # 5 sub-tasks planned_cost = plan_call_tokens + sum(step_tokens[s] for s in steps) # Break-even: plan wins when reactive backtracks ≥ 1 net_saving = reactive_cost - planned_cost print(f"Plan saves {net_saving} tokens" if net_saving > 0 else "React is cheaper")
base_loop_tokens * expected_iterationsplan_call_tokenssum(step_tokens[s] for s in steps)net_saving = reactive_cost - planned_costmid-run resets the clock — a plan that triggers one replan may still lose to a reactive agent that happened to converge cleanly.
reactive_cost = 400 × 4 = 1
The planner doesn't surface the underspecified criterion. It bakes a wrong assumption into the dependency graph. Every step executes correctly against the wrong goal. The agent returns a confident, well-structured answer to the wrong question. There is no error, no signal, just a silent wrong output.
— adds serialisation overhead at each handoff. The observable symptom is wall-clock latency that is 3–4× the reactive baseline while output quality is identical. The cost-benefit model above goes negative because sum(step_tokens) grows with step count, even when individual step tokens are small.
, paying the plan-call cost a second time. If the task has high observation variance — API responses that frequently diverge from expectations — reactive execution often wins because it never commits to a stale graph.
Diagnose a time-sensitive lookup and a well-defined single-step operation where planning overhead doubles latency with zero quality gain. Revisit the reactive baseline from m1 to confirm that the simplest path wins here.
Identifies the task properties — single-step, fully specified, time-sensitive — that make planning overhead unjustifiable and shows how to diagnose over-decomposition.
Why this matters: Prevents a common architectural mistake where adding planning to simple tasks doubles latency with zero quality gain, directly improving the cost and reliability of any agent you build.
Decision this forces: Is this task simple, time-sensitive, or fully specified enough that planning adds only latency?
The three signals are complexity (steps that can't be scripted in advance), ambiguity (goals that require interpretation before acting), and (failure paths that require replanning, not just a retry).
This module flips the lens: when none of those signals are present, is pure overhead.
How do you recognize a task where planning doubles latency and changes nothing?
planning unjustifiable: single-step (one tool call resolves it), fully specified (no ambiguity in goal or output), and time-sensitive (latency is a first-class constraint).
plan representation that maps to exactly one action the reactive baseline would take anyway.
Quality delta is zero. Latency delta is one full model call plus serialization overhead.
— five agents where one loop suffices — multiplies latency, coordination cost, and failure surface without adding capability.
A customer-facing agent answers "What is the current price of SKU-4821?" The tool is a direct database read. The goal is unambiguous. The SLA is 300 ms. A teammate proposes wrapping this in a planner that decomposes the request, identifies the relevant tool, and emits a structured plan before calling it.
Predict: does the planner improve answer quality? Does it change which tool gets called? What does it cost?
reactive baseline from module 1 calls that tool directly from the first loop iteration.
The plan is structurally identical to the reactive path. Quality is unchanged. Latency is up 30–50%. The planner fails the SLA the reactive agent would have met.
The tell: when you can write the plan by inspection before the model runs, the model call that produces it is waste.
Drag to see how planning overhead (a fixed extra model call) changes as a fraction of total latency. For a 1-step task, planning can double end-to-end time. For a 5-step task, the same overhead is a small fraction.
These failures are invisible in unit tests that only check final output. They surface in latency percentiles, token-cost dashboards, and SLA breach logs.
# Task: "What is the current price of SKU-4821?" # Both paths call the same tool. Predict: what differs? # --- Path A: reactive baseline (module 1) --- def reactive_agent(goal): obs = call_tool("price_lookup", sku="SKU-4821") return obs["price"] # --- Path B: planner-first --- def planning_agent(goal): plan = model_call(f"Decompose: {goal}") # extra inference step = plan["steps"][0] # always one step obs = call_tool(step["tool"], **step["args"]) return obs["price"]
model_call(f"Decompose: {goal}")plan["steps"][0]call_tool(step["tool"], **step["args"])model_call() in Path B — which adds latency, a new failure point, and zero quality gain on a fully-specified single-step task.
Path B pays one extra model_call() — an additional inference round-trip — before touching the tool. For this task the plan always has one step, so quality is identical. The cost becomes a correctness problem when the planning call itself is flaky or rate-limited: Path B can fail or time out on the planning step even when the tool is healthy, while Path A has no such failure surface. On a 300 ms SLA, Path B's planning call alone can exhaust the budget.
reflection — a separate model call that checks the agent's output — earn its cost any better?
self-correction loop spins without converging — the reflection equivalent of the planning overhead trap you just diagnosed.
Examine reflection mechanisms — separate model calls, structured checklists, deterministic tests — and the failure mode where reflection loops spin without converging. Apply the same cost-benefit lens from m3/m4 to decide when self-correction catches real errors versus when it burns budget.
Examines three reflection mechanisms — model-call critique, structured checklist, and deterministic test — and the failure modes that cause reflection loops to spin rather than converge.
Why this matters: Prevents runaway self-correction from burning token budget without improving output quality, and gives you a concrete gate design to stop it.
Decision this forces: Does this task's error profile justify a model-call reflection, a deterministic check, or no reflection at all?
That same cost-benefit lens now applies one tier deeper. It is not about whether you plan. It is about whether you reflect after acting.
Like planning, it has a real token and latency cost. Unlike planning, it can loop. A loop that does not converge burns budget without fixing anything.
Reflection mechanisms fall into three categories, each with a different cost-to-yield ratio.
The decision isn't which is best in general — it's which error class your task actually produces. Structural errors (malformed JSON, missing required field) are cheaper to catch with a deterministic test. Semantic errors (wrong tone, incomplete reasoning) require a model call.
Click a reflection type to see which error classes it catches well. Points closer together share similar cost-yield profiles.
Each iteration looks like progress, but the loop never reaches a stable fixed point.
Without a gate, the loop runs to timeout and charges every token.
You add a model-call reflection step that checks factual consistency across sections. After deploying, you notice costs are 3× the estimate and latency has doubled, but output quality is unchanged.
Quality is unchanged because the real errors were never the critic's target. Factual consistency across sections is a semantic property the model cannot verify reliably against external ground truth.
A cost cap, abort if cumulative reflection tokens exceed X, is the safety net, not the primary gate.
Reserve the model call for the one thing it is actually better at: checking whether the argument structure is coherent. Run it at most once.
MAX_ITERS = 2 MIN_DELTA = 0.05 # exit if <5% of tokens changed def reflect_with_gate(output, goal, cost_tracker): for i in range(MAX_ITERS): critique = model_call(f"Critique this output vs goal: {goal}\n{output}") cost_tracker.add(critique.tokens) revised = model_call(f"Revise based on critique:\n{critique.text}\n{output}") cost_tracker.add(revised.tokens) if token_delta(output, revised.text) < MIN_DELTA: break # converged or oscillating — exit early output = revised.text return output
token_delta(output, revised.text)cost_tracker.add(critique.tokens)breakThe cost tracker is threaded through so a caller can enforce a token budget cap independently of the iteration count — the two gates are orthogonal.
It runs iteration 2. The 6% delta exceeds MIN_DELTA (0.05), so the diff check does NOT trigger an early exit. The loop continues to iteration 2, then the hard cap (MAX_ITERS=2) stops it. This is why the diff threshold and the iteration cap are both necessary — neither alone is sufficient.
| Option | Error class caught | Hallucination risk | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Model-call reflection | Semantic, reasoning, tone, subtle logic | Critic can invent flaws or miss real ones | Output has semantic, reasoning, or tone errors that can't be specified as a rule — e.g. argument coherence, factual consistency against a known corpus. | High — 1–2 extra model calls per reflection pass, multiplied by iteration count. | High — requires iteration cap, diff gate, and cost cap to prevent spin. |
| Deterministic test | Structural, schema, range, format errors only | None — fully deterministic | Output has structural or verifiable errors — schema validity, required fields, numeric ranges, citation presence. Use as the first gate before any model-call reflection. | Near-zero — pure code execution, no model tokens. | Low — write once, runs in microseconds, no convergence risk. |
| No reflection | None — errors pass through unchecked | N/A — no reflection call to hallucinate | Task is single-step, latency-sensitive, or the error cost is lower than the reflection cost — e.g. a lookup, a classification with a downstream human review. | Zero. | Zero — but shifts error-detection burden to the caller or downstream. |
The remaining gap is a single rubric that maps any task's property set, including complexity, ambiguity, error-recovery need, and latency budget, to the right depth across all four tiers at once.
The next module builds exactly that. It gives a concrete decision rubric that takes the full signal set from modules 1–5 and outputs a planning-depth recommendation — none, upfront, iterative, or reflection-gated — for any task you hand it.
Build a concrete cost-benefit rubric that maps task properties to planning depth — none, upfront, iterative, plus reflection tier — using the full set of signals from m1–m5. Leave with a decision checklist you can apply to any agent design.
A four-signal rubric that maps any task's complexity, ambiguity, time-sensitivity, and error-recovery cost to the correct planning depth and reflection tier.
Why this matters: Gives you a concrete decision framework to apply immediately when designing or auditing any agent — so you stop guessing and start calibrating deliberately.
Decision this forces: Given this task's complexity, ambiguity, time-sensitivity, and error-recovery needs, what planning depth and reflection tier is correct?
The symptom is repeated self-critique that never changes the output. The agent revises, re-evaluates, finds a new flaw, and revises again without converging. The fix is a : an iteration cap, a deterministic test, or a structured checklist. Exit the loop once a threshold is met, not when the model decides it's satisfied.
That cost-benefit lens from m3–m5 applies here — but now across all four planning depths simultaneously. You can assign the right tier to any task you encounter.
Every task emits four signals that determine the correct depth. Complexity: step count and dependency depth. Ambiguity: how underspecified the goal is at start. Time-sensitivity: latency budget vs. planning overhead. Error-recovery cost: how expensive a wrong branch is to undo.
These signals interact non-additively. A high-ambiguity, low-complexity task (e.g. a single clarification lookup) belongs at the reactive tier. has nothing to decompose.
The non-obvious trap: treat signals as independent checkboxes. A task scoring high on complexity but low on ambiguity and error-recovery cost (a well-specified batch transform) is a candidate for upfront planning. But NOT iterative : it adds overhead without buying adaptability.
Reflection tier is orthogonal to planning depth. You can layer onto any depth. But it only earns its extra model call when output quality is verifiable and the cost of a silent error exceeds the reflection overhead.
| Option | Complexity / dependency depth | Ambiguity at task start | Error-recovery cost | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| None (Reactive) | 1–2 steps, flat | Goal fully specified | Cheap — retry is fine | Single-step or well-scripted lookups where latency is the primary constraint and errors are cheap to retry. | Lowest | Minimal — zero extra model calls |
| Upfront Planning | 3–7 steps, moderate deps | Partially specified | Medium — wrong branch wastes work | Multi-step tasks with known structure and moderate ambiguity where a single decomposition pass stabilises the execution path. | Low–Medium | One extra model call at start; plan stored as a structured representation |
| Iterative (Replanning) | 7+ steps, deep deps | Highly underspecified | High — wrong branch is expensive | Long-horizon tasks where observations materially change the optimal next step and the latency budget absorbs multiple planning calls. | High | Multiple planning calls; plan representation updated each cycle |
| + Reflection Tier | Any — orthogonal to depth | Output must be verifiable | High — silent error is costly | Any depth where output quality is verifiable, silent errors are costly, and the reflection call is cheaper than downstream correction. | Additive to base depth | Adds one model call (or deterministic check) per output |
The agent issues a decomposition call for a single-step lookup. Then it executes the one sub-task it generated. Latency doubles; quality is identical. The tell: plan representation contains exactly one node. The completes in one cycle regardless.
The agent commits to the first plausible interpretation of an underspecified goal. It executes five steps before discovering the goal was wrong. The symptom is a high rate after step 3+. The system pays replan cost anyway, but reactively. It lacks the structured that would have caught the ambiguity earlier.
The most expensive miscalibration: a loop with no iteration cap. The task has no verifiable signal for the model's self-critique. The agent spins, consuming tokens and latency. It may return a WORSE answer than the first draft because later revisions drift from the original intent. Fix: tie the reflection exit to a deterministic test (schema check, unit test pass, diff below a threshold). Never tie it to the model's own confidence score.
Adding reflection to a reactive lookup wastes a model call on every request. Errors are cheap and immediately visible to the user. Conversely, skipping reflection on an iterative-planning task leaves silent errors. The task produces a long-form artifact (a report, a code file). The user sees a confident wrong answer with no audit trail.
Task: an agent must produce a competitive-intelligence brief — it searches five sources, extracts key claims, cross-checks contradictions, and synthesises a 500-word summary with citations. Apply the four-signal rubric before reading the assignment below.
The strongest signal here is error-recovery cost combined with a verifiable output — that combination is what makes reflection earn its call, regardless of planning depth.
Drag to see how planning overhead shifts from a net cost to a net gain as task complexity rises. Each stop shows the recommended depth and the dominant tradeoff at that point.
Before reviewing the rubric: reconstruct from memory the three task signals that make planning pay, the two that make it wasteful, and the stop-condition design that prevents reflection from spinning. What does the cost-benefit calculation look like for a task that is complex but also time-sensitive?
Apply what you learned to Agent Planning and Reflection.
Before choosing a design, you profile a bare reactive loop on a task. Which comparison best captures the cost structure you should measure?
The correct answer separates a one-shot call from a loop whose token and latency costs accumulate across repeated model calls, observations, and tool outputs. The single-turn-is-higher option reverses the usual cost shape; the constant-cost option ignores carried context and repeated calls; the no-wall-clock-cost option ignores model latency even without tools.
A customer-support agent must troubleshoot a vague billing issue across account history, payment-provider status, and policy exceptions. Failures often require trying a different branch rather than starting over. When would upfront planning earn its cost here?
The correct answer matches high ambiguity, branching recovery, and likely reactive thrashing, where upfront structure can reduce total retries and wasted calls. The guaranteed-first-tool option describes a simple task; the under-one-second option makes planning overhead hard to justify; the observations-never-change option removes the main value of graph-based replanning.
What is wrong with this design for a password-reset request that is fully specified and time-sensitive?
Plan: ask a planner model for 6 subtasks
Then run 5 specialist agents before sending the reset link
The correct answer identifies planning-only-adds-latency conditions: single-step, time-sensitive, and fully specified. Iterative planning would worsen the mismatch; adding more agents increases coordination cost rather than reducing it; mandatory model-call reflection is false because reflection must be justified by the error profile.
You need a self-correction loop for generated SQL. The database can run a syntax check and a small test query cheaply. Which reflection tier is the best first choice?
The correct answer uses cheap, reliable signals and gates the loop so it converges or stops. Unlimited model reflection risks spin and spends tokens on subjective judgment; no reflection ignores available deterministic checks; planning can reduce mistakes but cannot prove the generated SQL passes syntax or behavior checks.
Given this task, assign planning depth and reflection tier, and justify briefly: Generate a daily executive summary from 12 noisy data feeds, handle missing feeds gracefully, and deliver within 10 minutes. Incorrect numbers are costly, but deterministic validation can compare totals against source records.
The expected choice balances decomposition against reactive thrashing and uses the cheapest reliable error detector for numbers. Answers choosing no planning miss the complexity and recovery needs; answers choosing unlimited model reflection miss the availability of deterministic validation and the need for gates; answers choosing planning without any validation ignore the high cost of incorrect numbers.