An AI coding agent is a language model wired into a reasoning-action loop that lets it plan, write, execute, and iteratively fix code to complete a…
Trace the full read-reason-act-observe cycle of an AI coding agent using a concrete 'write a CSV parser' task as the running example. You see exactly how one loop iteration produces a tool call and how the observation feeds the next turn.
Traces the four-phase read-reason-act-observe loop that every AI coding agent runs on, using a CSV parser task as the running example.
Why this matters: Understanding the loop is the foundation for every other agent concept — planning, tool use, debugging, and stopping all happen inside it.
You ask an LLM to write a CSV parser in one shot. It generates something plausible — but it can't run the code, see the error, and fix it. That's the gap a coding agent fills.
An lets the model take an action, read what happened, and decide the next move — repeating until the task is done. A single-shot call has no mechanism to observe results or recover from mistakes.
The loop is what turns a language model into a system that can pursue a multi-step goal autonomously. Every AI coding agent you'll encounter is built on this pattern.
Each loop turn runs four phases in order. Reason: the model reads its goal and context, then decides what to do next.
Act: it emits a — for example, "run this code" or "read this file."
Observe: the tool's output is returned as an and appended to the model's .
Update: the model re-reads the enriched context. It either loops again or triggers a .
The agent receives: "Write a Python function that parses a CSV file and returns a list of dicts." Here's one full loop iteration.
run_code("def parse_csv(path): ...")KeyError: 'name' — the header row wasn't stripped.The is raw tool output — not a summary. The model reads the actual error string, enabling self-correction without human help.
# Single-shot: no loop, no observation response = llm.complete( "Write a Python function that parses a CSV file " "and returns a list of dicts." ) print(response.text) # → Outputs code, but never runs it. # → If the CSV has a BOM or quoted commas, the # generated code silently returns wrong data.
A plain LLM call produces code but has no way to execute it, observe the result, or fix mistakes. The model can't know whether its output is correct — it has no feedback channel.
The script prints generated code that likely uses csv.DictReader without encoding='utf-8-sig'. When run, the first key in every dict will be '\ufeffname' instead of 'name' — a silent data corruption. The single-shot call has no way to detect or fix this because it never executes the code.
goal = "Write a Python CSV parser returning a list of dicts." context = [goal] for turn in range(MAX_TURNS): action = llm.reason(context) # Reason phase if action.is_stop(): # stop_condition met break result = run_tool(action) # Act + Observe context.append(result.output) # Update: append observation print(context[-1]) # final output
The loop appends each tool result to context so the model's next reason call sees the full history — including any error messages. This is the minimal skeleton every AI coding agent runs on.
context holds [original_goal, 'KeyError: name\n File ...']. The model reads both the goal AND the error, so it can reason about the fix rather than regenerating from scratch.
goal = "Parse CSV with quoted commas; return list of dicts." context = [goal] for turn in range(MAX_TURNS): action = llm.reason(context) if action.is_stop(): break result = run_tool(action) # TODO: append result.output to context # so the next turn sees the observation # Hint 1: context is a plain list. # Hint 2: the observation is result.output (a string).
Stop — attempt the TODO before revealing the answer. The missing line is the crux of the loop: without it, the model reasons blind on every turn.
Replace TODO with: context.append(result.output)
Changed line vs Stage 2: same line, new scenario (quoted commas). If omitted, context never grows — the model re-reasons from only the original goal every turn, ignoring every error and producing the same broken code in an infinite loop until MAX_TURNS is hit.
Three failure patterns account for most agent loop bugs in coding tasks.
Each grows the . After many turns, older outputs are silently dropped.
The model then re-generates code it already fixed. You see the same error reappear on turn 8 that was resolved on turn 3.
If the check is too strict (e.g., requires exact "DONE" match), the loop hits MAX_TURNS and exits with no output.
Observable symptom: the agent returns an empty result after consuming the full turn budget.
If the code that appends the is missing or conditional, the model reasons from stale context every turn.
It produces the same repeatedly — no error is raised, so this is easy to miss.
context.append(result.output) must be unconditional, (2) a numeric MAX_TURNS guard must exist, and (3) the stop condition must match the model's actual output format — not a hardcoded string the model never emits.The loop is reactive: it fixes problems as they appear, one turn at a time. For a task like "write a CSV parser with error handling, tests, and a CLI," jumping straight into the loop means the agent may tackle sub-tasks in the wrong order or miss one entirely.
The next module — Planning: Decomposing the Goal — shows how an AI coding agent breaks that multi-part goal into an ordered sub-task list before the loop starts.
Upfront decomposition is the main lever for reducing on complex tasks.
See how an AI coding agent breaks the CSV-parser goal into an ordered sub-task list before writing any code, and learn why explicit decomposition reduces hallucinated steps and wasted execution cycles.
How an AI coding agent decomposes a goal into an ordered sub-task list before writing any code.
Why this matters: Explicit planning is what separates a reliable coding agent from a one-shot prompt — it directly controls how well your agent handles multi-step SEO/GEO page builds without hallucinating steps.
Decision this forces: When should the agent re-plan mid-task versus continue with the original plan?
Answer: the agent feeds the observation back into its reasoning step. It re-reads the result and updates its understanding. Then it decides what to do next. That loop works well for a single, well-scoped step.
But what happens when the goal is too large? The agent can't hold the whole path in one reasoning step. That's the problem this module solves.
is the agent's first action. Before touching any file or tool, it converts the user's goal into an ordered list of sub-tasks via .
Without a plan, the agent reasons step-by-step inside a single prompt. It frequently missing steps or repeats work already done. It has no shared record of what's complete.
A written plan fits inside the as a compact reference. Every later loop iteration can check progress without re-reading all prior output.
The payoff is fewer wasted execution cycles. The agent doesn't start writing code for step 4 before step 2 is verified.
Imagine you hand the agent this goal: "Write a robust CSV parser that handles quoted fields, empty rows, and UTF-8 encoding."
The agent gets the whole goal at once and starts generating code immediately. It often skips the UTF-8 edge case entirely, or writes a test before the parser exists. When it fails, there's no plan to diff against — you can't tell which step went wrong.
The agent first emits a numbered sub-task list:
Steps 1–4 are strictly sequential (each builds on the last). Step 5 could draft tests in parallel with step 4 if the agent has two workers. Step 6 is always last — it depends on both code and tests existing.
goal = "Write a robust CSV parser: quoted fields, empty rows, UTF-8." planning_prompt = f""" You are a planning agent. Break the goal below into a numbered list of sub-tasks. Each sub-task must be concrete and independently verifiable. Mark any sub-tasks that can run in parallel with [P]. Goal: {goal} """ plan_text = llm(planning_prompt) # returns a string print(plan_text)
This prompt asks the model to produce a numbered, verifiable sub-task list — not code yet. The [P] marker makes parallelism explicit so the executor can schedule accordingly.
4180.
[P]
...
The model returns a plain-text numbered list. It does NOT return code yet — that's the point.
import re def parse_plan(plan_text: str) -> list[dict]: tasks = [] for line in plan_text.strip().splitlines(): match = re.match(r"(\d+)\. (\[P\] )?(.*)", line) if match: tasks.append({ "id": int(match.group(1)), "parallel": bool(match.group(2)), "task": match.group(3).strip(), }) return tasks subtasks = parse_plan(plan_text)
This delta converts the raw plan string into a structured list the agent loop can iterate over. Each dict carries an id for ordering, a parallel flag for scheduling, and the task description the next module will hand to the code-generation step.
{'id': 4, 'parallel': True, 'task': 'Add UTF-8 decode with error=\'replace\' fallback.'}
Key insight: parallel=True comes from the [P] marker — the regex captures group(2) only when [P] is present, so sequential tasks get parallel=False automatically.
def execute_plan(subtasks, llm, max_retries=1): results = {} for task in subtasks: outcome = run_task(task["task"], llm) # returns {"ok": bool, "output": str} if not outcome["ok"]: # TODO: call llm() here to produce a revised sub-task list # for the REMAINING tasks only (task["id"] and later). # Hint: pass the failed task description + outcome["output"] # into a new planning prompt, then re-parse and reassign subtasks. pass results[task["id"]] = outcome["output"] return results
Stop — attempt the TODO before revealing. The missing piece is the re-plan call: when a sub-task fails, the agent should revise only the remaining steps, not restart from scratch.
replan_prompt = (
f"Task '{task['task']}' failed with: {outcome['output']}\n"
f"Revise the remaining sub-tasks and return a new numbered list."
)
new_plan_text = llm(replan_prompt)
subtasks = parse_plan(new_plan_text) # replaces remaining tasks
break # restart the loop with the new plan
--- What changed and why ---
The re-plan prompt scopes to the failure context (not the whole goal), so the model produces a targeted fix rather than rewriting completed steps. The break restarts the for-loop over the revised subtasks list.
Three failure patterns show up repeatedly. Two of them are silent.
FileNotFoundError with no obvious cause.Before trusting an AI-generated plan, check three things. Every step must have a single, testable output. Parallel steps must share no output files. The re-plan prompt must explicitly exclude already-completed step IDs.
Walk through how the agent converts one CSV-parser sub-task into a concrete Python function, including how tool context, file contents, and prior observations are packed into the generation prompt to constrain the output.
Shows how an AI coding agent constructs a generation prompt — packing file context, prior observations, and an output contract around a single sub-task to produce a correct, compatible Python function.
Why this matters: Every code-generation step in your agent pipeline depends on a well-formed prompt; understanding what goes in (and what to cut when context is tight) directly determines whether the generated code fits your project or breaks it.
Decision this forces: How much existing code context should the agent include in the generation prompt — and what gets cut when the context window is tight?
In module 2, turned "write a CSV parser" into an ordered list of concrete sub-tasks. Each was small enough to attempt in one generation step.
That list is the handoff. The agent picks the first incomplete sub-task — say, "parse a header row into a list of column names" — and turns it into a real Python function.
This module answers: what does the agent pack into the beyond that one-line task, and why does each piece matter?
A generation prompt bundles four things beyond the sub-task: available to the agent, relevant file contents from disk, prior from earlier iterations, and explicit output constraints.
Each piece constrains the model differently. File contents anchor the function to real column names and delimiters. Prior observations rule out failed approaches. Output constraints prevent inventing signatures nothing else calls.
Together they shrink the model's solution space from "any Python" to "the function this codebase needs."
The agent is on sub-task 1: "write parse_header(path) → list[str]". Here is how it assembles the prompt.
With all four pieces, the model generates a function using the real delimiter, real encoding, and the exact signature the codebase expects.
# Agent sends ONLY the sub-task description — no file context, no contract prompt = """ Write a Python function to parse a CSV header row. """ generated_code = model.generate(prompt) print(generated_code)
This is the obvious-but-wrong approach: hand the model only the task string and let it fill in the blanks.
Predict what the model returns before revealing — specifically, what will the function be named and what will it import?
The model might generate 'def get_headers(filepath)' using 'import pandas as pd' — a name and dependency the rest of the codebase never agreed on. Without an output contract, the model invents a plausible-but-incompatible signature. Without file context, it may import pandas even if the project uses only the stdlib csv module. Both mismatches break the next sub-task silently.
file_snippet = "id,name,price\n1,Widget,9.99" # from read_file observation prior_obs = "delimiter=comma, encoding=UTF-8" # from earlier loop turn prompt = f""" File context (first 2 lines of products.csv): {file_snippet} Prior observations: {prior_obs} Output contract: - function name : parse_header - signature : parse_header(path: str) -> list[str] - no third-party imports Sub-task: Parse the header row and return column names as a list. """
Each section of the prompt does one job: file context anchors real column names, prior observations rule out wrong delimiters, and the output contract locks the signature.
The model now has no room to invent a pandas import or a mismatched name — the constraints close those doors before generation starts.
def parse_header(path: str) -> list[str]:
with open(path, encoding='utf-8') as f:
The def line matches the contract exactly. The open() call uses UTF-8 because the prior observation stated the encoding — the model didn't guess.
Two failures dominate in practice. Both trace back to missing prompt sections, not model quality.
Cause: no file context and no "no third-party imports" constraint. The model picks the library it sees most often in training — pandas, numpy — regardless of what the project uses.
What you see: the generated file runs fine in isolation but raises ModuleNotFoundError: No module named 'pandas' in the project's environment.
Cause: no output contract in the prompt. The model generates get_headers(filepath) instead of the required parse_header(path).
What you see: the next sub-task calls parse_header(path) and raises NameError: name 'parse_header' is not defined — a mismatch that surfaces only at execution time.
# New sub-task: "write parse_rows(path) -> list[dict] that reads data rows" # The agent already ran read_file and got this observation: # first data row -> '1,Widget,9.99' (columns: id, name, price) file_snippet = "id,name,price\n1,Widget,9.99" prior_obs = "delimiter=comma, encoding=UTF-8, header parsed by parse_header" prompt = f""" File context: {file_snippet} Prior observations: {prior_obs} Output contract: # TODO: fill in the three contract lines for parse_rows Sub-task: Read all data rows and return them as a list of dicts. """
The file context and prior observations are already filled in — your job is to write the three output-contract lines the agent needs to constrain the model.
Stop and attempt the TODO before revealing. Hint 1: the function name must pair with parse_header. Hint 2: the return type must match the sub-task description exactly.
Changed lines vs. Stage 2: (1) name is parse_rows, not parse_header — it's the next sub-task in the plan. (2) return type is list[dict] because each row maps column names to values. (3) The import constraint stays the same — it's a project-wide rule, not sub-task-specific. Without all three lines, the model may invent a pandas-based DictReader wrapper with a mismatched name.
See how the agent submits the generated CSV-parser function to a sandboxed executor, captures stdout/stderr and exit codes, and converts that raw output into a structured observation that re-enters the reasoning step.
Shows how an AI coding agent submits generated code to a sandboxed executor, captures stdout, stderr, and exit codes, and converts that raw output into a structured observation that drives the next reasoning step.
Why this matters: Understanding code execution and sandboxing lets you configure safe, bounded agent runs and interpret the observations that control whether your agent fixes, retries, or advances.
Decision this forces: What execution timeout and resource limits should bound an agent's code runs to prevent runaway loops?
Module 3 packed tool context, file contents, and prior observations into the prompt. The agent produced a CSV-parser function. But generating code is not the same as knowing it works.
An agent that only generates code and never runs it cannot verify correctness autonomously. It can only guess, based on training, whether the output is right.
This module follows the generated function into the executor. It shows what comes back. observation.
sandbox is an isolated execution environment. It runs agent-generated code with strict boundaries: no filesystem access, no outbound network calls, and hard caps on CPU time and memory.
Running untrusted, model-generated code outside a sandbox exposes the host to file deletion, credential theft, or infinite loops.
exit code (0 = success, non-zero = failure). Together these form the raw execution result.
The agent never sees the host environment. It only sees what the sandbox returns, which is exactly the isolation guarantee you need for autonomous code runs.
The agent has generated parse_csv(path)tool call.
If the exit code is 0 and stdout contains the expected rows, the agent marks the sub-task done and moves to the next one. If exit code is non-zero, stderr tells it where to look.
import subprocess, json def run_in_sandbox(code: str, timeout: int = 10) -> dict: result = subprocess.run( ["python", "-c", code], capture_output=True, text=True, timeout=timeout ) return { "stdout": result.stdout, "stderr": result.stderr, "exit_code": result.returncode, }
This pattern shows the minimal shape of a sandboxed executor: run the code in a subprocess, collect all three output channels, and return them as a structured dict.
In production you'd replace subprocess.run with a container or managed sandbox API, but the three-field result shape stays the same.
stdout: '' (empty — nothing was printed before the crash). stderr: the full Python traceback ending in 'ZeroDivisionError: division by zero'. exit_code: 1 (non-zero, signalling failure). The agent reads exit_code != 0 and routes to a fix step.
def execution_result_to_observation(raw: dict) -> str: if raw["exit_code"] == 0: return f"SUCCESS\nOutput:\n{raw['stdout'].strip()}" else: return ( f"FAILURE (exit {raw['exit_code']})\n" f"Stderr:\n{raw['stderr'].strip()}" ) # Agent appends this string to the context window observation = execution_result_to_observation(run_in_sandbox(generated_code))
The agent doesn't reason over raw subprocess output — it reads a structured observation string that labels the outcome and surfaces the right channel (stdout on success, stderr on failure).
context window so the next reasoning step sees exactly what happened and can decide: mark done, retry, or escalate.
'SUCCESS\nOutput:\n3 rows parsed' — the agent reads this, sees SUCCESS, and advances to the next sub-task. No stderr is included because exit_code == 0.
def run_and_observe(code: str, timeout: int = 10) -> str: result = subprocess.run( ["python", "-c", code], capture_output=True, text=True, timeout=timeout ) if result.returncode == 0: return f"SUCCESS\nOutput:\n{result.stdout.strip()}" # TODO: return a FAILURE observation that includes # the exit code AND the stderr text. # Hint 1: mirror the SUCCESS branch structure. # Hint 2: use result.returncode and result.stderr.
Stop — attempt the TODO before revealing. The missing branch is the crux: without it, every failed run returns None and the agent has no observation to reason from.
CHANGED LINES — the TODO becomes:
return (
f"FAILURE (exit {result.returncode})\n"
f"Stderr:\n{result.stderr.strip()}"
)
Why: the agent needs the exit code to classify the failure and the stderr text to localize the bug. Returning only one or the other leaves the next reasoning step with half the picture. This mirrors Stage 2 exactly but is now wired directly into the subprocess call — no separate helper needed.
Three failure patterns show up repeatedly when agents run generated code:
Before trusting AI-generated executor code, check: does it enforce a timeout? Does it capture stderr separately from stdout? Does it run in an isolated process or container, not inline? Does the test harness assert output shape, not just exit code?
Trace the agent's reflection step as it reads a TypeError from the CSV-parser run, localizes the root cause in the generated code, rewrites the failing function, and re-executes — revisiting the loop mechanics from Module 1 under real failure pressure.
How an AI coding agent reads a runtime error, diagnoses the root cause through a reflection sub-step, rewrites the failing code, and re-executes — with guardrails against infinite loops and context overflow.
Why this matters: Every agent you build will hit broken code; knowing how to wire reflection and a retry ceiling correctly is what separates a loop that converges from one that burns your token budget and stalls.
Decision this forces: How many retry attempts should the agent make before escalating or halting — and what signals should trigger that threshold?
branch.
re-enters the reasoning step — but what the agent does with it next is the subject of this module. The question is: does the agent just re-prompt, or does it do something smarter?
sub-step is a deliberate reasoning pass. The agent reads the error, localizes the fault to a specific line or function, and produces a targeted rewrite plan before touching code.
Plain re-prompting appends the error text and asks the model to "try again." Reflection adds a structured intermediate: the agent explicitly names the root cause, affected symbol, and intended fix.
This matters because a model that skips reflection tends to make surface edits — changing a variable name instead of fixing the type mismatch — and loops without converging.
and the next action: observe → reflect → rewrite → re-execute.
step and gets back: exit code 1, stderr = TypeError: '<' not supported between instances of 'str' and 'int' in sort_rows(), line 14.
A bare re-prompt would append that message and say "fix the code." The reflection step instead produces a structured diagnosis:
Only after producing that diagnosis does the agent emit a rewrite action targeting exactly those lines. The next execution returns exit code 0 — the loop advances.
def run_with_retry(agent, code, max_attempts=5): for attempt in range(max_attempts): result = execute_in_sandbox(code) if result.exit_code == 0: return result # ❌ No reflection — just re-prompt with raw error code = agent.generate_code( prompt=f"Fix this error: {result.stderr}" ) raise RuntimeError("Max retries exceeded")
This loop retries up to five times but skips reflection entirely — it hands the raw stderr back to the model and hopes for a better guess.
The model sees the same error context each time and tends to make cosmetic edits, so the loop often exhausts all attempts without converging.
The agent re-generates code five times, each time making a small surface change (e.g. renaming the variable or reordering lines) without fixing the cast. On attempt 5 it raises RuntimeError: 'Max retries exceeded' — the bug is still present and the task is abandoned with no useful diagnostic logged.
def reflect_on_error(agent, code, error_text): diagnosis = agent.reason( prompt=( f"Error: {error_text}\n" f"Code:\n{code}\n" "Identify: root cause, affected function, fix plan." ) ) return diagnosis # e.g. {cause, symbol, fix_plan} def run_with_reflection(agent, code, max_attempts=3): for attempt in range(max_attempts): result = execute_in_sandbox(code) if result.exit_code == 0: return result diagnosis = reflect_on_error(agent, code, result.stderr) code = agent.rewrite(code, diagnosis)
The key delta from Stage 1: reflect_on_error() runs between the failed execution and the rewrite, producing a structured diagnosis the rewrite call can act on precisely.
Notice max_attempts drops from 5 to 3 — targeted rewrites converge faster, so a lower ceiling is safer against runaway loops.
agent.rewrite() receives the original code plus the structured diagnosis dict. Unlike Stage 1's raw stderr string, the diagnosis names the exact function and line, so the model targets only sort_rows() line 14 rather than guessing across the whole file. The fix is surgical, not speculative.
def run_agent_debug_loop(agent, code, error_budget): history = [] for attempt in range(error_budget): result = execute_in_sandbox(code) if result.exit_code == 0: return {"status": "success", "attempts": attempt + 1} diagnosis = reflect_on_error(agent, code, result.stderr) history.append(diagnosis) if _context_near_limit(history): # TODO: what should happen here? ??? code = agent.rewrite(code, diagnosis) return {"status": "escalate", "last_error": result.stderr}
is approaching its limit mid-loop.
Stop — attempt this before revealing. Hint 1: letting the loop continue risks truncating earlier diagnoses, which causes error misattribution. Hint 2: the return at the bottom shows what a normal budget-exhaustion looks like — the context-limit case needs a different status.
Changed lines:
if _context_near_limit(history):
return {"status": "escalate", "reason": "context_overflow", "last_error": result.stderr}
Why: once the history of diagnoses fills the context window, any further reflection call will silently drop the oldest entries. The model then re-diagnoses errors it already fixed, causing misattribution loops. Returning early with a distinct 'context_overflow' reason lets the orchestrator decide whether to summarize history and restart or hand off to a human — rather than burning the remaining error_budget on degraded reasoning.
These are the three failure modes you'll hit in practice. Each has a concrete symptom so you can spot them in a trace.
The model blames the wrong function. Symptom: the rewrite changes parse_header() but the TypeError is in sort_rows(). Cause: the reflection prompt included too much surrounding code, so the model anchored on the first suspicious-looking line.
The agent cycles through the same fix repeatedly without a hard attempt ceiling. Symptom: the same exit-code-1 observation appears five or more times with nearly identical rewrites. Fix: enforce max_attempts and return a structured escalation — never let the loop run unbounded.
. Symptom: the agent re-introduces bugs it already fixed — it has lost earlier diagnosis entries to truncation. Guard with a token-count check before each reflection call, and escalate or summarize when the limit is near.
Examine how the agent evaluates its own stop condition for the CSV-parser task (all tests pass, no open sub-tasks), where premature termination and infinite loops occur, and how to verify AI-generated code output before shipping it.
Teaches the three strategies an AI coding agent uses to decide it is finished, the two ways that decision fails, and how to verify the agent's output before shipping it.
Why this matters: Without a reliable stop condition, your agent either quits too early and ships broken code or loops forever — both waste time and erode trust in the tool.
Decision this forces: Which stop-condition strategy best fits the task at hand — and how do you verify the agent's output is actually correct before trusting it?
Reflection produces a root-cause diagnosis and a rewrite plan — the agent localises which line is wrong, then emits a corrected function before re-executing. That loop keeps spinning as long as there is an error to fix.
But what tells the loop to stop? That is the question this module answers.
A is the rule the agent evaluates to decide whether to exit the loop or take another action. Without one, the loop either quits too early or never quits.
Three strategies cover most tasks:
For the CSV-parser task, the agent uses all three in sequence: tests pass → checklist clear → reflection confirms no open sub-tasks.
| Option | Objectivity | Works without tests | Hallucination risk | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Test-pass gate | Fully objective — exit code is binary | Fails if no tests exist | None — machine-checked | Task has a comprehensive, automated test suite and a clear exit code. | Cost of running the full suite each iteration | Low — just check exit code 0 |
| Sub-task checklist | Semi-objective — depends on checklist quality | Yes — checklist is independent of tests | Low if checklist is pre-defined | Task mixes code and non-code steps, or test coverage is incomplete. | Minimal extra inference cost | Medium — must maintain checklist state |
| Reflection verdict | Subjective — model self-reports | Yes — fully model-driven | High — model may declare done prematurely | Open-ended or creative tasks where success criteria are hard to enumerate upfront. | Extra model call per iteration | High — requires a structured DONE signal and prompt discipline |
Two failure modes dominate in practice, and they pull in opposite directions.
The agent stops before the task is actually done. The most common trigger: a partial test run returns exit code 0 because the failing tests were never discovered or were silently skipped. The symptom is a "task complete" signal while several sub-tasks remain unchecked in the plan.
Mitigation: always run the full suite (not a subset), and cross-check the test count against the expected total before accepting exit code 0.
The agent never stops — it rewrites the same function repeatedly, each attempt introducing a new bug that the next iteration tries to fix. The observable symptom: iteration count climbs past a sane ceiling (say, 10) with no change in the failing test names.
Mitigation: set a hard iteration cap (e.g. max_iterations=10) and a cost/token budget. When the cap triggers, surface the last error to a human rather than silently failing.
def run_agent_loop(agent, max_iterations=20): for i in range(max_iterations): action = agent.step() # reason + pick action if action.type == "run_tests": result = run_tests(subset=["test_basic"]) # BUG: partial suite if result.exit_code == 0: return "DONE" # stops too early agent.observe(result) return "MAX_ITERATIONS"
This loop checks exit code 0 — but only against a single test file. Passing one file while the full suite still has failures is the classic premature-termination trap.
run_tests(subset=['test_basic']) returns exit_code=0 even though test_edge_cases.py is failing. The loop returns 'DONE' immediately. The caller believes the task is complete — but the CSV parser is broken for edge-case inputs. The bug is the subset argument, not the exit-code check itself.
def run_agent_loop(agent, plan, max_iterations=10): for i in range(max_iterations): action = agent.step() if action.type == "run_tests": result = run_tests(suite="all") # gate 1: full suite agent.observe(result) if result.exit_code != 0: continue all_done = all(t["done"] for t in plan) # gate 2: checklist verdict = agent.reflect() # gate 3: reflection if all_done and verdict == "DONE": return "DONE" raise RuntimeError(f"Stopped after {max_iterations} iterations — review last error")
Three gates in sequence: the full test suite must pass, every plan item must be marked done, and the model's reflection must emit DONE. The hard cap raises an exception instead of silently returning — forcing human review.
The condition 'all_done and verdict == "DONE"' is False, so the loop does not return. It calls agent.step() again on the next iteration. The agent gets another chance to resolve whatever concern it flagged in the reflection — or it eventually hits the max_iterations cap and raises RuntimeError.
# CSV-parser agent — variation: checklist-only stop (no reflection call) # The test suite is incomplete, so we rely on the sub-task checklist. def run_checklist_loop(agent, plan, max_iterations=10): for i in range(max_iterations): action = agent.step() result = execute_action(action) # runs code or tests agent.observe(result) # TODO: add the stop condition here # Hint 1: check that every item in `plan` has item["done"] == True # Hint 2: also guard against stopping if the last action raised an error raise RuntimeError("Cap reached — inspect agent state")
Stop — attempt the TODO before revealing the answer. The missing lines are the crux of this module: they decide when the loop exits.
Replace the TODO with:
if all(item['done'] for item in plan) and result.error is None:
return 'DONE'
Changed lines vs Stage 2: no run_tests() call and no agent.reflect() — this variation trusts the checklist alone.
Why the error guard: without it, the loop could exit the moment the last action crashes AND all prior items happen to be marked done — returning 'DONE' on a broken state. The guard ensures the final action actually succeeded before stopping.
The agent says it's done — here is what to check before you trust the output.
You now have the full agent loop — from the first reasoning step in Module 1 to a verified, shipped output here. The capstone challenge asks you to build and evaluate a complete agent run end-to-end, applying every layer you've traced.
Before reading the summary: reconstruct from memory the six stages of the agent loop for the CSV-parser task — what happens at each stage, what gets passed to the next, and which stage is most likely to fail. Write it out, then check it against the spine.
Apply what you learned to SEO/GEO page — the page title is the H1 and the exact search query. OPEN with a 40-60 word self-contained, quotable answer capsule that names the entity explicitly in sentence one (no "in this lesson…" preamble) — this is the sentence an LLM lifts. Name the entity by its full name every time (not pronoun-only). Structure each section as ONE sub-question (How it works · X vs Y · When to use it · Example · Common mistakes), each opening with its own 1-2 sentence direct answer then depth. End with a genuine 3-5 question FAQ phrased as real user questions ending in "?" (these become FAQPage schema + the knowledge check). Prefer quotable specifics — concrete numbers, defaults, version names, one short snippet — over vague prose. Self-contained for a reader who arrived cold from a search engine or an LLM. Intent = explainer: mechanism + example + when-to-use. Target query: "ai coding agents"..
An agent trace shows these four lines in order:
Which phase of the agent loop is executing at line 3?
Line 3 is the agent reading the result returned by the tool it just called — that is the Observe phase, where raw execution output (stdout, stderr, exit code) is captured. Line 1 is Reason (deciding what to do), line 2 is Act (calling the tool), and line 4 is Update (writing the result back into the agent's working memory).
A developer gives an AI coding agent this single prompt: "Build a REST API with auth, a database layer, three endpoints, and full test coverage." The agent returns broken, incomplete code. Which planning failure best explains this outcome?
A flat single-prompt approach asks the model to reason about, generate, and coordinate all sub-tasks simultaneously without a planning step — this routinely produces incomplete or incoherent output for multi-step tasks. A test-pass gate is a stop-condition strategy, not a planning concept. Context windows being too large is not a recognized failure mode here. Re-planning mid-task is a decision point that only arises after a plan already exists.
An agent's generation prompt for a sub-task includes only the sub-task description and nothing else. Which two failure modes does this most directly cause?
When the generation prompt omits existing code context — the signatures already defined, the imports already present, the conventions in use — the model invents plausible-sounding but incorrect imports and writes function signatures that do not match the rest of the codebase. Premature stop and runaway loop are stop-condition failure modes. Error misattribution and context overflow are debugging failure modes. Sandboxing errors relate to execution, not generation.
When would you choose a test-pass gate stop condition over a sub-task checklist?
A test-pass gate is the strongest stop condition because it uses an objective, executable signal — all tests green — rather than the agent's self-assessment. It only applies when a runnable test suite exists. Without tests, a reflection verdict or sub-task checklist is the fallback. Exceeding retry attempts is an escalation trigger, not a stop-condition strategy. Parallelizability is a planning concept, not a stop-condition selector.
An agent has attempted to fix the same TypeError three times and failed each time. Describe what the reflection sub-step should do at this point that simply re-prompting the model does NOT do, and name the signal that should trigger escalation or halt instead of a fourth retry.
Reflection is a deliberate reasoning step: the agent reads the error, diagnoses the likely cause, and updates its understanding before acting again. Plain re-prompting skips this diagnosis and often repeats the same mistake. A retry-count threshold — typically set before the run — is the primary escalation signal; a secondary signal is when successive errors are identical, confirming no progress is being made.