An autonomous AI agent is a system that perceives its environment, decides what to do, acts, and loops — pursuing a goal without step-by-step human…
You trace exactly what an autonomous AI agent 'sees': text prompts, tool outputs, file contents, and API responses — using a web-research agent as the running example. You leave knowing what counts as an agent's input and why raw data must be structured before the agent can use it.
This module explains what an autonomous AI agent perceives — text prompts, tool outputs, file contents, and API responses — and why raw data must be formatted before the agent can use it.
Why this matters: Understanding perception is the foundation for every agent you build: if the input is wrong, every downstream decision is wrong.
is a program using an to take actions toward a .
To act, it must first perceive: read everything relevant from its .
This module covers perception. The driving question: exactly what does an agent 'see', and why does format matter?
Here is what the web-research agent perceives in one cycle.
# The agent fetches a news page. # This is what arrives BEFORE any formatting. raw_html = fetch_page("https://news.example.com/ai") print(raw_html[:300]) # Output: # <html><head><title>AI News</title></head><body> # <div class="ad-banner">Buy now!</div> # <article><h1>OpenAI releases new model</h1> # <p>Today, OpenAI announced...</p></article>
Raw HTML is the agent's first contact with a web page. Without stripping the tags, the agent cannot tell ad copy from article text.
The LLM sees tag names, ad text, and article text all mixed together. It may summarise 'Buy now!' as news, miss the real headline, or hallucinate structure that isn't there. The output is unreliable — garbage in, garbage out.
# Strip HTML tags → clean text the agent can actually use. def parse_html(raw_html): text = remove_tags(raw_html) # removes <...> markup text = collapse_whitespace(text) return text.strip() clean_text = parse_html(raw_html) # Build the observation the agent will read. observation = { "source": "https://news.example.com/ai", "content": clean_text } print(observation["content"][:80]) # Output: 'OpenAI releases new model\nToday, OpenAI announced...'
Formatting turns raw environment data into a structured the agent can reason over. The source label lets the agent cite where the fact came from.
The context window now holds a clean 'source' label plus readable article text — no tags, no ads. The agent can quote the headline accurately and attribute it to the right URL.
# A news API returns JSON with a 'articles' list. # Each article has 'title' and 'description' fields. import json api_response = '{"articles": [{"title": "AI Act signed", "description": "The EU signed..."}]}' def format_api_observation(raw_json_string): data = json.loads(raw_json_string) # parse JSON text → Python dict articles = data["articles"] # TODO: return a plain-text string listing each article's # title and description, one per line. ???
This is the crux of perception: converting a raw API response into text the LLM can read. The TODO is the exact transformation this module is about — not boilerplate.
lines = [f"{a['title']}: {a['description']}" for a in articles]
return "\n".join(lines)
# Changed lines: the list comprehension builds one string per article;
# join() stitches them into a single readable block the agent can read.
# This is the core perception move: raw JSON → structured plain text.
Three ways the perception step fails — and what you'd actually see.
Perception is step one of the : the agent reads its environment, then decides what to do next.
Once the observation is clean and in the context window, the agent can reason. That reasoning step is where begins.
You watch the same web-research agent reason through a goal — breaking it into sub-tasks using the ReAct pattern (Reason → Act → Observe) — and see a short annotated code snippet showing the prompt structure that drives this loop. You leave able to read an agent's reasoning trace and spot where planning happens.
This module teaches the ReAct pattern — the Reason → Act → Observe loop that lets an agent plan and execute multi-step tasks.
Why this matters: Understanding ReAct lets you read your agent's reasoning trace, spot planning failures, and write the prompt structure that drives the loop — essential for building a reliable SEO research agent.
Module 1 showed that an reads its world through a — its working memory.
That window holds text prompts, tool outputs, file contents, and API responses.
Now the question is: once the agent has all that input, how does it decide what to do next?
Without a Thought step, the agent fires tools at random. With planning, one plan covers two tool calls — half the cost.
You can now read a reasoning trace and name each step. Next, module 3 shows how the agent executes actions — calling a search tool, parsing results, and writing files — with a Python snippet.
stands for Reason → Act → Observe. The agent repeats this loop until its goal is met.
happens in the Reason step. The agent breaks the goal into sub-tasks first.
This loop repeats — each cycle is one turn — until a is met.
Goal: "Find the top-ranking page for 'autonomous AI agents' and summarise why it ranks."
Here is the agent's full reasoning trace:
Notice: the agent planned sub-tasks in step 1 before calling any tool. That single Thought saved a wasted search call.
SYSTEM_PROMPT = """
You are a research agent. For every step, write:
Thought: <what you plan to do and why>
Action: <tool_name>(<args>)
Observation: <you will receive this from the tool>
Repeat until you can write:
Final Answer: <your answer>
"""This system prompt is the engine of the ReAct loop. exactly what format to follow on every turn.
Before reading on — predict: what does the LLM write in the 'Observation' line?
The LLM does NOT write the Observation. Your code runs the tool and injects the tool's real output into that slot. The LLM only writes Thought and Action.
def react_turn(goal, history, tools): prompt = SYSTEM_PROMPT + history + f"\nGoal: {goal}" response = llm(prompt) # LLM writes Thought + Action action, args = parse_action(response) result = tools[action](**args) # run the real tool observation = f"Observation: {result}" history += response + "\n" + observation return history, action == "finish"
Each call to react_turn is one full Reason → Act → Observe cycle.
The function returns the updated history and a flag: True when the agent calls finish.
react_turn returns (history, True). The caller sees True and stops the loop — the agent is done. If it returned False, the caller would call react_turn again with the updated history.
tools = {"search": search_web, "fetch_page": fetch_page, "finish": finish}
history = ""
goal = "Find the top page for 'autonomous AI agents' and summarise why it ranks."
for _ in range(10): # safety cap — max 10 turns
history, done = react_turn(goal, history, tools)
if done:
# TODO: extract and print the Final Answer from history
breakThis is the outer loop that drives the agent. One line is missing — the key step that surfaces the agent's answer.
Stop and attempt the TODO before revealing. Hint 1: the Final Answer is the last line of history that starts with 'Final Answer:'. Hint 2: use str.split or a simple search.
Replace the TODO with:
answer = [l for l in history.splitlines() if l.startswith('Final Answer:')][-1]
print(answer)
Changed: added two lines inside the 'if done' block.
Why: 'history' is a running string of all turns. We scan it for the line the LLM wrote as its final output — that's the crux of this module: the agent's answer lives inside the accumulated ReAct trace, not in a separate variable.
Three failure patterns to watch for — and what they look like:
Action: search(query="autonomous AI agents") with no Thought line above it. Fix: add an explicit instruction in the system prompt: 'Always write Thought before Action.'finish. You see: the loop hits your safety cap (e.g. 10 turns) and stops with no answer. Fix: add a 'you MUST call finish within N steps' instruction and a hard turn cap.Observation: The page has 800 words. — but no tool was actually called. Fix: parse the Action line and inject the real tool result before the LLM continues.finish is in the tools dict and the loop actually stops when it's called.You see the web-research agent call a search tool, parse the result, and write a file — with a worked Python snippet showing tool registration and invocation. You also revisit perception (from Module 1) to see how the tool's output becomes the next observation, closing the first half of the loop.
Shows how an autonomous AI agent executes actions — registering tools, invoking them, and feeding results back into its memory.
Why this matters: Directly powers your SEO/GEO page agent: understanding tool calls lets you control what your agent can do and debug it when it breaks.
Answer: the agent picks an . It calls a to do something in the world.
Module 2 showed how the agent decides what to do using the . This module shows what happens next. The agent executes that action. It feeds the result back into its memory.
can take three broad types of .
Each action is wrapped in a — a function the agent is allowed to call.
Your web-research agent has one goal: write a short report on a topic.
search("best SEO practices 2024"). The tool returns ten result snippets.write_file("report.md", content). The file is saved to disk.After each step, the tool's output becomes a new . It is added to the agent's . The next decision has the latest facts.
# A tool is just a Python function with a clear docstring. def search(query: str) -> str: """Search the web and return the top result snippet.""" # Imagine this calls a real search API. return f"Top result for '{query}': SEO means matching user intent." # Register the tool so the agent knows it exists. tools = [search] agent = Agent(instructions="Research topics and write reports.", tools=tools)
A tool is a plain Python function — the agent reads its name and docstring to know what it does.
You pass the function in a tools list when you create the . That list is the agent's menu of allowed actions.
The agent reads the tool's name and docstring. From those, it infers the tool searches the web and expects a query string. The LLM generates the argument (the query) based on the current goal in its context window.
# The agent loop calls the tool and appends the result. def agent_step(agent, goal): tool_name, args = agent.decide(goal) # LLM picks tool + args result = call_tool(tools, tool_name, args) # run the function agent.context.append({ # add result to memory "role": "tool", "content": result }) return result
This is the action-to-observation handoff: the tool runs, and its output is appended to the agent's as a new .
The next call to agent.decide() sees this result — closing the Observe step of the .
Nothing new — the append never ran. The context still holds only the goal and prior messages. The agent has no observation for this step, so it may retry, hallucinate an answer, or stall — depending on how errors are handled. This is a real failure mode (see the next block).
def search(query: str) -> str: """Search the web and return the top result snippet.""" return f"Top result for '{query}': SEO means matching user intent." # TODO: define write_file(filename: str, content: str) -> str # It should save content to filename and return a confirmation string. tools = [search, write_file] agent = Agent(instructions="Research topics and write reports.", tools=tools) result = agent.run("Research SEO best practices and save to report.md") print(result)
This is your guided practice: the search tool is done; you supply the write_file tool (the key new action from this module).
Once both tools are registered, the agent can search and write — completing the full read → decide → write cycle you traced in the scenario.
def write_file(filename: str, content: str) -> str:
"""Save content to a local file and return a confirmation."""
with open(filename, "w") as f:
f.write(content)
return f"Saved {filename} successfully."
# CHANGED LINES vs the worked example:
#
#
# The agent now sees: {"role": "tool", "content": "Saved report.md successfully."}
Three failure modes to watch for when an agent executes actions:
search(query=42) instead of a string. You see: TypeError: expected str, got int. The loop crashes.context_length_exceeded. The agent stops mid-task.Fix: wrap every tool call in try/except. Return a structured error string. The agent always gets an observation — even a failure message.
Before trusting AI-generated tool registration or invocation code, check these four things:
You trace the full Perceive → Decide → Act → Observe cycle for the web-research agent across three iterations, including the termination condition that stops the loop. You complete a partially-filled loop diagram (the first completion exercise) and spot what happens when the loop never stops.
This module traces the full Perceive → Decide → Act → Observe agent loop across three real iterations, teaches three ways to stop it, and shows what happens when it never stops.
Why this matters: Every autonomous agent you build runs this loop — understanding it lets you control when it stops, catch infinite loops early, and write reliable termination logic for your SEO/GEO page agent.
Decision this forces: When should the loop stop — fixed turn limit, goal-met check, or human approval?
Answer: the result is an . It enters the agent's (working memory).
That hand-off — tool result → observation → context — bridges into this module. Now: what happens after the observation lands? How does the agent decide to stop?
An is the repeating cycle that lets an pursue a across many steps.
The loop stops when a is met — otherwise it keeps cycling.
Goal: "Find three recent studies on sleep and memory, then write a summary."
search("sleep memory studies 2024"). Act: tool returns 5 URLs. Observe: URLs added to context.fetch_page(url_1). Act: tool returns article text. Observe: text added to context.write_file("summary.md", ...). Act: file written. Observe: success message. Termination condition met — goal achieved, loop exits.Each iteration is one full Perceive → Decide → Act → Observe cycle. The agent never skips a step.
Every agent needs an explicit — a rule that ends the loop. Without one, the agent runs forever.
DONE when it believes the goal is satisfied. Fast, but the LLM can be wrong.MAX_TURNS = 5 history = [] for turn in range(MAX_TURNS): decision = llm_decide(goal, history) # Perceive + Decide if decision["done"]: break # goal-met check result = run_tool(decision["action"]) # Act history.append(result) # Observe print("Loop ended at turn", turn)
This is the minimal agent loop: four lines of logic, one safety net. It combines a goal-met check (decision["done"]) with a fixed turn limit (MAX_TURNS) as a backstop.
"Loop ended at turn 4" — Python's range(5) gives turns 0–4, so the last value is 4. The loop exits via the for-range ceiling, not the goal-met check. This is the fixed-turn-limit safety net firing.
MAX_TURNS = 5 history = [] for turn in range(MAX_TURNS): decision = llm_decide(goal, history) if decision["done"]: break result = run_tool(decision["action"]) history.append(result) # TODO: add a second stop rule here # so the loop also exits when len(history) >= 3 print("Sources collected:", len(history))
This is a variation of Stage 1 for the research agent. The goal now requires exactly 3 sources — the loop should stop as soon as 3 observations are collected, even if MAX_TURNS isn't reached yet.
Replace the TODO with:
if len(history) >= 3:
break
Changed lines vs Stage 1: only this two-line block is new.
Why it works: after each Observe step, history grows by one. Once it hits 3, the loop exits cleanly — the fixed turn limit is still the backstop if the LLM never fills 3 slots. This is the crux: combining two termination conditions so neither alone can cause runaway or premature exit.
| Option | Stops reliably | Handles partial progress | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Goal-met check | LLM may misjudge 'done' | LLM can summarise what it found so far | Low-stakes tasks where the LLM can judge completion (e.g. summarisation, Q&A). | Low | Low |
| Fixed turn limit | Always stops — may cut off early | Stops mid-task with no graceful wrap-up | Any task where runaway loops are a real risk; use as a safety backstop alongside other conditions. | Low | Low |
| Human approval | Human decides — most reliable | Human can review and redirect | High-stakes or irreversible actions (publishing, payments, deleting data). | Medium | Medium |
The agent calls tools forever. API costs climb and no output appears. Fix: set MAX_TURNS as a hard ceiling.
The LLM sets done=True after one search. The goal needed three sources. Output exists but is incomplete. Fix: add a goal-check prompt listing required deliverables.
The agent calls the same search query every turn. History grows and context fills. You get a overflow error. Fix: track seen actions and skip duplicates.
You compare fully autonomous agents against human-in-the-loop designs, using the web-research agent to show three real failure modes (hallucinated sources, infinite loops, irreversible actions) and the guardrails that fix them. You finish by deciding where your own SEO-page agent would sit on the autonomy dial.
This module compares fully autonomous agents with human-in-the-loop designs, shows three real failure modes, and teaches the guardrails that fix them.
Why this matters: Choosing the wrong autonomy level for your SEO agent can publish broken pages or run up API costs — this module gives you the decision framework to avoid both.
Decision this forces: For a given task, should the agent act fully autonomously, pause for human approval, or require human confirmation before irreversible steps?
Module 4 showed that the runs Perceive → Decide → Act → Observe. It stops when a fires.
The condition checks two things: is the met? Did we hit the step limit? Now: who decides when to stop — the agent or you?
Every sits somewhere on an dial between two extremes.
Most real agents live in between — autonomous for safe steps, HITL for risky ones.
Full autonomy creates three specific failure modes. Each is invisible until it causes real damage.
Your web-research agent is writing an SEO page about "autonomous AI agents". Watch all three failures appear in one run.
Each failure has a matching — you will add them in the code stage next.
def run_agent(goal, tools, max_steps=None): steps = 0 while True: # no step limit — infinite loop risk action = llm_decide(goal) # may hallucinate a source if action.name == "publish_page": tools["publish_page"](action.args) # irreversible, no confirm else: result = tools[action.name](action.args) if llm_is_done(goal, result): break
This agent has all three failure modes baked in. Spot them before reading the fix.
IRREVERSIBLE = {"publish_page", "send_email", "delete_file"}
def run_agent(goal, tools, max_steps=10): # guardrail 1: step limit
steps = 0
while steps < max_steps:
action = llm_decide(goal)
if action.name in IRREVERSIBLE: # guardrail 2: human confirm
ok = input(f"Allow {action.name}? (y/n): ")
if ok.lower() != "y":
break
result = tools[action.name](action.args)
verify_sources(result) # guardrail 3: source check
if llm_is_done(goal, result):
break
steps += 1Three lines fix three failure modes. max_steps kills infinite loops; IRREVERSIBLE forces human approval; verify_sources() catches hallucinated URLs before they reach the page.
steps tracks the count (incremented at the bottom of the loop).
IRREVERSIBLE = {"publish_page", "send_email", "delete_file"} gates dangerous actions.
Removing the IRREVERSIBLE check brings back irreversible actions — the agent publishes or deletes without asking. The changed lines vs Stage 1: added max_steps param, added steps counter, added IRREVERSIBLE set + confirm block, added verify_sources() call.
| Option | Risk of irreversible harm | Need for speed | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Fully Autonomous | High — no human gate before destructive actions | Best — no waiting for human input | Low-stakes, reversible tasks with a clear termination condition — e.g. drafting a first-pass outline. | Low human time; higher API cost if loops run long | Low — no approval logic needed |
| Human-in-the-Loop | Low — human confirms before each dangerous action | Slower — waits for human at each gate | Any step that publishes, deletes, sends, or spends money — e.g. publishing your SEO page. | Higher human time; lower risk of costly mistakes | Medium — requires approval UI or prompt |
| Hybrid (Autonomous + HITL gates) | Low for gated actions; autonomous elsewhere | Fast for safe steps; brief pause at risky ones | Most production agents — autonomous for research and drafting, HITL only for publish/delete. | Balanced — human time only at high-risk steps | Medium — define the IRREVERSIBLE set carefully |
max_steps actually increment? Trace the counter — AI often places it in the wrong branch.IRREVERSIBLE set? AI forgets domain-specific ones like update_index().Your SEO agent decision: use Hybrid — autonomous for research and drafting, HITL gate before publish_page().
You now have every piece — perception, planning, execution, the loop, and autonomy controls. The capstone challenge asks you to wire them all together into one working SEO agent.
Before reading on — from memory, name the four steps of the agent loop in order, then name one thing that can go wrong at each step. Check your answer against the spine: Perceive → Decide → Act → Observe → (repeat or stop).
Apply what you learned to SEO/GEO page — the page title is the H1 and the exact search query. OPEN with a 40-60 word self-contained, quotable answer capsule that names the entity explicitly in sentence one (no "in this lesson…" preamble) — this is the sentence an LLM lifts. Name the entity by its full name every time (not pronoun-only). Structure each section as ONE sub-question (How it works · X vs Y · When to use it · Example · Common mistakes), each opening with its own 1-2 sentence direct answer then depth. End with a genuine 3-5 question FAQ phrased as real user questions ending in "?" (these become FAQPage schema + the knowledge check). Prefer quotable specifics — concrete numbers, defaults, version names, one short snippet — over vague prose. Self-contained for a reader who arrived cold from a search engine or an LLM. Intent = definition: define the named entity + how it works + a concrete example. Target query: "autonomous ai agents"..
An autonomous AI agent receives this raw tool output:
{"status": 200, "body": "<html>...</html>"}
Before the agent can reason about this page, what must happen to this data?
Raw environment data — like an HTML blob — must be parsed and structured before the agent can act on it; this is the perception step. The language model does not 'read HTML natively' in any useful planning sense — unformatted HTML floods the context with noise. Storing and skipping defeats the purpose of perception. HTTP responses are a perfectly valid input type (alongside text, images, sensor streams, and more).
In the ReAct pattern, what is the correct order of the three steps?
ReAct stands for Reason + Act. The agent first produces a Thought (its reasoning about what to do), then issues an Action (a tool call or step), then receives an Observation (the result). Starting with Action skips planning and wastes tool calls. Starting with Observation makes no sense before any action has been taken. Thought → Observation → Action would mean the agent observes before it has acted, which is out of order.
Read this tool-call snippet:
tool = "web_search"
args = {"query": "best running shoes 2024"}
result = run_tool(tool, args)
What happens to 'result' after this line executes?
After a tool call executes, the result becomes an observation that is appended to the agent's context window — this is how the action output feeds back into the reasoning loop. Discarding the result would break the loop entirely. Writing only to long-term storage without surfacing it to the model means the agent cannot use it in the current cycle. Tool results do not automatically replace goals; goal management is separate.
An autonomous AI agent is deleting old customer records from a production database. Which termination and control strategy is most appropriate?
Deleting production records is irreversible, so a guardrail requiring human confirmation before each destructive action is essential. A fixed turn limit alone does not prevent harmful deletions within those turns. A goal-met check alone gives the agent unchecked authority over irreversible steps. Fully autonomous operation with no guardrails is the textbook failure mode for high-stakes irreversible actions — one of the core breakdown patterns covered in the autonomy module.
Name two failure modes specific to autonomous AI agents (not general software bugs) and, for each one, state one concrete fix.
Autonomous agents have failure modes that don't appear in ordinary software: they can loop forever (no exit condition), pursue the wrong goal (misaligned termination), take irreversible actions without oversight, or be hijacked by adversarial content in their environment. Each requires a targeted guardrail — a generic 'add error handling' answer is not sufficient because these failures stem from the agent's autonomy and reasoning loop, not from code exceptions.