Define functions, execute calls safely, and return tool results to the model.
A plain chatbot guesses from training data; a tool-calling chatbot can fetch live facts and take actions. This module pins the single user-visible task you'll make work first.
Defines the user-visible job and which calls must hit a real tool, not the model's guess.
Why this matters: Every later step is judged against this target; without it you can't tell done from drifting.
A plain answers from frozen training data. Ask it today's order status, the current exchange rate, or to actually send an email, and it can only guess or apologize.
A tool-calling chatbot fixes this. You hand the model a list of functions it may use, and when a question needs live data or a real action, the model asks you to run one. Your code runs it and feeds the result back, so the answer is grounded in reality, not memory.
Before writing any code, pin one concrete target task. A sharp target tells you exactly which calls must hit a real tool.
A user types "Has my order shipped yet? It's #4821." The right behavior: the bot does NOT guess. It calls get_order_status("4821"), gets back "in transit, ETA Thursday", and only then writes a reply.
Contrast the failure: a plain chatbot says "Your order should arrive in 3-5 business days" — plausible, confident, and completely made up. That gap is exactly why you reach for tool calling.
The smallest round trip: user asks, model requests a tool, your code runs it, model answers.
Write a single sentence: "Given an order ID, tell the user the current shipping status." One task, not a platform.
Identify what the model cannot know: the order's real status. That fact must come from a tool, so it becomes your first function: get_order_status.
Pick the failure to catch: the bot inventing a delivery date. Your test asserts that the date in the reply matches what the tool returned.
Draw a hard line. The model decides WHICH tool to call and with WHAT arguments. Your application decides whether that call is allowed, runs it, and owns the real side effects (database reads, payments, emails).
Now you have a task. This module shows how to declare the tool to the model with a schema, and the minimal loop that runs whatever call the model asks for and feeds the result back.
How you declare a tool to the LLM and the minimal observe-decide-act-update loop around it.
Why this matters: The schema and loop are the skeleton; get this wrong and no amount of prompting fixes it.
The model can only call tools you describe to it. A tool schema is a small structured description you pass alongside the user message. Every schema has three parts.
Around the schema sits the : send message + tools, read the model's reply, if it is a tool call then run it and append the result, then call the model again. Repeat until the model returns plain text instead of a tool call.
Given the user message "Where is order 4821?" and a get_order_status schema, the model does not reply with prose. It replies with a structured tool call: name = "get_order_status", arguments = {"order_id": "4821"}.
Your loop sees that shape, knows it is not a final answer, runs the matching function, and appends {"status": "in transit"}. On the next call the model — now holding the real status — returns plain text, and the loop ends.
TOOLS = [{
"name": "get_order_status",
"description": "Look up the live shipping status of an order by its ID.",
"parameters": {"order_id": "string"},
}]
def run_turn(messages):
reply = model.call(messages, tools=TOOLS)
if reply.tool_call: # model wants a tool
result = dispatch(reply.tool_call)
messages.append(result) # feed it back
return run_turn(messages) # loop again
return reply.text # plain text = doneTOOLSreply.tool_callreturn run_turn(messages)The schema teaches the model the tool exists; run_turn is the loop. The recursion is the loop body — it keeps calling the model until the reply is plain text instead of a tool call.
The model never sees the tool's result, so it asks for the exact same tool again — an infinite loop. The append is what lets the loop make progress.
You call the model with the user's question AND the list of tool schemas it may use. The model now knows get_order_status exists and what it takes.
The reply is one of two shapes: plain text (the model is done) or a tool call (it wants you to run something). You branch on which it is.
If it is a tool call, run the function, append the result to the conversation, and call the model again. The new result is now part of what the model sees on the next turn.
The loop has caught a tool call. This module builds the dispatch layer that maps the model's requested name to your real function, runs it safely, and returns a result the model can use.
The dispatch table that maps a requested tool name to real code, runs it, and returns the result.
Why this matters: This is where the chatbot actually touches the world; the core of the whole build.
Decision this forces: Choose the dispatch rigor that matches a tool's worst side effect: bare dict, validated, or gated.
(Answer: a tool call, containing a name and a set of arguments. Dispatch is what we do with those two fields next.)
The model hands you a tool call with a name and arguments. Dispatch is the lookup that turns that name into a real function and runs it with those arguments. The simplest form is a dictionary mapping names to functions.
Three things every dispatch layer must do: reject any name not in the table (the model can hallucinate one), pass the arguments through, and return the result in a shape the model can read back.
Map the schema name to real code: REGISTRY = {"get_order_status": get_order_status}. The key must match the schema's name exactly.
If the model asks for a name not in REGISTRY, return a clear error string rather than crashing. Hallucinated tool names are common, so never call REGISTRY[name] blindly.
Invoke the function with the arguments, then return its output as JSON so the model can parse it on the next turn.
| Option | Argument handling | Side-effect safety | When it fits | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Bare dict dispatch | Passes args straight through, no checks. | None — any registered tool just runs. | Read-only tools in a prototype. | Use to get a demo working when every tool is read-only and safe. | Low | Low |
| Validated dispatch | Validates args against the schema before calling. | Catches bad types and missing fields early. | Any tool with real reads users depend on. | Use once real users see the output and bad arguments would surface as wrong answers. | Medium | Medium |
| Gated dispatch | Validates, plus per-tool permission checks. | Requires confirmation for writes/payments. | Tools that send email, charge cards, or delete data. | Use whenever a tool has irreversible side effects. | High | High |
If you let an AI assistant generate your dispatch code, check these four things before running it — they are exactly where generated tool-calling code goes wrong.
Tool arguments come from the model, which means they are untrusted. This module adds argument validation and a trace of every call, so bad inputs and silent failures surface before a user does.
Argument validation, error handling, and a trace of each tool call so failures are visible early.
Why this matters: Tool calls are where untrusted model output meets real systems; this is the safety layer.
The arguments in a tool call are generated by the , not typed by your code. The model can omit a required field, send the wrong type, or invent a plausible-but-wrong value. Treat every argument as untrusted until validated.
Two habits make tool calls safe and debuggable.
The model emits get_order_status(order_id=4821) — an integer, but the schema says string. Without validation, the database lookup silently returns nothing and the bot says "I couldn't find that order," confusing a user whose order exists.
With validation, the layer catches the type mismatch, returns "order_id must be a string" to the model, and the model retries with "4821". The trace shows both attempts, so you can see the model self-corrected.
def safe_dispatch(call, registry, schema): if call.name not in registry: return {"error": f"unknown tool {call.name}"} for field, ftype in schema[call.name].items(): # TODO: reject the call if a required field is # missing or the wrong type — return an error dict ... try: result = registry[call.name](**call.args) trace.log(call.name, call.args, result) return {"result": result} except Exception as e: trace.log(call.name, call.args, error=e) return {"error": str(e)}
if call.name not in registryschema[call.name].items()try / exceptThe dispatch, error handling, and tracing are wired. The one missing piece is the validation guard — predict what it should do, then reveal the answer.
Check that the field exists in call.args and that type(call.args[field]) matches ftype. If a field is missing or mistyped, return {"error": f"{field} must be {ftype}"} so the model can retry — do NOT raise, or you lose the chance for the model to self-correct.
Before calling the function, confirm each required parameter is present and the right type. If not, return a validation error to the model so it can retry with corrected arguments.
Real tools fail: the order API times out, the row is missing. Catch the error and return it as a result the model can read, instead of letting the loop crash.
Log {name, args, result_or_error, ms} for every call. When a user reports a bad answer, the trace tells you exactly which tool call went wrong.
The loop works on the happy path. This module covers the ways tool-calling bots break in production — infinite loops, runaway costs, unsafe writes — and the guardrails that separate a demo from a shippable bot.
Turn budgets, retry and fallback rules, and the failure modes that bite tool-calling bots in production.
Why this matters: The gap between a demo and a shippable bot is almost entirely these guardrails.
Decision this forces: Choose which guardrails must ship based on whether your tools are read-only, reversible, or irreversible writes.
Your runs, dispatches, and validates. What still separates it from production is that nothing stops it from misbehaving when things go wrong. Hardening adds the limits.
These are the failures that don't show up in a quick demo but bite once real traffic hits. Each maps to a guardrail.
The model keeps calling tools and never decides it is done — often because a tool returns an error it doesn't know how to recover from. Without a turn budget, this runs until you hit a rate limit or a large bill. Fix: cap turns and force a final answer when the cap is hit.
A tool fails, the model tries the exact same call again, it fails again. The loop "works" but makes no progress. Fix: detect identical consecutive calls and break out with a fallback message.
The model, trying to be helpful, calls a write tool — cancel_order, send_refund — that it shouldn't have. Read tools are recoverable; writes are not. Fix: gate every side-effecting tool behind a confirmation or a scoped permission, so a single bad decision can't charge a card.
Every tool result is appended to the conversation. After many turns the context window fills, early instructions get dropped, and the bot "forgets" the original task. Fix: summarize or prune old tool results once the conversation gets long.
Pass max_turns into run_turn and decrement it each pass. At zero, stop looping and ask the model for a final answer with the tools it has — never let it loop unbounded.
Track the last tool call. If the model emits the identical call after a failure, break out and return a graceful fallback rather than retrying forever.
Tag each tool as read or write. Read tools run freely; write tools require a confirmation step before dispatch executes them.
| Option | Tool type | Worst case if it breaks | Minimum guardrail | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Read-only bot | Lookups only: status, search, FAQ. | A wrong or stale answer. | Turn budget + trace. | Ship with turn budget and tracing once read tools are validated. | Low | Low |
| Read + reversible write | Creates drafts, tickets, notes you can undo. | A bad record you can delete later. | Above, plus retry/fallback + repeat detection. | Add fallback handling and repeated-call detection before exposing writes. | Medium | Medium |
| Irreversible write | Charges cards, sends email, deletes data. | Real money lost; cannot undo. | Above, plus confirmation gating + scoped permissions. | Never ship without a human or rule-based confirmation in front of the write. | High | High |
Before deploying a tool-calling chatbot, confirm each of these is in place. They map one-to-one to the failure modes above.
From memory, reconstruct the build: name the one task that needs a tool, the three parts of a tool schema, the four steps of the loop, what dispatch does with a tool call, the two safety habits at the dispatch boundary, and the failure mode each guardrail prevents. If any step is fuzzy, reopen that module before you build.
Build a working order-status bot end to end: define one tool schema, write the call-back loop, add a guarded dispatch table, validate arguments, trace every call, and cap the loop with a turn budget. Test it on a question the model can't answer from memory. NEXT RUNG: add a second tool (e.g. cancel_order) that is an irreversible write, and gate it behind a confirmation step — then move on to multi-tool agents that chain several calls per turn.
From memory, name the four steps in one tool-calling exchange, from the user's request to the final reply.
A tool-calling bot works because the model requests a tool, your code runs it, and the model then answers using the returned result.
Your bot is asked, “What is the current status of order #1842?” Why should it use a tool instead of answering from training data?
Tools are for real, current, private, or action-based information that the model should not invent from training data.
Which tool schema is best for a function that looks up weather by city and number of forecast days?
A useful schema has a clear name, a specific description, and typed parameters the model can fill correctly.
You are building dispatch for a tool that permanently deletes a customer record. Which dispatch rigor is the best fit?
Irreversible or high-impact writes need gated dispatch because the worst side effect is too serious to trust a simple mapping alone.
A model emits a tool call with an unknown tool name and an argument type that does not match the schema. What should your app do first?
Validation and tracing make bad calls visible and safe, while retry/fallback policies prevent endless or unsafe loops.