Trace model calls, retrieval, tools, scores, and costs end to end.
Install the Langfuse Python SDK, configure your API keys, and initialize the `Langfuse` client that every subsequent span will attach to. You'll also open your first trace object — the root container for one user request.
Install the Langfuse Python SDK, configure credentials via environment variables, and open your first trace object — the root container every span attaches to.
Why this matters: This is the foundation layer: without a correctly initialized client and a root trace, none of the spans, generation records, or cost data you'll add in later modules will reach Langfuse.
Decision this forces: Decorator-based auto-instrumentation vs. manual SDK calls — choose based on how much control you need over span boundaries.
Your agent just gave a user a wrong answer — but which step failed? The LLM call? The retrieval? A bad prompt? Without , you're guessing.
Langfuse solves this by wrapping every user request in a — a root container — and every internal operation in a . Together they give you a causal timeline: what ran, in what order, with what inputs and outputs.
This module covers the one-time setup: install the SDK, wire in your credentials, and open your first trace. Every span you create in later modules attaches to this root.
flush() before your process exits, what happens to spans that haven't been sent yet?from langfuse import Langfuse # Credentials hardcoded — never do this in real code client = Langfuse( public_key="pk-lf-...", secret_key="sk-lf-...", host="https://cloud.langfuse.com" ) trace = client.trace(name="user-request") print(trace.id)
Langfuse(...)client.trace(name=...)trace.idThis runs — but hardcoding credentials means they leak into version control the moment you commit.
The real failure is silent: if public_key or secret_key is wrong, Langfuse accepts the client object without error. Spans are queued locally and then silently dropped when the first flush attempt returns a 401.
Nothing — the constructor succeeds. You only discover the bad key later when flush() fires and the HTTP response is 401 Unauthorized. No local exception is raised at construction time.
import os from langfuse import Langfuse # Set in your shell or .env file — never in source: # export LANGFUSE_PUBLIC_KEY="pk-lf-..." # export LANGFUSE_SECRET_KEY="sk-lf-..." # export LANGFUSE_HOST="https://cloud.langfuse.com" client = Langfuse() # SDK reads the env vars automatically print(client.auth_check()) # True = credentials valid
Langfuse()client.auth_check()When the three environment variables are set, Langfuse() needs no arguments — the SDK picks them up automatically.
Call auth_check() once at startup to validate credentials eagerly, before any real traffic hits. It returns True on success and raises an exception on a bad key — the only moment you get a synchronous error.
It raises a requests.exceptions.ConnectionError (or similar network error) — not a Langfuse-specific exception. This is the one place to catch connectivity problems before your app starts serving traffic.
trace = client.trace( name="lesson-generation", user_id="user-42", session_id="sess-abc123", release="v1.4.2", metadata={"kb_version": "2024-06-01"}, ) # ... your agent logic runs here ... client.flush() # send all queued spans before process exits print("Trace ID:", trace.id)
user_id=session_id=release=metadata={...}client.flush()This is the complete, production-safe pattern: one per user request, tagged with stable identifiers so you can filter by user, session, or release in the Langfuse dashboard.
The call at shutdown drains the background queue. Without it, any spans buffered in the last few seconds of your process are lost — the answer to the prediction you made at the start.
Add metadata={"kb_version": "2024-06-01", "experiment": "variant-B"} inside the trace() call — or call trace.update(metadata={"experiment": "variant-B"}) later. The metadata dict is a free-form key-value store; use stable, low-cardinality keys so dashboard filters stay usable. Changed lines: the metadata dict gains one key.
Three failure patterns account for most lost traces in production:
flush() returns HTTP 401 and drops the batch silently. Fix: call auth_check() at startup.client.flush() in your shutdown hook or atexit handler.name field (e.g. "request-1719432001") makes the Langfuse dashboard unsearchable — every trace is unique. Keep name a stable label like "lesson-generation"; put dynamic values in metadata or user_id.If you used an AI assistant to generate your SDK initialization, check these four things before shipping:
pk-lf- or sk-lf- — any match is a leak.finally block, atexit handler, or framework lifecycle hook — not just at the bottom of a script that might raise before reaching it.name, move it to metadata.Drag to see how the flush interval trades off network calls against the window of data you could lose on an unclean exit.
Wrap each LLM call in a `generation` span that records the model name, prompt messages, completion, token counts, and latency. You'll see how Langfuse auto-computes cost from token usage and model name, and how to attach the span to the parent trace.
Wrap each LLM call in a generation span that records the prompt, completion, token counts, and latency — and let Langfuse auto-compute cost.
Why this matters: Without generation spans, your traces are empty shells — you can't debug what the model saw, what it returned, or what it cost.
Decision this forces: Pass `usage` explicitly vs. rely on Langfuse's model-price table — explicit is safer when using fine-tuned or custom models.
A is the root container for one user request. Every span attaches to it via the you initialized. This parent-child link builds the call tree in the UI.
This module focuses on the — the most important child. It wraps each LLM call. The key question: how do you record what the model saw, what it returned, and what it cost?
A is a specialised for LLM calls. It carries the model name, prompt messages, completion text, , and latency.
Langfuse uses the model name and token counts to auto-compute from its price table. You get dollar figures in the UI without extra code, if the model name matches a known entry.
Latency derives automatically from the span's start and end timestamps. Langfuse records these when you call generation.end() or the context manager exits.
model — the exact model ID string (e.g. gpt-4o).input — the prompt messages array sent to the model.output — the completion text returned.usage — prompt_tokens, completion_tokens, total_tokens.Imagine your app answers a user question by calling GPT-4o. Without a generation span, Langfuse sees the trace but the LLM call is a black box. No prompt, no cost, no latency breakdown.
With a generation span, the flow is: open the trace, start a generation span before the API call, make the call, then end the span with the completion and token counts. Langfuse stitches them into a call tree and computes cost automatically.
End the span after you receive the response, not before. That gives you accurate latency. If you forget to call end(), the span stays open and latency shows as zero or null in the UI.
# A common first attempt — span opened but never ended trace = langfuse.trace(name="answer_question") generation = trace.generation( name="llm_call", model="gpt-4o", input=[{"role": "user", "content": user_question}], ) response = call_llm(model="gpt-4o", messages=[...]) # ❌ forgot: generation.end(output=..., usage=...)
trace.generation(...)model="gpt-4o"input=[{"role": ..., "content": ...}]This is the most common first mistake: the span is created but generation.end() is never called, so Langfuse has no completion, no tokens, and no end timestamp.
Latency shows as 0 ms (or null) because the span was never closed. Cost shows as $0.00 because token counts were never recorded. The span appears 'open' in the trace view — a silent failure with no error thrown.
trace = langfuse.trace(name="answer_question", user_id=user_id) generation = trace.generation( name="llm_call", model="gpt-4o", input=[{"role": "user", "content": user_question}], ) response = call_llm(model="gpt-4o", messages=[...]) generation.end( output=response.choices[0].message.content, usage={"prompt_tokens": response.usage.prompt_tokens, "completion_tokens": response.usage.completion_tokens}, )
generation.end(output=..., usage=...)usage={"prompt_tokens": ..., "completion_tokens": ...}trace.generation(...)Calling generation.end() with output and usage gives Langfuse everything it needs to compute latency, total tokens, and cost automatically — no extra calls required.
1) Latency — from the timestamp difference between generation() and end(). 2) Total tokens — by summing prompt_tokens + completion_tokens from the usage dict. 3) Cost — by looking up the model name 'gpt-4o' in its price table and multiplying by token counts.
# Scenario: wrap a summarisation call for a support ticket trace = langfuse.trace(name="summarise_ticket", user_id="agent-99") generation = trace.generation( name="summarise", model="gpt-4o", input=[{"role": "system", "content": "Summarise the ticket."}, {"role": "user", "content": ticket_text}], ) response = call_llm(model="gpt-4o", messages=[...]) # TODO: call generation.end() with output and usage # Hint 1: pull the text from response.choices[0].message.content # Hint 2: pull token counts from response.usage.*_tokens
# TODO: call generation.end(...)response.usage.prompt_tokensresponse.choices[0].message.contentThis is a variation of the Stage 2 pattern applied to a new scenario — a support-ticket summariser. The crux is supplying the right output and usage values to generation.end() — the same two fields that unlock latency and cost in the UI.
Once this pattern is solid, the next module extends it: you'll wrap your vector-store lookup in a — logging the query, the chunks returned, and similarity scores — so the full RAG pipeline appears as a linked call tree.
generation.end(
output=response.choices[0].message.content,
usage={
"prompt_tokens": response.usage.prompt_tokens,
"completion_tokens": response.usage.completion_tokens,
},
)
Changed lines vs. Stage 2: only the span name ('summarise') and input messages differ — the end() call is identical in structure. That's the point: the pattern is reusable across every LLM call in your app.
trace.generation() but an exception fires before generation.end(). Symptom: latency = 0 ms, cost = $0.00, span status = 'open' in the UI. Fix: wrap the call in try/finally and call end() in the finally block.model="gpt-4o-2024-08" but Langfuse's table has "gpt-4o". Cost shows $0.00 silently. Fix: pass explicit usage so cost is always computed from real token counts.langfuse.flush() before the process exits.Wrap your vector-store or search call in a `span` of type `retrieval`, logging the query string, the chunks returned, and any similarity scores. You'll revisit the parent-trace concept from Module 1 to correctly nest retrieval before the generation that consumes it.
How to wrap a vector-store search in a Langfuse retrieval span that logs the query, returned chunks, and similarity scores — nested correctly before the generation span.
Why this matters: Without a retrieval span you can't tell whether a bad RAG answer came from the retriever or the LLM — this module gives you the visibility to diagnose and fix it.
Decision this forces: Log full chunk text vs. chunk IDs only — full text aids debugging but increases payload size; IDs keep traces lean.
Answer: model name, token usage (prompt + completion tokens), and the completion text. Langfuse multiplies the token counts against its model price table to derive cost automatically.
That generation span sits inside a parent — the root container for one user request. This module adds the that must come before that generation in the same trace. The question is: what does a retrieval span log, and how do you nest it correctly?
A wraps your vector-store or search call. It logs three things: the query string sent to the index, the chunks returned, and the similarity scores for each chunk.
You create it by calling trace.span() with type="retrieval" before your search call. Then end it after. Langfuse records retrieval latency separately from the LLM call.
Nesting matters. The retrieval span's parent must be the same or a parent span that the generation span also attaches to. If you open a new trace for retrieval, the two steps appear as unrelated requests in the UI. You lose the causal chain.
Empty results and low scores are the two retrieval failure signals you can read directly from the trace view. But only if you log them.
Your RAG (retrieval-augmented generation) assistant answers a user's question about refund policy with a vague, hedged response. The LLM call looks fine in the trace — correct model, reasonable latency. But without a retrieval span, you can't tell whether the retriever returned the right chunks.
You add a retrieval span around the vector-store call. On the next run you open the trace and see: query = "refund policy for annual plan", three chunks returned, top similarity score = 0.41. That score is well below your 0.70 threshold — the retriever is surfacing loosely related content, not the policy document.
Without the logged score you'd have blamed the LLM. With it, you know the fix is in the index or the query — not the prompt.
from langfuse import Langfuse client = Langfuse() # keys from env trace = client.trace(name="rag-request", input={"question": "refund policy for annual plan"}) retrieval_span = trace.span( name="vector-search", input={"query": "refund policy for annual plan"}, ) chunks = vector_store.search("refund policy for annual plan", top_k=3) retrieval_span.end(output={"chunks": [c.text for c in chunks]})
client.trace(...)trace.span(name=..., input=...)retrieval_span.end(output=...)This opens a retrieval span on the existing trace, runs the search, then closes the span with the returned chunks as output. The span's start and end timestamps give you retrieval latency in the Langfuse UI.
The retrieval span stays open with no end timestamp. Langfuse shows it as 'pending' or with a null duration, and the timeline order between retrieval and generation becomes unreadable — you can't tell which finished first.
retrieval_span = trace.span( name="vector-search", input={"query": query}, ) chunks = vector_store.search(query, top_k=3) retrieval_span.end( output={ "chunks": [c.text for c in chunks], "scores": [round(c.score, 3) for c in chunks], } ) gen = trace.generation(name="answer-llm", input=build_prompt(chunks), model="gpt-4o") response = llm.complete(gen.input) gen.end(output=response.text, usage={"promptTokens": response.prompt_tokens, "completionTokens": response.completion_tokens})
[round(c.score, 3) for c in chunks]trace.generation(name=..., input=..., model=...)gen.end(output=..., usage=...)The retrieval span closes before the generation span opens — this is the correct ordering. Both attach to the same , so the UI timeline shows retrieval → generation as a causal sequence, not two parallel or unrelated spans.
Open the retrieval span's output: all three scores are below 0.50, meaning the vector store returned loosely related chunks. The generation span's input (the prompt) will show those weak chunks were fed to the LLM — confirming the retriever, not the prompt template, is the problem.
# Log session metadata on trace; log chunk IDs+scores (not full text) on span. trace = client.trace( name="rag-request", input={"question": query}, # TODO: add session_id="user-42" as metadata here ) retrieval_span = trace.span(name="vector-search", input={"query": query}) chunks = vector_store.search(query, top_k=3) retrieval_span.end( output={ # TODO: log chunk IDs (c.id) and scores — NOT full text } )
metadata={"session_id": ...}"chunk_ids": [c.id for c in chunks]Stop — attempt both TODOs before revealing. Hint 1: client.trace() accepts a metadata dict for arbitrary key-value pairs. Hint 2: the output dict keys are your choice — name them clearly so the trace is self-documenting.
Changed lines:
trace = client.trace(..., metadata={"session_id": "user-42"})
retrieval_span.end(output={"chunk_ids": [c.id for c in chunks], "scores": [round(c.score, 3) for c in chunks]})
Why: metadata={} is the standard place for session/run context that isn't the primary input. Logging c.id instead of c.text keeps the payload small — the tradeoff covered next.
| Option | Debugging speed | Payload size | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Full chunk text | Instant — chunk content visible inline in the span output | Can be large; long chunks or many results bloat the trace significantly | During active development or when diagnosing retrieval quality issues — you need to read the chunks without leaving the trace view. | Higher — large payloads increase Langfuse storage and slow the UI on long traces. | Low |
| Chunk IDs only | Slower — requires a separate lookup in your vector store to read chunk content | Minimal; IDs are short strings regardless of chunk length | In production or high-volume pipelines where trace storage cost matters and you can look up chunks from your index by ID. | Lower — minimal payload, fast trace ingestion. | Low |
Three failure patterns show up repeatedly. Two of them are silent without a retrieval span.
"chunks": []. Add a guard that logs a warning and short-circuits before generation."scores": [...] in the span output.client.trace() again inside the retrieval function instead of passing the existing trace, retrieval and generation appear as two unrelated requests. The timeline is broken. Latency attribution is wrong.Verify AI-generated retrieval spans before trusting them. Check that (1) retrieval_span.end() is called in every code path, including exception handlers; (2) the output dict contains both chunks/IDs and scores; and (3) the span's parent is the same trace object the generation span uses, not a freshly created one.
Apply the same span pattern to tool calls — code interpreter, web search, API calls — capturing function name, input arguments, output, and errors. You'll also handle the async case where a tool fires mid-generation and must be nested inside the generation span.
How to wrap tool calls — web search, API calls, code interpreter — in Langfuse spans that capture function name, arguments, output, and errors, and how to nest them correctly inside generation spans.
Why this matters: Tool calls are where agentic workflows most often fail silently; tracing them gives you a searchable, visual record of every invocation so you can pinpoint failures without reading raw logs.
Decision this forces: One span per tool invocation vs. one span per tool type — per-invocation gives finer granularity; per-type reduces noise for high-frequency tools.
Answer: you logged the query string, the chunks returned, and similarity scores. You attached the as a child of the root .
Tool calls follow the same nesting pattern. But now the parent is often a , not the trace root. That shift is what this module covers.
Every tool invocation — code interpreter, web search, API call — deserves its own . Record four things: function name, input arguments, output, and error.
When a tool fires mid-generation (model calls it while streaming), nest the tool span inside the generation span. This shows causality in the timeline.
Errors are first-class. Set level="ERROR" and write the exception message to status_message. The span appears red in Langfuse UI.
Key design: one span per invocation vs. one per tool type. Per-invocation gives searchable records. Per-type reduces timeline noise.
An agent answers a question about today's stock price. The model calls a web_search tool mid-generation. Without a tool span, the trace shows only generation start and end — the search call and timeout error are invisible.
With a nested tool span, Langfuse shows: generation opens → tool span opens (query: "AAPL price today") → tool span closes (output: "$213.42" or error: "timeout") → generation closes.
If the search times out, the tool span turns red. The generation span stays amber. You immediately see which call failed.
trace = langfuse.trace(name="stock_agent") tool_span = trace.span( name="web_search", input={"query": "AAPL price today"}, ) result = web_search(query="AAPL price today") # your actual tool tool_span.end(output={"result": result})
trace.span(name=..., input=...)tool_span.end(output=...)This is the minimal tool-span pattern: open the span with structured input, run the tool, then close with structured output.
The span is a direct child of the trace here — next stage nests it inside a generation span instead.
The span is buffered in memory but never sent — the trace appears empty or incomplete in the UI. Always call langfuse.flush() (or use a context manager) before the process exits.
gen_span = trace.generation( name="answer_generation", input=messages, model="gpt-4o", ) # tool fires mid-generation tool_span = gen_span.span( name="web_search", input={"query": "AAPL price today"}, ) result = web_search(query="AAPL price today") tool_span.end(output={"result": result}) gen_span.end(output={"answer": final_answer})
gen_span.span(name=..., input=...)gen_span.end(output=...)The delta from Stage 1: tool_span is now opened on gen_span, not on trace — this makes the tool a child of the generation in the UI timeline.
Always close the tool span before closing the generation span; closing the parent first orphans the child and breaks the timeline.
The tool span is orphaned — it either attaches to the trace root (appearing as a sibling of the generation, not a child) or its timestamps fall outside the generation span's window, making the timeline misleading. Close children before parents.
gen_span = trace.generation( name="answer_generation", input=messages, model="gpt-4o", ) try: tool_span = gen_span.span( name="web_search", input={"query": "AAPL price today"}, ) result = web_search(query="AAPL price today") # TODO: close tool_span with the result AND mark it successful except Exception as e: tool_span.end(level="ERROR", status_message=str(e)) gen_span.end(output={"answer": final_answer})
level="ERROR"status_message=str(e)try / except around tool callStop — attempt the TODO before revealing. The missing line closes the tool span on the happy path.
Hints: (1) use the same method you saw in Stage 1; (2) pass the result as a structured dict, not a raw string.
Changed line (replaces TODO):
tool_span.end(output={"result": result})
Why: .end(output=...) closes the span and records structured output — omitting it leaves the span open indefinitely, which Langfuse shows as a 'dangling' span with no end timestamp. The except branch already handles the error path with level="ERROR" and status_message=str(e), which turns the span red in the UI.
| Option | Debuggability (can you pinpoint which call failed?) | Timeline noise (how cluttered is the trace UI?) | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| One span per invocation | Full: each call has its own input, output, and error record | High noise when the same tool fires dozens of times | Use when tools fire infrequently (< ~10 calls per trace) or when you need to debug individual call failures — e.g. a web-search agent that calls 3–5 URLs per run. | Trace storage grows linearly with call count | Low — wrap each call site once |
| One span per tool type | Limited: errors are aggregated, not individually addressable | Clean: one node per tool type keeps the timeline readable | Use when a single tool fires at high frequency (e.g. a calculator called 50+ times per trace) and individual call details are less important than aggregate behavior. | Flat storage regardless of call count | Medium — requires batching or summarising calls before closing the span |
Three failure patterns repeat — two produce no error message and are easy to miss.
.end(). Langfuse shows the span as still-running with "—" duration. Fix: wrap the tool call in try/finally. Call .end() in the finally block.trace.span() instead of gen_span.span() for a mid-generation tool. The tool appears as a sibling of the generation — causality is lost. Observable symptom: the tool node sits at the same indent level as the generation, not indented beneath it.output=str(result) means error strings and success values look identical in the UI. Use a dict with consistent shape: {"result": ..., "error": null}..end(); (2) the span opens on the correct parent; (3) level="ERROR" is set in the except branch; (4) langfuse.flush() is called before process exit.Post numeric or categorical scores to a trace using `langfuse.score()`, linking automated eval results, human feedback, or guardrail outputs to the exact run that produced them. You'll also revisit token-cost tracking from Module 2 and see how to aggregate costs across a session.
How to attach named quality scores and aggregate token costs to Langfuse traces and spans after a run completes.
Why this matters: Scores and cost rollups turn raw traces into actionable data — you can see which runs were expensive, which failed quality checks, and where in the pipeline the problem started.
Decision this forces: Score at trace level vs. span level — trace-level scores suit overall quality; span-level scores pinpoint which step caused a problem.
Answer: Langfuse needs the model name and the token counts (prompt + completion tokens). It looks up the per-token price from its model registry and multiplies.
. This module adds the next layer: attaching a — a named quality signal — to the or span that produced the output. Together, cost and score let you answer: "Was this run good, and what did it cost?"
in Langfuse is a named, typed signal — numeric or categorical. It posts to a specific or after the run completes. Sources include automated , human review, or guardrails.
The key design choice is where you attach it. Trace-level scores summarise the whole request — useful for overall quality or satisfaction. Span-level scores pin the signal to the exact step that produced output, isolating failures.
Langfuse distinguishes score sources in the UI. Scores posted via SDK with comment set to "automated" or "human" appear under separate tabs. This lets you compare automated evals against user ratings.
Your RAG pipeline answers a user question. After sending the response, an LLM judge checks if the answer is grounded in retrieved chunks — a faithfulness score from 0.0 to 1.0. The user clicks thumbs-up, producing a user_rating of 1.
faithfulness score goes on the . It evaluates that specific model output.user_rating goes on the . It reflects the user's overall experience.This pattern extends the running RAG trace from earlier modules with scores.
# Assumes: trace already created, generation span completed faithfulness_value = run_faithfulness_check(answer, retrieved_chunks) langfuse.score( trace_id=trace.id, name="faithfulness", value=faithfulness_value, # float 0.0–1.0 comment="automated", ) print(f"Scored trace {trace.id}: faithfulness={faithfulness_value}")
langfuse.score(...)trace_id=trace.idname="faithfulness"value=faithfulness_valuecomment="automated"This posts a numeric score named faithfulness to the whole trace. The comment field is how you tag the source so the Langfuse UI can separate automated scores from human ones.
The UI filters by the comment field. Setting comment="automated" tags this as a machine-generated score; a human-feedback score would use comment="human" (or a reviewer name). Without a consistent comment convention, both score types appear mixed in the same list.
# generation_span was created in Module 2/4 pattern user_rating = get_user_thumbs_up() # returns 1 or 0 langfuse.score( trace_id=trace.id, observation_id=generation_span.id, # pins to this span name="user_rating", value=user_rating, comment="human", ) print(f"Span {generation_span.id} rated: {user_rating}")
observation_id=generation_span.idcomment="human"Adding observation_id is the only change from Stage 1 — it moves the score from the trace root down to the specific . In the UI, this score appears inline with that span, not at the top-level summary.
It appears on the trace root. observation_id is what pins a score to a specific span; without it, Langfuse attaches the score to the trace as a whole, regardless of which span you intended.
# Fetch the full trace tree after all spans are flushed trace_data = langfuse.get_trace(trace.id) total_cost = sum( obs.usage.total_cost for obs in trace_data.observations if obs.type == "GENERATION" and obs.usage is not None ) print(f"Session cost: ${total_cost:.6f}")
langfuse.get_trace(trace.id)trace_data.observationsobs.type == "GENERATION"obs.usage.total_costEach inside the trace carries a usage.total_cost field that Langfuse auto-computes from token counts and model pricing. Summing across all GENERATION observations gives you the full session cost — useful for per-user budget tracking or cost-per-query dashboards.
If usage is None, that span contributes $0 to the sum — the guard skips it safely. This happens when the generation span was created without passing token counts (prompt_tokens, completion_tokens) or when the model name isn't in Langfuse's pricing registry. The changed line vs. Stage 1/2: the if obs.usage is not None guard is the critical addition — without it, accessing .total_cost on a None object raises an AttributeError at runtime.
langfuse.score() then exit before langfuse.flush() runs. The score never reaches the server. The trace appears in the UI with no scores."observation does not belong to trace". Verify that observation_id and trace_id come from the same run.total_cost stays 0.0. Register the custom model in Langfuse settings with its per-token price.langfuse.score() call is followed by langfuse.flush() before process exit.trace_id and observation_id are sourced from the same trace object — not hardcoded.obs.usage for each generation span. A None value means token counts were never attached.With scores and costs wired up, the next module — Debugging and Analyzing Traces End to End — shows how to filter the UI by score, model, latency, or error to pinpoint failures and compare prompt versions.
Navigate the Langfuse trace UI to filter by score, model, latency, or error; compare prompt versions side by side; and identify the root cause of a bad output by walking the span tree. You'll also learn the three most common instrumentation mistakes and how to verify your spans are correct before shipping.
How to navigate the Langfuse UI to filter traces, walk span trees, and fix the three most common instrumentation bugs.
Why this matters: This is where observability pays off — you turn raw trace data into a root-cause diagnosis and ship instrumentation you can actually trust.
Your pipeline just returned a wrong answer. The question is: was it retrieval, the model, or a tool that failed? The Langfuse trace UI answers this by letting you filter the trace list, then walk the tree inside one .
Filters narrow the list to the runs worth inspecting: score below a threshold, a specific model name, latency above a ceiling, or error status. Once you open a trace, the span tree shows every operation in causal order — retrieval before generation, tools nested inside the generation that called them.
Each span carries its own input, output, latency, and token count, so you can pinpoint the exact node where the answer went wrong without re-running anything.
A user reports that your RAG assistant cited a document that doesn't exist. You open Langfuse and apply two filters: faithfulness score < 0.5 and model = gpt-4o. That cuts 3 000 traces down to 11.
You open the worst-scoring trace and expand the span tree. The shows three chunks returned — but their similarity scores are all below 0.6, meaning the retriever surfaced weak context. The received that weak context and hallucinated a citation to fill the gap.
Root cause: retrieval, not the model. You raise the similarity threshold in your vector store and re-run the same 11 traces as a dataset to confirm the fix.
from langfuse import Langfuse lf = Langfuse() # Fetch traces where faithfulness score is below threshold traces = lf.get_traces( scores=[{"name": "faithfulness", "operator": "lt", "value": 0.5}], limit=50, ) for t in traces.data: print(t.id, t.latency, t.scores)
lf.get_traces(...)scores=[{"name": ..., "operator": "lt", "value": 0.5}]traces.dataThis fetches the 50 most recent traces whose faithfulness is below 0.5 — the same filter you'd click in the UI, but scriptable. Each item in traces.data carries the trace ID, latency, and attached scores so you can triage without opening the browser.
Latency is returned in milliseconds as an integer. So a 2-second call shows as roughly 2000. If you see 0, the trace was flushed before the root span closed — a sign of a missing flush() call.
# Continue from Stage 1 — inspect spans of the worst trace bad_trace_id = traces.data[0].id observations = lf.get_observations( trace_id=bad_trace_id, type="SPAN", # retrieval + tool spans ) for obs in observations.data: print(obs.name, obs.type, obs.start_time, obs.output[:80] if obs.output else "<no output>")
lf.get_observations(trace_id=..., type="SPAN")obs.output[:80]With the worst trace ID in hand, get_observations() returns every nested inside it, ordered by start time. Printing obs.name and obs.output lets you see exactly what each node received and returned — retrieval chunks, tool results, or model completions.
The span was opened but its output was never logged — the developer called span.end() without passing the output argument. This is the 'unlogged usage' bug: the span exists in the tree but carries no data, making it useless for debugging.
Most broken traces trace back to one of three mistakes. Knowing the symptom lets you fix it in seconds rather than re-reading your whole pipeline.
flush() — Langfuse batches events and sends them in the background. If your process exits before the batch drains, spans vanish silently. You see the trace in the UI with latency = 0 and no child spans. Fix: call langfuse.flush() before process exit or at the end of each request handler.trace_id or parent_observation_id appears as a detached root trace instead of a child. The symptom: you see two separate traces for what should be one request. Fix: pass the parent's id explicitly, especially across async boundaries.usage={"input": n, "output": m} shows $0 cost and blank token counts. Cost tracking silently breaks. Fix: always pass the token counts from the model response into the span's usage field.from langfuse import Langfuse lf = Langfuse() # Fetch traces that ended with an error AND have high latency traces = lf.get_traces( # TODO: add a filter for traces where status == "ERROR" tags=["production"], limit=20, ) for t in traces.data: print(t.id, t.status, t.latency)
status="ERROR"tags=["production"]This is a near-complete script that fetches production traces — your job is to add the error-status filter. The pattern mirrors Stage 1, but targets a different field.
Replace the TODO with:
status="ERROR",
Changed line: the status keyword argument filters by the trace's terminal status. Unlike the scores filter (which takes a list of predicates), status is a simple string match. The full call becomes:
lf.get_traces(
status="ERROR",
tags=["production"],
limit=20,
)
If you see an empty result, check that your traces actually set status='ERROR' when an exception is caught — unhandled exceptions that bypass trace.update() leave status as None, not ERROR.
When you use an AI assistant to generate Langfuse instrumentation, check these four things before shipping:
flush; if it's absent, spans will drop on process exit.span.end() with no arguments; those are silent data holes.trace_id is passed into every spawned coroutine, not captured by closure from a different request.usage dict is populated from the actual model response, not hardcoded to zero.Before looking at the build order below, reconstruct it from memory: what do you create first, what do you attach to it, and in what order do retrieval, tool, and score spans join the tree? Write the sequence out, then check it against the list.
Apply what you learned to Observability with Langfuse.
You shut down your application and notice that several traces never appear in the Langfuse UI, even though your code ran without errors. What is the most likely cause?
Langfuse batches spans in memory and sends them asynchronously. If the process exits before flush() is called, those buffered spans are discarded — they never reach the server. Environment variables are the recommended credential method, not a problem. User ID and session ID are valid trace metadata. Span creation order affects the call tree but does not cause data loss on its own.
You are building an agent that calls a fine-tuned GPT model. You create a generation span and leave the usage field empty, expecting Langfuse to fill in the cost automatically. What will happen?
Langfuse's auto-cost feature relies on a built-in model price table that covers well-known public models. Fine-tuned or custom model IDs are not in that table, so no cost can be computed — the field stays empty or zero. Langfuse does not fall back to the base model's price, and usage is optional (not required) for saving a span.
Look at this instrumentation snippet:
retrieval_span = trace.span(name="retrieve")
generation_span = trace.span(name="generate")
What is wrong with it for a RAG pipeline where retrieval feeds the model?
In a RAG pipeline the generation step consumes the retrieval output, so the generation span should be nested inside — or at least ordered after — the retrieval span with a clear parent-child relationship. Creating both as direct children of the root trace makes them siblings, which hides the dependency in the timeline. Multiple spans on one trace are perfectly valid, and Langfuse has no reserved span-name keywords.
A trace shows a correct final answer but a low faithfulness score. You want to know whether the problem originated in retrieval or in the model's generation step. Which approach in Langfuse lets you pinpoint this most directly?
Span-level scores let you attach a faithfulness score directly to the generation span; you can then open the span tree and compare that score against the documents logged in the retrieval span to see whether bad context or bad generation is to blame. A trace-level score tells you overall quality but does not point to the specific step. Deleting spans destroys evidence. Switching to chunk IDs reduces payload size but makes debugging harder, not easier.
Name the three most common Langfuse instrumentation bugs covered in the debugging module, and for each one state what symptom it produces in the UI.
These three bugs are the canonical failure modes taught in the final module. Missing flush causes data loss. Wrong parent ID corrupts the span hierarchy so you cannot walk the tree correctly. Unlogged usage silently breaks cost tracking — the span saves fine but financial and token data is missing.