Trace prompts, retrieval, tools, costs, and scores for LLM apps.
Map a single user request through Langfuse's trace → span → event hierarchy using a document-Q&A app as the running example. You'll see why this three-level structure is the only way to attribute failures to their exact cause.
Explains how Langfuse structures a user request into traces, spans, and events — and how to initialize the client with stable metadata.
Why this matters: Getting this hierarchy right is the foundation for every other observability feature: without correct traces you can't attribute failures, measure latency, or track costs.
Your document Q&A app returns a wrong answer. The LLM call looks fine in isolation — so where did it break?
A flat log shows that something failed, not which step caused it or timing. Langfuse uses a three-level hierarchy. A wraps the entire request. capture each operation inside it. record discrete moments (cache hit, guardrail trigger) with no duration.
This nesting enables attribution. When retrieval returns stale chunks, you see it on the retrieval span — not buried in mixed output.
A user asks: "What does the contract say about termination?" Here's how that request maps to Langfuse's data model.
answer_question — one per user question. Carries session_id, release_sha, model_name for deployment filtering.retrieve_docs — child of trace. Records embedded query, chunk count, and lookup latency.generate_answer — sibling of retrieve_docs. A span capturing prompt, reply, and token usage.guardrail_pass — nested inside generate_answer. Marks output cleared the content filter.If the answer is wrong, open the trace. Check retrieve_docs output. See if the right chunk was retrieved before blaming the model.
from langfuse import Langfuse client = Langfuse() # no keys set in environment trace = client.trace( name=f"answer_question_{user_id}_{question[:30]}", # ⚠️ ) print(trace.id)
Langfuse()client.trace(name=...)f"answer_question_{user_id}_{question[:30]}"This is the obvious first attempt — and it has two problems that will hurt you in production. Predict what goes wrong before reading the answer.
Problem 1 — AuthenticationError (or silent no-op depending on SDK version): Langfuse() with no keys in the environment raises an error or drops data silently. You must set LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY (plus LANGFUSE_HOST for self-hosted). Problem 2 — high-cardinality trace name: 'answer_question_u42_What does the cont' is unique per question. The Langfuse dashboard groups traces by name, so you get thousands of one-row groups instead of one filterable 'answer_question' group. Dashboards become unusable.
import os from langfuse import Langfuse client = Langfuse( public_key=os.environ["LANGFUSE_PUBLIC_KEY"], secret_key=os.environ["LANGFUSE_SECRET_KEY"], ) trace = client.trace( name="answer_question", # stable — groups all Q&A requests session_id=session_id, # ties turns in a conversation together release=os.environ["GIT_SHA"], # links to the deployed code version metadata={"model": "gpt-4o"}, # queryable; not in the trace name )
os.environ["LANGFUSE_PUBLIC_KEY"]name="answer_question"session_id=session_idrelease=os.environ["GIT_SHA"]metadata={"model": "gpt-4o"}The fix: keep the trace name a static label and push variable data into dedicated metadata fields. Now every Q&A request rolls up under one answer_question group in the dashboard, and you can still filter by session, release, or model.
trace.id is a UUID (e.g. '3f7a1c2e-…') generated by the SDK. Langfuse owns the ID so it can guarantee uniqueness across distributed services — if you generated it yourself you'd risk collisions or reuse across retries.
retrieve_span = trace.span( name="retrieve_docs", input={"query": user_question}, ) chunks = vector_store.search(user_question) retrieve_span.end(output={"chunk_count": len(chunks)}) gen_span = trace.generation( name="generate_answer", input={"prompt": build_prompt(chunks, user_question)}, model="gpt-4o", ) answer = llm.complete(gen_span.input["prompt"]) gen_span.end(output={"answer": answer})
trace.span(name=..., input=...)retrieve_span.end(output=...)trace.generation(name=..., model=...)Spans are opened on the trace (or on a parent span), do real work, then closed with .end(output=...). The span is a typed subclass that unlocks token-count and cost tracking in the next module.
The span will appear as 'still open' or show a null/missing duration in the latency waterfall. Langfuse can't compute latency without an end timestamp, so the span shows up as incomplete — a common silent data-quality bug.
Follow a single Q&A request through the Langfuse data model. Each level adds a finer grain of attribution.
# Continuing from Stage 3 — gen_span is already open # The guardrail runs after the LLM responds guardrail_result = run_guardrail(answer) # TODO: record a point-in-time event on gen_span # name: "guardrail_check" # metadata: {"passed": guardrail_result.passed} # Hint: events have no duration — use .event(), not .span() gen_span.end(output={"answer": answer})
gen_span.event(name=..., metadata=...)metadata={"passed": ...}Stop — attempt the TODO before revealing. The missing line is the crux of the event concept: events are point-in-time, not timed operations, so the method call is different from a span.
gen_span.event(
name="guardrail_check",
metadata={"passed": guardrail_result.passed},
)
Changed lines vs Stage 3: one new call — .event() instead of .span(). No .end() because events are instantaneous; Langfuse records a single timestamp, not a duration. This is the key distinction: spans bracket a duration, events mark a moment.
Three failure patterns appear when teams first instrument a Q&A app.
name creates a unique group per request. The dashboard shows thousands of one-row groups. Fix: static name, variable data in metadata or session_id..end() leaves a span with no duration. The shows it as perpetually open. No error is raised; data silently looks wrong.LANGFUSE_PUBLIC_KEY or LANGFUSE_SECRET_KEY are absent, some SDKs drop all data silently. You see zero traces with no exception in logs. Always assert env vars are set before the app starts..span() has a matching .end(), (3) session_id and release are passed at trace creation, (4) credentials are read from environment, not hardcoded.With the trace skeleton solid, the next module instruments the LLM call — capturing prompt, token counts, and latency as a span.
Instrument a real chat-completion call in the Q&A app, capturing the full prompt, model response, token counts, and latency as a generation span. You'll also see how to attach a prompt version ID so dashboard filters work correctly.
How to instrument an LLM call as a generation span in Langfuse, capturing prompt, completion, token usage, and prompt version.
Why this matters: Without a properly populated generation span, your Langfuse dashboard shows $0.00 cost and blank token counts — making it impossible to debug or optimise your Q&A app.
Decision this forces: Whether to use the Langfuse SDK directly or a framework integration for generation spans.
A is the top-level envelope for one user request. are timed operations nested inside it: retrieval, model calls, tool use.
Module 1 showed the hierarchy shape. This module zooms into the model-call span — specifically the span. It shows which fields to populate so cost roll-up and dashboard filters work.
Your Q&A app calls an LLM, but the dashboard shows $0.00 cost and no token counts. What did you forget to record?
A span records one LLM call: prompt in, completion out, model name, latency, and .
Four fields are required for : model (exact model ID), input (prompt), output (completion), and usage (token counts).
Without model, Langfuse cannot look up . Without usage, it has no tokens to multiply. Both gaps leave cost at $0.00.
Optionally attach a ID from Langfuse's prompt registry. This lets you filter dashboard views by version and compare quality across deploys.
model — exact model ID (e.g. gpt-4o-mini); drives pricing lookupinput — the full prompt sent to the modeloutput — the completion text returnedusage — { prompt_tokens, completion_tokens, total_tokens }prompt_version_id — optional; links to the in the registry# Q&A app — naive: trace exists, but no generation span trace = langfuse.trace(name="qa_request", input={"question": question}) response = llm_client.chat(model="gpt-4o-mini", messages=messages) answer = response.choices[0].message.content trace.update(output={"answer": answer}) trace.flush()
langfuse.trace(...)trace.update(output=...)trace.flush()This creates a trace and records the final answer, but never opens a generation span inside it.
The dashboard will show the trace with input and output — but cost is $0.00, token counts are blank, and latency is unattributed to any span.
Cost: $0.00, token counts: blank. No generation span means no model field and no usage dict — Langfuse has nothing to look up in its pricing table. The trace exists but is effectively a black box.
trace = langfuse.trace(name="qa_request", input={"question": question}) gen = trace.generation( name="answer_llm_call", model="gpt-4o-mini", input=messages, ) response = llm_client.chat(model="gpt-4o-mini", messages=messages) answer = response.choices[0].message.content usage = response.usage # {prompt_tokens, completion_tokens, total_tokens} gen.end(output=answer, usage=usage) trace.update(output={"answer": answer}) trace.flush()
trace.generation(name=..., model=..., input=...)gen.end(output=..., usage=...)response.usageOpening the generation span before the call and ending it after captures latency automatically.
Passing model and usage to gen.end() is what unlocks cost roll-up — Langfuse multiplies token counts by the model's price per token.
1) Prompt token count, 2) completion token count (and total), 3) calculated cost in USD — because model + usage are now present for the pricing lookup.
# Fetch the versioned prompt from Langfuse registry prompt_obj = langfuse.get_prompt("qa_system_prompt", version=3) messages = build_messages(prompt_obj.prompt, question=question) trace = langfuse.trace(name="qa_request", input={"question": question}) gen = trace.generation( name="answer_llm_call", model="gpt-4o-mini", input=messages, prompt=prompt_obj, # links prompt version ID to this span ) response = llm_client.chat(model="gpt-4o-mini", messages=messages) gen.end(output=response.choices[0].message.content, usage=response.usage) trace.flush()
langfuse.get_prompt(name, version=...)prompt=prompt_objPassing prompt=prompt_obj writes the ID onto the generation span, enabling dashboard filters like "show all traces using v3 of qa_system_prompt".
This is the delta from Stage 2: one extra langfuse.get_prompt() call and one extra keyword argument — everything else stays the same.
Line changed: langfuse.get_prompt("qa_system_prompt", version=4)
Everything else is identical. The prompt_obj now carries version 4's ID, so gen.end() stamps the span with v4 — no other edits needed. This is the key mechanic: the version ID travels with the object, not hardcoded elsewhere.
Your Q&A app now records every LLM call as a generation span with cost, latency, and prompt version.
But the trace waterfall shows a blank gap: the retrieval step — the vector search that fetches document chunks before the model call.
You can see the model was slow, but can't tell if latency came from retrieval or the LLM itself.
The next module wraps the vector-search step in a . It records the query, top-k chunks, and similarity scores — so you can attribute latency and quality issues to their exact source.
| Option | Token usage auto-captured | Prompt version linking | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Langfuse SDK (direct) | You pass usage dict manually from the API response | Full control — set prompt_version_id explicitly on the span | When you call the LLM API directly (no framework), or need fine-grained control over every span field. | No added latency; runs in-process | Low — a few extra lines per call |
| Framework integration (e.g. LangChain callback) | Captured automatically from the framework's callback events | Requires metadata injection; not always first-class | When you already use a framework that has a Langfuse integration and want minimal instrumentation code. | No added latency; callback-based | Low setup, but less control over individual span fields |
Three failure patterns account for most broken generation spans in production:
"gpt4o" instead of "gpt-4o". Langfuse can't match it to pricing — cost stays $0.00 silently. Fix: copy the model ID from the API response, not memory.gen.end(), the span stays open forever. Fix: wrap the call in try/finally and call gen.end() in finally, passing the error as status_message.trace.flush() before process exit.gen.end() is in a finally block, (3) usage comes from the response — not hardcoded — and (4) flush() is called before exit.Wrap the vector-search step in the Q&A app as a retrieval span, recording the query, top-k chunks, scores, and corpus version. You'll see how this data feeds both cost analysis and retrieval-quality scores in later modules.
How to wrap the vector-search step in a retrieval span that records the query, top-k chunks, similarity scores, and corpus version in Langfuse.
Why this matters: Without a retrieval span, you can't tell whether a bad answer came from poor retrieval or a bad generation — this module gives you the data to separate the two.
A records the full prompt, the completion, and token counts. All link to a for dashboard filtering. That structure is your foundation here.
The Q&A app now has its LLM call instrumented. But the retrieval step — the vector search feeding the model — is still a black box. This module wraps it in a so you see exactly what evidence the model received.
A is a child wrapping one vector-search call. It captures query text, chunks returned, similarity scores, and a tag.
It sits as a sibling to the generation span inside the same . This placement lets Langfuse show retrieval and generation latency side by side in the .
The corpus version tag is critical: without it, you can't tell if answer quality dropped because the index changed or the query changed.
Unlike a , a retrieval span has no token counts or model name. Required fields: query, chunks, scores, corpus version.
A user asks: "What is the refund policy for enterprise plans?" The app runs a vector search and gets three chunks back. Cosine similarity scores: 0.91, 0.87, 0.74.
Without a retrieval span, Langfuse shows the generation span receiving context. But you can't see which chunks were chosen, their scores, or which index version was queried.
With a retrieval span wrapping the search call, Langfuse shows: query = "refund policy enterprise plans", chunks = [{id, text, score}, …], corpus_version = "docs-v4". When a score drops to 0.55 next week, you can check if the index was re-embedded that day.
def retrieve(query: str, k: int = 3) -> list[dict]: query_vec = embed(query) results = vector_index.search(query_vec, top_k=k) # returns [{"id": ..., "text": ..., "score": ...}, ...] return results chunks = retrieve("refund policy enterprise plans") response = generate(chunks, query)
embed(query)vector_index.search(query_vec, top_k=k)generate(chunks, query)This is the baseline: retrieval runs, but nothing is recorded in Langfuse.
The generation span sees the chunks, but the Langfuse trace has no record of the query, the scores, or the index version.
The trace shows only the generation span. You cannot see: the query string sent to the vector index, the similarity scores of the returned chunks, which chunks were actually retrieved, or the corpus version. If the model gives a wrong answer, you have no way to tell whether retrieval or generation caused it.
def retrieve_traced(query: str, trace, k: int = 3) -> list[dict]: span = trace.span( name="retrieval", input={"query": query, "top_k": k}, metadata={"corpus_version": "docs-v4"}, ) results = vector_index.search(embed(query), top_k=k) span.end( output={"chunks": results}, # full text OR ids — your call level="DEFAULT", ) return results
trace.span(name=..., input=..., metadata=...)metadata={"corpus_version": "docs-v4"}span.end(output=..., level=...)Each call to trace.span() creates a child under the current , recording the query and on open, and the chunks on close.
The output field is where the full-chunks-vs-IDs decision lands: log results for full text, or [r["id"] for r in results] for IDs only.
You'll now see: (1) query = "refund policy enterprise plans", (2) top_k = 3, (3) corpus_version = "docs-v4", (4) the chunks array with text and scores, and (5) the retrieval span's own latency — separate from the generation span's latency in the waterfall.
def retrieve_traced_v2(query: str, trace, k: int = 5) -> list[dict]: span = trace.span( name="retrieval", input={"query": query, "top_k": k}, metadata={"corpus_version": "docs-v5"}, ) results = vector_index.search(embed(query), top_k=k) if not results: span.end(level="ERROR", status_message="No chunks returned") return [] # TODO: close the span, logging only chunk IDs (not full text) # Hint 1: use span.end(output=..., level=...) # Hint 2: extract just the "id" field from each result dict return results
[r["id"] for r in results]level="ERROR"status_message="No chunks returned"Stop — attempt the TODO before revealing the answer. The scenario has changed: k is now 5 and you must log IDs only to respect a data-privacy constraint.
The error path is already handled; your job is the success path — the one line that closes the span with the right output.
span.end(output={"chunk_ids": [r["id"] for r in results]}, level="DEFAULT")
What changed and why:
Click a span type to see which fields it owns. Points that cluster together share the same observability concern.
trace.span() and span.end(), the span stays open. Langfuse shows it as dangling with no output or latency. Fix: wrap the body in try/finally. Call span.end(level="ERROR") in the finally block.span.end() on every code path, including exceptions?corpus_version set from a variable or config — not hardcoded as a stale string?trace_id matches the parent request.Once your retrieval span is solid, the next module adds child spans for tool calls. A web-search tool and a calculator. Now you can trace every action the agent takes, not just retrieval.
Instrument two tool calls in the Q&A app (a web-search tool and a calculator) as child spans, capturing inputs, outputs, status codes, and errors. You'll practice the completion pattern: a partially instrumented tool is given and you add the error-capture branch.
Instrument web-search and calculator tool calls as child spans in Langfuse, capturing inputs, outputs, and errors.
Why this matters: Tool calls are the most common silent failure point in agentic Q&A apps — without spans here, you're debugging blind.
Decision this forces: Whether to instrument tools at the framework level (auto) or manually wrap each call.
Answer: you recorded the query text, top-k chunks with scores, and corpus version. You nested the span under the correct . This lets Langfuse draw the causal chain: user request → retrieval → generation. That parent-child rule applies to every tool call you'll instrument now.
Your Q&A app calls two tools — web-search and calculator. Right now those calls are invisible in the . When a tool fails silently, the trace shows a gap between retrieval and generation. You have no idea which tool caused it or why.
A wraps one tool invocation. It records three things: the input (arguments the agent passed), the output (raw result), and the (OK or ERROR). Errors must be recorded explicitly — the SDK does not catch exceptions for you.
The span must be a child of the that triggered the tool call. Usually that's the agent reasoning span. Without the correct parent, Langfuse places the tool span at the wrong depth. Causal order is lost.
def web_search(query: str, parent_span) -> str: span = parent_span.span( name="tool:web_search", input={"query": query}, ) try: result = search_api.call(query) # your real search call span.end(output={"result": result}, status="OK") return result except Exception as e: span.end(output=None, status="ERROR", status_message=str(e)) raise
parent_span.span(...)input={"query": query}span.end(output=..., status=...)status_message=str(e)This wraps the web-search tool as a child of whatever span called it. The finally pattern is replaced here by explicit try/except branches so each path sets a distinct — OK or ERROR — before the span closes.
The except branch fires: span.end() is called with status='ERROR' and status_message set to the string representation of the TimeoutError (e.g. 'Request timed out after 5s'). The span closes immediately, so no dangling span. The exception is then re-raised so the caller still sees it.
def calculator(expression: str, parent_span) -> float: span = parent_span.span( name="tool:calculator", input={"expression": expression}, ) try: result = eval_math(expression) # safe math evaluator span.end(output={"result": result}, status="OK") return result except Exception as e: # TODO: close the span with the correct status and message raise
eval_math(expression)span.end(output=None, status='ERROR', status_message=str(e))The happy path is complete. Your job is to fill in the except branch so a failed calculation appears as a in Langfuse rather than a silent gap. The web-search stage above is your model.
Replace the TODO with:
span.end(output=None, status='ERROR', status_message=str(e))
Changed line vs Stage 1: identical pattern — the variation is that this is a calculator, so the error might be a ZeroDivisionError or a parse error from eval_math.
If you write output=str(e): the span closes with status='OK' (the default when status is omitted) and the error text sits in the output field. Dashboard filters for ERROR spans return zero hits, hiding the failure entirely — the silent-gap failure mode from the notes block.
Three failure patterns show up repeatedly when instrumenting tool calls:
span.end() in a finally block, the span stays open indefinitely. In the Langfuse UI it appears as a dangling span with no end time — easy to miss on a busy trace.output field instead of setting status='ERROR' means the span looks green in dashboards. Filters for failed spans return zero results even though errors occurred.Before trusting AI-generated instrumentation code for tool spans, check these four things specifically:
Drag to see how each additional field you record on a tool span changes what you can diagnose in Langfuse.
Analyze the complete Q&A trace — retrieval + two tool calls + generation — to see how Langfuse aggregates token spend and wall-clock time, and where to set model pricing so roll-up is accurate. You'll identify the two most common reasons cost data goes missing.
Shows how Langfuse aggregates token costs and wall-clock time across a full Q&A trace, and how to configure model pricing so the roll-up is accurate.
Why this matters: Missing cost data is one of the most common silent failures in LLM observability — this module gives you the exact two causes and the fix so your dashboards reflect real spend.
Decision this forces: Whether to rely on Langfuse's built-in model pricing table or override it with custom pricing per deployment.
Answer from memory, then check: you attached the inputs, outputs, and a (success / error) to each . Those same spans are now the raw material for cost and latency analysis — which is exactly what this module adds.
Langfuse builds a by summing token costs across every span in the . Retrieval, tool calls, and the final answer generation all contribute.
The shows each 's wall-clock start and end as a horizontal bar. You can see which step dominates total response time.
For cost to appear, two things must be true. The span must carry a model name that Langfuse can look up in its pricing table. It must also report (input + output counts).
If either field is missing, Langfuse shows a dash for that span's cost. The trace total is silently understated.
Drag to see how the dominant span shifts as generation latency grows relative to retrieval and tool calls.
Langfuse ships a built-in table keyed on model name strings, such as "gpt-4o" and "claude-3-5-sonnet". When a generation span's model name matches an entry, Langfuse multiplies token counts by the stored per-token rate. It writes the result to the span.
For fine-tuned or self-hosted models, the built-in table has no entry. You register a custom price via the Langfuse UI (Settings → Models) or the API. Map your model name string to input and output cost per million tokens.
The model name you log in the span must match exactly. It is case-sensitive. It must match what is registered in the pricing table. A mismatch produces a dash in the cost column with no warning.
# Stage 1 — broken: model name missing, token fields absent gen_span = trace.generation( name="qa_generation", input=prompt, output=response_text, # model= ← omitted: cost column will show a dash ) # Stage 2 — fixed: model + usage attached so roll-up works gen_span = trace.generation( name="qa_generation", input=prompt, output=response_text, model="gpt-4o", # must match pricing table exactly usage={"input": usage.input_tokens, # from provider response "output": usage.output_tokens}, )
trace.generation(...)model="gpt-4o"usage={"input": ..., "output": ...}Stage 1 shows the broken pattern: a generation span with no model name and no usage — the exact two failure modes from the note above.
Stage 2 adds both fields. Langfuse now multiplies usage.input_tokens and usage.output_tokens by the gpt-4o per-token rate and writes the result to the span, which feeds the trace-level .
A dash (—). The span has no model name, so Langfuse cannot look up a price. Even if a model were set, the missing usage fields mean there are no token counts to multiply. Both conditions must be satisfied for a cost to appear.
# The web-search tool now calls an LLM to summarise results. # Your job: make its generation span contribute to the cost roll-up. web_search_span = trace.span(name="web_search", parent_observation_id=retrieval_span.id) summary, usage = llm_summarise(search_results, model_id="gpt-4o-mini") web_search_span.generation( name="search_summary", input=search_results, output=summary, model=TODO, # ← which string goes here, and why does case matter? usage={"input": TODO, # ← fill from usage object "output": TODO}, )
web_search_span.generation(...)parent_observation_id=retrieval_span.idTODOThis is a variation of the Stage 2 pattern: the web-search tool now has its own generation span nested under the tool span.
Stop — attempt the three TODOs before revealing. Hint 1: the model ID comes from the variable already in scope. Hint 2: the usage object has the same shape as in Stage 2.
Changed lines:
model="gpt-4o-mini" # from model_id variable — exact case required
usage={"input": usage.input_tokens,
"output": usage.output_tokens}
If you write "GPT-4o-mini" (capital G), Langfuse finds no matching entry in the pricing table and shows a dash for this span's cost — the trace total is understated with no warning. The model name comparison is case-sensitive.
| Option | Accuracy for your contract rate | Maintenance burden | Works for private/fine-tuned models | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Built-in pricing table | List price only; may differ from your negotiated rate | Langfuse maintains it; you do nothing | No — unknown model names get a dash | You use standard public models (GPT-4o, Claude 3.x, Gemini) at list prices and don't need cent-level accuracy. | Free | None — zero config |
| Custom pricing override | Exact — you set the per-token rate | You own updates when rates change | Yes — register any model name string | You have negotiated rates, use fine-tuned models, or run self-hosted inference where list prices don't apply. | Free (Langfuse feature) | Low — one-time UI or API registration per model |
These two failures are responsible for most "$0.00" traces in production. Both are silent. No error is raised. The span still appears. The cost column just shows a dash.
model field is None, empty, or differs in case from the pricing table entry. For example, "GPT-4o" vs "gpt-4o". Fix: log the exact model ID string returned by the provider, not a human-readable alias.usage.input_tokens and usage.output_tokens are both zero or absent. This is common when you wrap a streaming response and forget to accumulate the usage chunk. Fix: always read the final usage object from the stream's last chunk, and pass it explicitly to the generation span.Quick check: open the span in the Langfuse UI. Look at the "Model" and "Usage" fields directly. If either is blank, you've found the root cause.
Add three score types to the Q&A trace — a human thumbs-up/down, a heuristic faithfulness check, and an LLM-judge score — and wire them to the correct trace ID. You'll solo-instrument a new score type and verify it appears in the Langfuse scores dashboard.
Attach human, heuristic, and LLM-judge scores to a Q&A trace and use them to filter for low-quality responses in Langfuse.
Why this matters: Scores turn raw traces into a quality feedback loop — without them you can observe failures but can't systematically find or rank them.
Decision this forces: Which scoring method to run in the hot path (synchronous) vs. asynchronously after the response is returned.
Every span you instrumented in modules 1–5 — retrieval, tool calls, generation — hangs off one , identified by a single . That ID anchors everything. It powers cost roll-up, latency waterfall, and now .
Scores bridge raw traces and actionable quality signals. Without them, you have logs. With them, you have a feedback loop.
Langfuse accepts three score types, each attached to the same : , a deterministic heuristic, and an .
The key design question is which of these runs in the hot path (blocking the response) and which runs asynchronously after the answer is returned.
import langfuse client = langfuse.Langfuse() # Assume trace_id was captured when the Q&A trace was created def score_citation_presence(trace_id: str, answer: str) -> None: has_citation = "[" in answer and "]" in answer client.score( trace_id=trace_id, name="citation_present", value=1.0 if has_citation else 0.0, comment="Heuristic: bracketed citation detected", )
client.score(...)name="citation_present"value=1.0 if has_citation else 0.0comment=...This attaches a binary heuristic score to the Q&A trace immediately after the answer is generated. The score is keyed to trace_id — if that ID doesn't exist in Langfuse, the score is silently dropped (more on this below).
value=0.0 — the string contains no '[' or ']', so has_citation is False and the score is 0.0.
import json FAITHFULNESS_PROMPT = """ Given the context and the answer, rate faithfulness 0.0–1.0. Context: {context} Answer: {answer} Return JSON: {{"score": <float>, "reason": <str>}} """ def llm_judge_faithfulness(trace_id, context, answer, llm_call): raw = llm_call(FAITHFULNESS_PROMPT.format(context=context, answer=answer)) result = json.loads(raw) # e.g. {"score": 0.85, "reason": "All claims match the context."} client.score(trace_id=trace_id, name="faithfulness", value=result["score"], comment=result["reason"])
FAITHFULNESS_PROMPT.format(...)json.loads(raw)name="faithfulness"This stage adds an LLM-judge score to the same trace — call it asynchronously after the response is returned to avoid blocking the user. The judge reads the retrieved context and the answer, then returns a structured score you post to Langfuse.
value=0.85 (a float), with comment="All claims match the context." — json.loads() parses the string into a dict before the score call.
def record_user_feedback(trace_id: str, thumbs_up: bool) -> None: """Called when the user clicks 👍 or 👎 in the Q&A UI.""" client.score( trace_id=trace_id, name="user_thumbs", # TODO: set value to 1.0 for thumbs_up, 0.0 otherwise value=___, data_type="BOOLEAN", comment="Human feedback from Q&A UI", )
data_type="BOOLEAN"1.0 if thumbs_up else 0.0Stop — attempt the TODO before revealing the answer. This is a small variation of Stage 1: the score name and data_type differ, and the value comes from a boolean argument instead of a string check.
value=1.0 if thumbs_up else 0.0
Changed lines vs. Stage 1:
| Option | Latency impact | Signal richness | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Heuristic (sync) | Adds <1 ms; safe in hot path | Binary or simple numeric only | When the check is deterministic and sub-millisecond — e.g. source-citation presence, length guard, regex match. | Negligible | Low |
| LLM Judge (async) | Adds 1–5 s if sync; zero if async | Nuanced rubric-based scores | When you need faithfulness, helpfulness, or tone scores that require reading the full answer — run after the response is returned. | Per-call LLM cost (~$0.001–0.01 per trace) | Medium |
| Human Feedback (async) | Always async — user-initiated | Highest trust, lowest volume | When you need ground-truth labels for model improvement or when automated scores need periodic calibration. | Human time; no API cost | Low (UI) / High (ops) |
client.score() returns 200 but nothing appears in the dashboard. Symptom: the scores tab shows zero entries for a trace you know ran."Faithfulness" in one release and "faithfulness" in the next creates two separate metrics in the dashboard. Your filter returns half the traces you expect.trace_id is captured before any async handoff. Do not reconstruct it later from a log.json.loads() is wrapped in a try/except. A malformed judge response will crash the scorer and drop the score silently.With all three score types wired and verified, you're ready for the capstone. Instrument a fresh Q&A variant from scratch. Add your own score dimension. Use the scores dashboard to surface the weakest responses. That closes the full observability loop.
Before looking at the summary: reconstruct from memory the six span types you added to the Q&A trace in order, and name the one field each span type needs for cost roll-up or quality scoring to work. Then check your answer against the build order below.
Apply what you learned to Observability with Langfuse.
You name every trace after the exact user query — for example, "What is the capital of France?" or "Summarize this 500-word article for me."
What problem does this cause in the Langfuse dashboard?
Langfuse groups and aggregates traces by name. When every trace has a unique, high-cardinality name (the raw user query), each trace stands alone — you cannot compute average latency or cost for a logical operation like "summarize" because no two traces share a name. The SDK does not enforce a length limit on trace names, token counts are unrelated to the name field, and the session ID is a separate metadata field that the trace name cannot overwrite.
You are building a generation span for an LLM call. Which combination of fields is the minimum required for Langfuse to compute a dollar cost for that span automatically?
Langfuse's cost roll-up multiplies token counts by the per-token price it looks up using the model name. Without the model name it cannot find a price; without both prompt and completion token counts there is nothing to multiply. The output text and session ID are useful for other purposes but play no role in cost calculation. The trace ID is always present implicitly — it is not a field you supply on the generation span itself to unlock cost.
A teammate adds a retrieval span but omits the corpus-version tag. The team later updates the vector index. What specific debugging problem does the missing tag create?
The corpus-version tag is the only field that ties a retrieval span to a specific state of the index. Without it, a quality regression visible in scores or chunk similarity could have occurred before or after the index update — you have no way to correlate the two. Langfuse does not enforce corpus-version as a required field, so the span is stored and displayed normally. Similarity scores are stored independently of the corpus-version tag. Parent-span nesting is a structural concern unrelated to metadata tags.
Consider this instrumentation code for a tool call:
span = trace.span(name="web_search", input=query)
result = search_api(query)
span.end(output=result)
The search_api call raises an exception. What is wrong with this pattern, and what should you do instead?
If search_api raises before span.end() is reached, the span is never closed and no error information is recorded — the trace shows a gap rather than a failed span. The fix is to wrap the call in a try/except, catch the exception, record it in the span's error field, then end the span in a finally block. Span names do not need to match function names. Tool spans support both input and output fields. Langfuse accepts Python dicts as input; explicit JSON serialization is not required.
You want to attach an LLM-judge score to every trace in production. A colleague says to run the judge synchronously so the score is always present before the trace closes. You disagree. Explain the tradeoff and state which path you would choose for a latency-sensitive user-facing feature, and why.
LLM-judge scoring is the highest-quality but also the highest-latency and highest-cost scoring method. Placing it in the hot path (synchronous) means every user waits for a second LLM call to complete before receiving their answer. Asynchronous scoring decouples evaluation from serving: the trace ID is used to attach the score after the fact. The risk of async scoring is that a score referencing a deleted or expired trace is silently dropped — but that is a data-retention concern, not a latency one. Heuristic scoring is the right candidate for synchronous use because it is fast and cheap.