Load docs, index chunks, query with sources, and evaluate retrieval.
The first LlamaIndex build target is a trustworthy query engine over a tiny corpus.
Learners write the expected behavior for query, source nodes, and answer synthesis.
Why this matters: This prevents agent workflows from masking weak retrieval quality.
Problem anchor: a course assistant must answer from curated notes and show which LlamaIndex source nodes supported the response. The first goal is not a chat agent; it is a query engine that retrieves, cites, and refuses correctly.
The contract names allowed documents, source metadata, similarity top-k, answer style, and refusal behavior when no node supports the question.
The LlamaIndex prototype accepts a question, retrieves evidence from local Markdown notes, returns an answer with source titles, and logs retrieved chunks or nodes. It passes when four factual questions cite the expected source and the negative question refuses.
This artifact is small enough for a reviewer to inspect in one sitting, but complete enough to catch the failures that matter: missing evidence, noisy retrieval, unsupported claims, and silent fallback.
Readers load data; node parsers decide the evidence units the index can retrieve.
Learners connect LlamaIndex nodes to chunking and provenance.
Why this matters: Poor node parsing leads to vague source nodes and unsupported synthesis.
A LlamaIndex reader loads source material, but the node parser decides what the retriever will later select. Nodes should preserve heading path, source title, URL or path, freshness date, and permission scope.
Parsing should respect the source: headings for Markdown, table context for tabular data, and code boundaries for developer docs.
The first prototype returns citations like “Document 2,” which makes review painful. After ingestion stores human-readable titles and heading paths, the same answer becomes auditable.
No model changed; the app simply preserved provenance.
LlamaIndex retrieval choices determine which nodes reach answer synthesis.
Learners tune retrievers, metadata filters, and postprocessors against eval questions.
Why this matters: Default settings can hide expected evidence below the synthesis cutoff.
A VectorStoreIndex plus query engine can feel automatic, but the important decisions remain explicit: which index, which retriever, which filters, how many nodes, and which postprocessors or rerankers.
The source_nodes returned by a response are the main debugging artifact. They should show whether the expected evidence reached synthesis.
| Option | Surface | Symptom | Fix | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Retrieval | Wrong or missing evidence. | Expected source absent from top-k. | Chunking, filters, hybrid search, reranking. | Inspect before prompt tuning. | Medium | Medium |
| Generation | Evidence present but answer misreads it. | Unsupported synthesis or overbroad claim. | Prompt, context packing, answer constraints. | Inspect with fixed evidence. | Low | Medium |
| Validation | Answer lacks citations or schema. | UI cannot trust or render output. | Structured output, citation checks, refusal gate. | Use before returning to users. | Low | Medium |
A useful LlamaIndex demo returns an answer and the source nodes behind it.
Learners code the smallest index-to-query path with inspection.
Why this matters: Source-node inspection distinguishes retrieval failures from synthesis failures.
The LlamaIndex response object can expose source_nodes. Use them. They are the bridge between retrieval evaluation and answer review.
Spaced recall: from faithfulness evaluation, a citation is only useful when the cited node supports the generated claim. Printing source_nodes makes that review possible.
from llama_index.core import Document, VectorStoreIndex DOCS = [ Document(text="Reranking reorders retrieved candidates before generation.", metadata={"source": "retrieval.md"}), Document(text="Chunking preserves answer evidence with headings and metadata.", metadata={"source": "chunking.md"}), ] index = VectorStoreIndex.from_documents(DOCS) query_engine = index.as_query_engine(similarity_top_k=2) response = query_engine.query("What does reranking do before generation?") print(response) for node in response.source_nodes: print("TRACE", node.metadata, node.score, node.text[:80])
retrieval.md should be the supporting source. If not, inspect node text, metadata, top-k, and embedding behavior.
LlamaIndex agents and workflows should sit on top of a tested retrieval layer.
Learners set gates before exposing the query engine to agents.
Why this matters: Tool-using agents amplify bad evidence with extra steps and longer traces.
If the query engine misses expected source nodes for direct questions, wrapping it in a FunctionAgent or workflow will not solve the evidence problem. It will just make the failure harder to read.
The gate should include source-node hit rate, faithful answer review, negative-case refusal, and metadata/citation quality. Only then should the query engine become an agent tool.
Use this checklist before calling the build complete.
Retrieval prompt: reconstruct the LlamaIndex build by naming reader, node parser, metadata fields, index, retriever settings, query engine, source_nodes inspection, and eval cases.
Build a LlamaIndex RAG prototype over three documents. Configure reader metadata, node parsing, VectorStoreIndex or chosen store, query engine top-k, source_nodes display, and a five-question eval with one insufficient-evidence case. Next rung: expose the query engine as an agent tool only after eval passes.
You pin the query-engine contract at the start of a LlamaIndex build. What is the PRIMARY reason for doing this before writing any code?
Pinning the contract first means you agree on answer and refusal behavior upfront, giving every subsequent build decision a concrete pass/fail criterion.
A reader ingests a 40-page PDF and a node parser splits it into chunks. Which outcome best describes what these two steps produce together?
Readers load raw content and node parsers split it into retrieval-ready chunks — the granular units the retriever can fetch and the LLM can cite individually.
In LlamaIndex, what does the source_nodes field on a query response tell you, and why does inspecting it matter during development?
source_nodes is the retriever's evidence trail — checking it is the only way to confirm the model's answer is grounded in the documents you indexed, not hallucinated.
You change similarity_top_k from 2 to 6 in your retriever settings. What is the most likely trade-off?
A higher similarity_top_k widens the evidence window — useful for complex queries, but it risks including loosely relevant nodes and raises prompt token usage.
Before wrapping your query engine as a tool for an agent, you run a citation evaluation pass. A claim in the generated answer has NO matching span in any source_node. What should you do?
An unsupported claim means the answer is not grounded in your documents; shipping it as a tool would propagate hallucinations downstream — fix retrieval or the prompt before wrapping.