Choose Retrieval-Augmented Generation when you need fresh, citable facts from external documents; choose fine-tuning when you need to reshape a model's…
Trace the full RAG pipeline from indexing through retrieval to generation, using a concrete document-QA scenario as the running example. You see exactly what happens at each stage so you can reason about where it helps and where it breaks.
A step-by-step trace of the RAG pipeline — from indexing documents through retrieving chunks to generating a grounded answer.
Why this matters: Understanding RAG's two paths (indexing and query-time) is the foundation for deciding when to use RAG versus fine-tuning for your SEO/GEO page content.
Your LLM was trained on a snapshot of the world — but your documents changed last week. How do you make it answer from today's facts without retraining?
) pairs the model with an external index. handles language and reasoning. The index supplies current facts. At query time, relevant chunks are fetched and injected into the prompt. in real evidence, not guessed from weights.
Updating knowledge means updating the index, not retraining the model. Every claim is traceable — you can show which chunk the answer came from. risk.
RAG has two distinct paths that run at different times — understanding both tells you where each failure can hide.
Imagine you're building a support bot over a 500-page product manual that updates monthly. A user asks: "What's the return policy for enterprise licenses?"
, embed each one, and store the vectors. When the manual updates, you re-index only the changed sections — the model itself never changes.
finds the three chunks most similar to the question and injects them into the prompt. answer — one that stays within what the retrieved text actually says.
Without RAG, the model would answer from training data — which predates the latest policy update. With RAG, the answer is always as fresh as the last index run.
# Naive: just ask the LLM directly — no retrieval question = "What is the return policy for enterprise licenses?" response = llm.complete(question) print(response) # Output: # "Enterprise licenses are typically non-refundable, # though a 30-day grace period may apply." # (Fabricated — the model has no access to your manual.)
. The model invents a plausible policy because it has no access to your actual document.
It answers confidently — but the policy details are fabricated. The model has no access to your manual, so it generates a plausible-sounding response from its training distribution. This is the hallucination failure RAG is designed to prevent.
# Stage 2: embed the query and retrieve top-k chunks question = "What is the return policy for enterprise licenses?" query_vector = embed(question) # same model used at index time results = vector_db.search( vector=query_vector, top_k=3 ) chunks = [r.text for r in results] print(chunks[0][:120]) # Output: # "Enterprise license holders may request a full refund # within 14 days of purchase by contacting billing@..."
from your manual. The first result already contains the real policy — exactly what the naive call missed.
It contains the actual manual text for enterprise refunds. The embedding search finds the semantically closest passage, not just a keyword match — so even if the user's phrasing differs from the document's wording, the right chunk surfaces.
# Stage 3: pack chunks into the prompt and generate context = "\n---\n".join(chunks) # join the 3 retrieved chunks prompt = f"""Answer using ONLY the context below. Cite the source. Context: {context} Question: {question}""" response = llm.complete(prompt) print(response) # Output: # "Enterprise license holders may request a full refund # within 14 days of purchase (Source: manual §7.2)."
response with a traceable source. — without it, the model may blend retrieved facts with memorized guesses.
Add a line like: system = "If the answer is not in the context, say 'I don't know.'" and prepend it to the prompt string, or pass it as a separate system message. The changed lines are the system string (new) and the prompt f-string (add {system} at the top). This guards against the model falling back to parametric memory when retrieval misses — the most common silent failure in production RAG.
Three failure modes account for most production RAG problems — two are silent.
RAG excels when knowledge is large, changing, or must be cited. cost. The model's output style, tone, and reasoning patterns remain unchanged.
When you need to change how the model behaves — not just what facts it cites — retrieval alone can't help. comes in. by training on examples of desired behavior.
Walk through supervised fine-tuning: how training examples reshape model weights, what LoRA adapters do, and what the same document-QA scenario looks like when the knowledge is baked into weights instead of retrieved at runtime.
A technical walkthrough of supervised fine-tuning: how training examples reshape model weights, what LoRA adapters do, and why fine-tuning shapes behaviour rather than reliably injecting facts.
Why this matters: Knowing how fine-tuning works lets you judge when it beats RAG for your SEO/GEO page use case — and when it will silently mislead users with stale or hallucinated facts.
Answer: at query time. The fetches relevant and injects them into the prompt on every request. The model weights never change.
Fine-tuning takes the opposite approach: knowledge is baked into the weights before any request arrives. Nothing is retrieved at runtime. That single difference drives every tradeoff in module 3.
) continues training a pretrained model on your own prompt–completion pairs, adjusting the model's weights so target responses become more probable by default.
The result is stored in — the weights themselves — rather than in an external index. Because the weights change, the knowledge or style is always present without any retrieval step.
Instruction tuning is a specific form of SFT where the training examples are (instruction, response) pairs designed to make a base model follow directions. SFT is the broader term; instruction tuning is one application of it.
Full fine-tuning updates every weight in the model, which is expensive and risks overwriting general capabilities. (Low-Rank Adaptation) freezes the base weights and instead trains two small matrices whose product approximates the weight update.
For a weight matrix of shape [d × k], LoRA adds matrices A [d × r] and B [r × k] where rank r ≪ d. Only A and B are trained; the base model is untouched. At inference the adapter is merged back: W_new = W + A·B.
The adapter is a small, portable artifact tied to a specific base model. You can swap adapters to change task or style without reloading the full model.
Recall the document-QA scenario from module 1: a support bot answers questions about a product manual. With RAG, the manual is chunked and retrieved at query time. With fine-tuning, you instead build a dataset of (question, answer) pairs drawn from the manual and train the model on them.
Each training example is a prompt–completion pair. The prompt is the user question (optionally with a system instruction); the completion is the correct answer. The model learns to predict the completion tokens — it never sees the raw manual at inference time.
What fine-tuning cannot reliably do is inject new facts. A model trained on Q&A pairs learns the answer patterns, but it may still details not well-represented in the training set. Fine-tuning is best for durable style, format, and decision boundaries — not for keeping facts current.
# Stage 1: build one prompt-completion pair for the doc-QA dataset example = { "prompt": ( "System: You are a product support assistant.\n" "User: How do I reset the device to factory settings?" ), "completion": ( "Press and hold the Reset button for 10 seconds " "until the LED flashes red, then release." ), } print(example["prompt"]) print("---") print(example["completion"])
This is the atomic unit of a fine-tuning dataset: one prompt and its target completion. The model is trained to predict the completion tokens given the prompt — the system instruction is part of the input context.
It prints the completion string: "Press and hold the Reset button for 10 seconds until the LED flashes red, then release." — exactly the target the model learns to predict.
# Stage 2: assemble a small dataset and check coverage dataset = [ {"prompt": "System: You are a product support assistant.\nUser: " + q, "completion": a} for q, a in [ ("How do I reset the device?", "Hold Reset 10 s until LED flashes red."), ("What is the battery life?", "Up to 12 hours on a full charge."), ("How do I pair via Bluetooth?", "Hold BT button 3 s; select device in settings."), ] ] print(f"{len(dataset)} examples | avg completion length: " f"{sum(len(d['completion']) for d in dataset) // len(dataset)} chars")
This stage scales the single pair into a dataset and surfaces a quick coverage check. In practice you'd load from JSONL; the structure is identical.
"3 examples | avg completion length: 44 chars" (exact char count may vary by a character). The key signal: 3 examples is far below the ~50–100 minimum for even a style shift — this dataset needs more coverage before training.
# Stage 3: wire a LoRA adapter config before training # Scenario twist: you need rank=16 and only target the query/value projections. lora_config = { "r": 16, # adapter rank "lora_alpha": 32, # scaling factor (alpha / r = effective scale) "target_modules": ["q_proj", "v_proj"], "lora_dropout": 0.05, "bias": "none", # TODO: add the key that tells the trainer ONLY adapter weights are updated # Hint 1: you want to freeze the base model parameters. # Hint 2: the field name reflects that intent directly. } print(lora_config)
Stop — attempt the TODO before revealing. This config dict is passed to a LoRA training wrapper; the missing field is the one that makes LoRA efficient.
Add "trainable_parameters": "lora_only" — or in HuggingFace PEFT terms, "inference_mode": False paired with calling model.freeze_base_model(). The canonical PEFT field is "bias": "none" (already present) plus setting trainable_parameters to adapter weights only. CHANGED LINE: the TODO becomes "trainable_parameters": "lora_only". Why it matters: without this, the trainer updates all weights and you lose LoRA's efficiency advantage entirely.
These are the three failure modes that catch teams off guard. Each has a concrete symptom to watch for.
lora_alpha / r is set intentionally (default 1.0 scale is often too low); (3) run a 10-example overfit test — if loss doesn't drop to near-zero, the data format is wrong; (4) verify the adapter is saved separately from the base model so you can swap it without reloading.Module 3 maps these failure modes against RAG's failure modes on four axes: update cost, latency, freshness, and control. This helps you choose the right tool for each scenario.
Map RAG and fine-tuning against four practical axes — update cost, inference latency, factual freshness, and output controllability — using the document-QA scenario to make each axis concrete. You finish with a populated comparison table you can drop straight into the SEO page.
Maps RAG and fine-tuning against four practical axes — update cost, latency, freshness, and controllability — and shows how to score a use case on each.
Why this matters: Gives you the comparison table and scoring method you need to justify a RAG-vs-fine-tuning choice on your SEO page or in a design doc.
Decision this forces: Which axis matters most for your specific use case — freshness/updatability vs. style/format consistency?
The answer is the . Fine-tuning bakes knowledge into . No retrieval call runs at inference time. That difference drives the latency trade-off you'll map next.
Module 2 showed how fine-tuning reshapes weights. This module maps both approaches against four axes: update cost, inference , factual freshness, and output controllability.
Every RAG-vs-fine-tuning decision reduces to four practical axes, each of which favors one approach.
None of the four axes is universally decisive — the right answer depends on which axis your use case weights most heavily.
Take the document-QA scenario from modules 1 and 2 — a support bot that answers questions from a company knowledge base — and walk each axis.
The knowledge base gains 50 new articles every week. With , you re-embed and re-index those 50 articles — a pipeline job measured in minutes. With , you queue a new training run, wait hours, validate, and redeploy. RAG wins on update cost.
Each user query triggers a vector search across thousands of before generation begins. That retrieval adds 50–200 ms on top of model inference. A fine-tuned model skips that step entirely. Fine-tuning wins on latency.
A product is discontinued today. RAG surfaces the updated article on the next query after re-indexing. The fine-tuned model still cites the old product until the next training run. RAG wins on freshness.
The support team wants every answer in a strict three-sentence format with a ticket-number prefix. Prompt instructions enforce this most of the time, but edge cases slip through. Fine-tuning on 500 labeled examples locks the format into weights — it holds even on unusual queries. Fine-tuning wins on controllability.
# Axis scores: 1 = RAG wins, -1 = fine-tuning wins, 0 = tie USE_CASE = "support-bot" # weekly doc updates, strict reply format axes = { "update_cost": 1, # re-index beats retraining "inference_latency": -1, # no retrieval step = faster "freshness": 1, # index refreshes daily "controllability": -1, # format must be locked } rag_score = sum(v for v in axes.values() if v > 0) ft_score = sum(abs(v) for v in axes.values() if v < 0) print(f"RAG wins on {rag_score} axes, Fine-tuning wins on {ft_score} axes")
This snippet encodes the four-axis judgment as a signed score and tallies which approach wins more axes for a given use case. The output for the support-bot scenario is: RAG wins on 2 axes, Fine-tuning wins on 2 axes — a tie that signals the hybrid approach.
RAG wins on 2 axes, Fine-tuning wins on 2 axes
The scores tie because freshness and update_cost favor RAG (+1 each) while latency and controllability favor fine-tuning (-1 each). A tie is the signal to consider a hybrid approach.
# New use case: legal-research bot # - Case law updates monthly (not daily) # - Answers must cite exact statute numbers (controllability critical) # - Latency tolerance is high (lawyers wait for accuracy) axes = { "update_cost": 1, # monthly re-index is fine "inference_latency": 0, # latency not a priority "freshness": 1, # monthly updates still favor RAG "controllability": # TODO: fill in -1 (fine-tuning) or 1 (RAG) } rag_score = sum(v for v in axes.values() if v > 0) ft_score = sum(abs(v) for v in axes.values() if v < 0) print(f"RAG wins on {rag_score} axes, Fine-tuning wins on {ft_score} axes")
Stop — attempt the TODO before revealing. The legal-research bot must cite exact statute numbers consistently; which axis score does that map to, and why? The surrounding context is complete; only the controllability score is missing.
controllability: -1 ← CHANGED from the worked example
Reason: strict citation format is a controllability requirement, and fine-tuning wins on controllability (-1 = fine-tuning wins).
Full output:
RAG wins on 2 axes, Fine-tuning wins on 1 axes
RAG still wins overall (freshness + update_cost), but the controllability need is strong enough to consider a hybrid: RAG for fresh case law, fine-tuning for citation format.
| Option | Update cost | Factual freshness | Output controllability | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| RAG | Re-index changed docs only — minutes, not hours. | Reflects updates as soon as the index is refreshed. | Style enforced via prompt; can drift on edge cases. | Knowledge changes frequently, citations matter, or data volume is large and growing. | Low per update (re-index only changed docs); higher per query (retrieval + generation). | Medium — requires indexing pipeline and retriever maintenance. |
| Fine-Tuning | New training run required for every knowledge update. | Frozen at training cutoff; stale until retrained. | Style and format baked into weights — highly consistent. | Style, tone, or format consistency is the primary goal and knowledge is relatively stable. | High per update (full or LoRA training run); lower per query (no retrieval step). | High — requires labeled dataset, training run, evaluation, and redeployment. |
| Hybrid (RAG + Fine-Tuning) | Re-index for facts; retrain only when style drifts. | Fresh facts via retrieval; style stays consistent. | Best of both — style from weights, facts from index. | You need both fresh facts and consistent style — e.g., a branded support bot over a live knowledge base. | Highest upfront; retrieval cost per query plus periodic fine-tuning runs. | High — combines both pipelines; requires coordinated maintenance. |
Three failure patterns appear when teams map these axes without care.
You now have a populated comparison table and scoring method. The four axes tell you what each approach costs. They don't tell you which axis to weight most heavily.
Module 4 addresses routing. It uses four diagnostic questions — Is knowledge dynamic? Is the goal style or facts? Do you need citations? What is your data volume? — to produce a concrete recommendation.
Work through four diagnostic questions — Is the knowledge dynamic? Is the goal style or facts? Do you need citations? What is your data volume? — and apply them to the document-QA scenario to produce a routing decision. You complete a partially filled decision tree and supply the missing branch.
A four-question diagnostic framework that routes any use case to RAG, fine-tuning, or a justified hybrid.
Why this matters: Gives you a repeatable decision process for the RAG-vs-fine-tuning choice — the core judgment call on the SEO comparison page you're building.
Decision this forces: For your specific use case: does it route to RAG, fine-tuning, or a hybrid — and which of the four criteria drove that choice?
The four axes were: update cost, inference , factual freshness, and output controllability. RAG wins on freshness and update cost; wins on latency and style control. Those axes describe the cost of each choice — but they don't tell you which choice to make for your specific use case. That's what this module adds: four diagnostic questions that route the decision.
Four diagnostic questions turn the trade-off axes into a routing decision for your specific use case.
One-line decision rule: use RAG when facts change or citations matter; use fine-tuning when style or behavior must change.
Apply the four questions to the running document-QA scenario — a support bot that answers from a company's policy and product docs.
All four questions point the same way: this scenario routes cleanly to . When all four align, the decision is easy. The interesting case is when they split — which is where the decision tree earns its keep.
A system — fine-tuning plus RAG — is justified in exactly one scenario: you need both a style or behavior shift AND fresh, citable facts.
A concrete example: a legal-research assistant that must write in formal legal prose (style → fine-tune) while citing up-to-date case law (freshness + citations → RAG). Neither alone is sufficient.
def route_decision(dynamic: bool, style_shift: bool, needs_citations: bool, labeled_examples: int) -> str: if dynamic and style_shift: return "hybrid" if dynamic: return "rag" if style_shift and needs_citations: return "hybrid" if style_shift: return "fine-tuning" # TODO: handle the factual-QA, stable-knowledge branch # Hint 1: the data-volume threshold used in the decision tree is 500 examples. # Hint 2: citations force RAG even when data volume is high. ...
This function encodes the four-question framework as executable logic — each branch maps to one path in the decision tree.
Stop — attempt the TODO before revealing. The missing branch handles the case where knowledge is stable and the goal is factual Q&A. What should it return, and under what conditions?
# CHANGED LINES — this is the crux of the four-question framework:
if needs_citations:
return "rag" # citations require retrieved passages
if labeled_examples >= 500:
return "fine-tuning" # enough data to generalize
return "rag" # low data → lower-friction path
# Why these lines: the citation check comes first because it overrides
# data volume — parametric memory has no provenance trail.
# The 500-example threshold mirrors the decision tree's data-volume branch.
Three failure patterns appear repeatedly when teams apply this framework.
The four-question framework tells you which path to take. It doesn't guarantee the path works. A correct RAG routing decision still fails if the misses the right , or if the model on exact identifiers like product codes. A correct fine-tuning decision still fails if knowledge drifts or catastrophic forgetting erases earlier capabilities.
The next module maps the top failure modes for each path. It gives you concrete checks to run before trusting your system in production.
Examine the top failure modes for RAG (retrieval miss, hallucination on exact identifiers, context overflow) and fine-tuning (knowledge drift, catastrophic forgetting, data leakage), then apply retrieval-evaluation metrics from Ragas to the document-QA scenario to verify the RAG path is working.
A diagnostic module that names the top failure modes for RAG and fine-tuning, then shows how to measure and catch them using Ragas metrics and a drift smoke-test.
Why this matters: Knowing what breaks — and how to detect it — is what separates a deployed system from a prototype; these checks belong in every RAG or fine-tuning release pipeline.
Decision this forces: Which failure mode is most likely given your data and query type — and what is your mitigation?
Module 4's answer: "Is the knowledge dynamic?" If facts change faster than you can retrain, wins — you update the index, not weights.
But choosing the right path doesn't guarantee correct deployment. This module asks: what breaks after you've chosen — and how do you catch it?
RAG failures cluster into three distinct patterns, each with a different fix.
Fine-tuning failures are quieter than RAG failures. They often surface weeks after deployment, not at launch.
Ragas provides two metrics that map directly onto RAG's two main failure modes — use the right one or you'll diagnose the wrong problem.
Diagnostic rule: if Context Precision is high but Faithfulness is low, the retriever is working but the generator is hallucinating. If both are low, fix retrieval first — a bad context makes faithfulness meaningless.
from ragas import evaluate from ragas.metrics import faithfulness, context_precision from datasets import Dataset sample = Dataset.from_dict({ "question": ["What is the return window for Model X?"], "answer": ["The return window is 45 days."], "contexts": [["Model X has a 30-day return policy."]], "ground_truth": ["The return window is 30 days."] }) result = evaluate(sample, metrics=[faithfulness, context_precision]) print(result) # {'faithfulness': 0.0, 'context_precision': 1.0}
This snippet runs Ragas against a single document-QA example where the answer contradicts the retrieved chunk. Context Precision scores 1.0 (the right chunk was retrieved), but Faithfulness scores 0.0 — the generator invented "45 days" instead of copying "30 days" from the context. This is the exact identifier / hallucination failure mode in measurable form.
Faithfulness = 0.0. Ragas checks whether every claim in the answer is supported by the retrieved context. '45 days' is not in the context, so the claim fails — faithfulness collapses to zero even though the right chunk was retrieved (context_precision = 1.0). This is the hallucination-on-exact-identifiers failure mode.
# Running example: document-QA smoke test # CHANGE from Stage 1: add fine-tuned-model knowledge-drift probe ★ def smoke_test_ft_drift(model_fn, probe_pairs): # probe_pairs: list of (question, expected_answer) tuples failures = [] for question, expected in probe_pairs: response = model_fn(question) # TODO: replace the condition below to flag drift if expected.lower() not in response.lower(): # CHANGED LINE ★ failures.append({"q": question, "got": response, "want": expected}) return failures # Example call probes = [("What is the return window for Model X?", "30 days")] print(smoke_test_ft_drift(my_ft_model, probes))
This completion task extends the running example to the fine-tuning side: a minimal drift smoke-test that checks whether the model still returns expected factual answers after a training update. Your job is to fill the TODO condition — the logic that decides a response has drifted from the expected answer.
CHANGED LINE: if expected.lower() not in response.lower():
Why: the condition flags any response that doesn't contain the expected answer string (case-insensitive). This catches knowledge drift — e.g. the model saying '45 days' when the ground truth is '30 days'. For a production suite you'd use a semantic similarity check, but exact-match is the right minimal smoke test for known factual anchors. Everything else in the function stays the same.
When an LLM writes your Ragas evaluation harness or drift smoke-test, check four things before trusting results:
evaluate() matches your failure mode. A generated script often defaults to answer_relevancy, which won't catch hallucination."contexts" (a list of lists) and "ground_truth". A generated script may use "context" (singular). This silently scores everything as 1.0.assert result['faithfulness'] > 0.8) so CI catches regressions.Produce the complete comparison section of the SEO page: a filled comparison table (rows = RAG and fine-tuning, columns = the four criteria axes), the one-line decision rule, and a 3–5 question FAQ written as real user queries — the same content that becomes FAQPage schema and the lesson's knowledge check.
A synthesis module that assembles the full RAG vs fine-tuning comparison into a publishable SEO artifact: a four-axis table, a one-line decision rule, five FAQ entries, and a defensible routing choice.
Why this matters: This is the capstone module — it turns everything learned into a concrete, quotable deliverable (comparison table, decision rule, FAQ) that goes directly onto the SEO/GEO page you are building.
Decision this forces: Final routing: for your target use case, is the answer RAG, fine-tuning, or hybrid — and can you defend it against all four criteria and the top failure mode?
Answer: hallucinates on exact identifiers because dense embeddings match by meaning, not by character string — a SKU like "XR-4402" may retrieve the wrong chunk. The metric that catches this is : it measures whether the retrieved chunks actually contain the answer, not just something topically related.
That failure mode is exactly the kind of evidence you need when defending a routing decision. This module turns everything from modules 1–5 into a single, defensible choice.
Retrieval-Augmented Generation (RAG) and fine-tuning solve different problems. grounds answers in live documents, keeping facts fresh without retraining. bakes a style, format, or domain behavior into model weights. It suits stable knowledge and consistent output patterns. Most production systems combine both.
This capsule names both entities in sentence one. It states the core distinction and ends with the hybrid signal. That's exactly what an LLM or featured snippet will lift.
These five questions mirror real search intent for "rag vs fine tuning" and become your FAQPage schema entries. Each opens with a direct one-sentence answer, then adds the depth a searcher actually needs.
Use when your knowledge base changes frequently or when answers must cite a source. RAG re-indexes documents without retraining, so updates cost minutes, not GPU hours. It is the default choice for document QA, support bots, and any domain where facts shift weekly.
wins when you need a consistent output style, tone, or structured format — and the underlying knowledge is stable. A legal-brief formatter or a brand-voice chatbot benefits from fine-tuning because the goal is behavior, not fresh facts.
Yes — a system uses a fine-tuned model as the generator and a retrieval stack to supply fresh facts. Start with RAG alone; add fine-tuning only after you confirm that style gaps are hurting user experience.
Fine-tuning has lower because there is no retrieval step. RAG adds a round-trip to the vector index — typically 50–200 ms — before generation begins. For real-time voice or sub-100 ms UIs, that overhead matters.
No. Fine-tuning does not reduce on factual specifics — it can make the model more confidently wrong. RAG reduces hallucination by grounding answers in retrieved text, but only when the retriever surfaces the right chunk. Neither approach eliminates hallucination entirely.
criteria = ["freshness", "latency", "factual_control", "style_control"] table = { "RAG": ["good", "ok", "good", "bad"], "Fine-Tuning": ["bad", "good", "bad", "good"], # TODO: fill in the Hybrid row using the four ratings from the comparison table "Hybrid": [???], } for approach, ratings in table.items(): row = dict(zip(criteria, ratings)) print(f"{approach:14} | {row}") print("\nDecision rule: facts change → RAG | style matters → Fine-Tuning | both → Hybrid")
This snippet builds the four-axis comparison table as a Python dict and prints a one-line decision rule. It is the artifact your SEO page's structured data section needs — fill in the TODO before revealing.
Changed line — Hybrid row:
"Hybrid": ["good", "ok", "good", "good"]
Why each changed from the RAG-only row:
freshness → good (same as RAG — re-index handles updates)
latency → ok (retrieval hop still present)
factual_control → good (RAG layer grounds the facts)
style_control → good (fine-tuned weights control the voice — this is the NEW win vs plain RAG)
Printed output for the Hybrid row:
Hybrid | {'freshness': 'good', 'latency': 'ok', 'factual_control': 'good', 'style_control': 'good'}
You now have every tool to make and defend a routing decision. Here is the format your capstone answer should follow for your target use case.
This five-point structure is also the skeleton of your SEO page's conclusion. A reader arriving cold from a search for "rag vs fine tuning" should apply your decision rule to their own use case.
| Option | Freshness (knowledge update cost) | Latency (inference overhead) | Factual control (citations/grounding) | Style/format control | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|---|
| RAG | Re-index only; no retraining needed. | Extra retrieval hop adds ~50–200 ms. | Strong — answers are grounded in retrieved chunks. | Weak — style comes from the base model, not training. | Knowledge changes frequently, citations are required, or the corpus is too large to bake into weights. | Low retraining cost; ongoing infra cost for the vector index and retriever. | Moderate — requires chunking, embedding, and retrieval pipeline. |
| Fine-Tuning | Requires full or partial retraining on new data. | No retrieval step; inference latency is unchanged. | Weak — no source references; hallucination risk on specifics. | Strong — tone, format, and persona are baked in. | Output style, tone, or format must be consistent; knowledge is stable; labeled examples are available. | High upfront GPU cost; re-run whenever knowledge changes. | High — requires curated training data, compute, and evaluation. |
| Hybrid (RAG + Fine-Tuning) | RAG layer handles updates; weights stay stable. | Retrieval hop still present; adapter adds minimal overhead. | Strong — retrieval grounds the facts. | Strong — fine-tuned weights control the voice. | You need both consistent style and fresh, citable facts — e.g., a branded support bot over a live knowledge base. | Highest — both pipeline and training costs. | High — must maintain both the retrieval stack and the fine-tuned adapter. |
Three ways the routing decision fails in practice:
Before reading the summary: reconstruct from memory the four criteria that route a use case to RAG or fine-tuning, name one failure mode for each approach, and state the one-line decision rule. Then check your answer against the module sequence.
Apply what you learned to SEO/GEO page — the page title is the H1 and the exact search query. OPEN with a 40-60 word self-contained, quotable answer capsule that names the entity explicitly in sentence one (no "in this lesson…" preamble) — this is the sentence an LLM lifts. Name the entity by its full name every time (not pronoun-only). Structure each section as ONE sub-question (How it works · X vs Y · When to use it · Example · Common mistakes), each opening with its own 1-2 sentence direct answer then depth. End with a genuine 3-5 question FAQ phrased as real user questions ending in "?" (these become FAQPage schema + the knowledge check). Prefer quotable specifics — concrete numbers, defaults, version names, one short snippet — over vague prose. Self-contained for a reader who arrived cold from a search engine or an LLM. Intent = comparison: include a comparison TABLE (rows=options, columns=criteria) and a one-line decision rule. Target query: "rag vs fine tuning"..
A news aggregator needs its AI assistant to answer questions about articles published this morning. Which approach handles this requirement and why?
A. Fine-tuning, because it bakes new facts into model weights at training time.
B. RAG, because it retrieves documents at query time without retraining.
C. Fine-tuning, because instruction tuning teaches the model to follow news-style prompts.
D. RAG, because it reduces latency by caching yesterday's articles.
RAG retrieves live documents at query time, so it handles today's articles without any retraining — that is the core freshness advantage. Fine-tuning encodes knowledge into weights during a training run, so it cannot reflect articles published after that run without retraining again. Instruction tuning shapes response style, not factual currency. RAG does not reduce latency — the retrieval step is actually RAG's main latency cost, not a caching benefit.
Consider this minimal RAG retrieval snippet:
chunks = index.query(user_query, top_k=5)
prompt = system_prompt + '\n'.join(chunks) + user_query
response = llm.complete(prompt)
What is the component that makes the retrieved chunks grounded rather than hallucinated?
Grounding happens because index.query retrieves real source chunks and those chunks are placed in the prompt context the LLM reads — the model answers from supplied evidence rather than from parametric memory. llm.complete generates text; it does not filter hallucinations on its own. top_k=5 controls retrieval breadth, not answer correctness. A system prompt can ask for honesty but cannot supply the factual evidence that prevents hallucination.
Name TWO failure modes specific to RAG and TWO failure modes specific to fine-tuning. (One or two words each is enough — you are recalling from memory.)
RAG's two main failure modes are low context precision (retrieved chunks are off-topic, polluting the prompt) and low faithfulness (the model ignores the retrieved context and hallucinates from weights anyway). Fine-tuning's two main failure modes are knowledge drift (catastrophic forgetting of pre-training facts) and style overfitting (the model locks onto training-set phrasing and breaks on varied inputs). Knowing which failure mode is likely for your data type is the first step to choosing the right mitigation.
A legal firm wants its LLM to always respond in a precise, formal citation style it has defined internally — the style does not change week to week, and the corpus of case law is static. Which approach does the four-question framework route this use case to?
The four-question framework routes to fine-tuning when the dominant need is style/format consistency on data that does not change frequently — both conditions are true here. RAG is the right choice when freshness or a large, changing knowledge base is the bottleneck, which is not the case for a static corpus. Hybrid is justified only when both freshness AND style control are simultaneously required; the firm's corpus is static, so the retrieval half of a hybrid adds cost without benefit. Fine-tuning can absolutely learn citation style from prompt-completion pairs — that is a primary use case for instruction tuning.
You run a Ragas evaluation and find that context_precision is 0.91 but faithfulness is 0.43. What does this diagnosis tell you and what should you fix?
High context_precision means the retrieved chunks are on-topic — retrieval is working. Low faithfulness means the model's answer is not supported by those chunks — the model is drawing on parametric memory instead of the supplied context. The fix is on the generation side: tighten the system prompt to force grounding, or use a model better at following context instructions. Retrieval is not the problem here, so changing the embedding model or adding documents would not help. There is no universal 0.4 passing threshold for faithfulness; lower is always worse.