Choose the right adaptation lever for instructions, knowledge, or behavior.
A beginner mistake is using the most expensive lever first. The real question is whether the failure is instruction, knowledge, or learned behavior.
Prompting changes behavior by changing the visible request.
Why this matters: It is the first lever when the model already has the capability.
Prompting, RAG, and fine-tuning all change model behavior, but they act at different layers. Prompting changes the visible instructions. RAG changes the evidence sent with the request. Fine-tuning changes weights through training examples.
Problem anchor: a refund assistant is polite but cites old policy, formats answers inconsistently, and sometimes misroutes escalation cases. One lever will not fix all three failures.
Prompting is best for task framing, tone, format, refusal policy, and context use when the model already has the capability.
RAG supplies evidence at request time.
Why this matters: It avoids retraining when facts change or need citations.
A prompt can define role, task, constraints, examples, context boundaries, and output shape. It is the cheapest lever to change and the easiest to evaluate.
Prompting fails when the model lacks needed evidence, the behavior is too frequent and brittle, or the output requires a hard schema better enforced by structured outputs.
Role: support assistant. Context: current retrieved refund policy will appear below. Task: answer in three short bullets. Constraints: do not invent policy, cite the source section, and ask for invoice ID if missing. Output: answer, source_section, missing_info.
This prompt is useful before fine-tuning because it creates an eval target. If it works reliably, training is unnecessary.
RAG answers with external evidence. It is the right lever when the facts should be updated, cited, or access-controlled.
Fine-tuning updates model weights for a narrower behavior.
Why this matters: It helps when prompting is too brittle and facts are not the issue.
RAG loads, chunks, indexes, retrieves, and packs evidence for the model to use. The model still writes the answer, but the facts come from the retrieved source pack.
Use RAG for policies, product docs, customer records, internal knowledge, and other material that changes or needs provenance.
The bot says refunds are available for 60 days because that pattern appeared in old examples. The policy changed to 30 days. Fine-tuning would bake in another snapshot; RAG can retrieve the current policy section at answer time.
Fine-tuning is for frequent stable behavior, format, domain shorthand, or task patterns that prompts cannot stabilize.
Criteria prevent choosing the fanciest lever by default.
Why this matters: They guide beginners toward the smallest reliable change.
Fine-tuning continues training on examples. The goal is not to add one new fact. The goal is to make a repeated behavior more likely by default: style, labels, tool-argument shape, specialist formatting, or domain shorthand.
The KB verdict is direct: use fine-tuning when repeated behavior should become part of the model, not when the app only needs fresher facts.
def choose_lever(problem): if problem == "stale facts": return "RAG" if problem == "tone or format": return "Prompting" if problem == "stable repeated label error": return "Fine-tuning" for p in ["stale facts", "tone or format", "stable repeated label error"]: print(p, "->", choose_lever(p))
input datadecision ruleprint(...)Run this as a tiny model of the mechanism. The point is to predict behavior before trusting the output.
It prints stale facts -> RAG, tone or format -> Prompting, and stable repeated label error -> Fine-tuning. The point is root-cause diagnosis before adaptation.
When an AI recommends fine-tuning, make it justify the root cause.
The final decision should choose the smallest lever that solves the actual failure with acceptable cost and risk.
The final choice maps problem type to adaptation method.
Why this matters: It turns the comparison into an actionable next step.
Decision this forces: Choose prompting, RAG, or fine-tuning for an adaptation problem.
| Option | Best for | Cost/complexity | Failure to watch | When to choose | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|---|
| Prompting | Instructions, tone, examples, formatting, refusal policy. | Lowest; fast to edit and test. | Brittle behavior under long or adversarial context. | Use first when capability exists and facts are available. | Choose when clearer instructions and examples can solve the behavior. | Low | Low |
| RAG | Private, changing, cited, or large knowledge. | Medium; needs indexing, retrieval, metadata, evals. | Retriever misses or citations do not support claims. | Use when facts should stay external and current. | Choose when the answer depends on current or source-grounded knowledge. | Medium | Medium |
| Fine-tuning | Stable repeated behavior learned from examples. | Highest; needs data, training, registry, rollback. | Overfit, stale facts, sensitive data, regressions. | Use after evals show prompting/RAG are insufficient. | Choose when durable behavior belongs in weights and examples are representative. | High | High |
Failure A: “The answer uses last quarter’s policy.” Choose RAG. Failure B: “The tone is too verbose.” Try prompting. Failure C: “The model repeatedly maps billing_dispute tickets to technical_support despite clear examples.” Consider fine-tuning after evals.
From memory, compare the three levers: prompting changes the request, RAG changes the supplied evidence, and fine-tuning changes model weights. Then classify three failures: wrong tone, stale policy, and repeated malformed labels.
You own a refund-policy bot that answers in the wrong tone, misses new policy text, and sometimes mislabels escalation intent. Choose which parts use prompting, RAG, or fine-tuning, and define the evals you would run first. Next rung: build golden datasets.
A customer-support bot keeps answering in a casual, jokey tone when it should be professional. No facts are missing — it just behaves wrong. Which lever should you reach for FIRST?
Tone is a behavior problem, not a missing-knowledge problem — a well-written prompt instruction is the smallest, fastest lever to try first.
Your company's product catalog changes every week. The model keeps citing outdated prices. What is the BEST fix?
RAG keeps knowledge outside the model's weights so it can be updated without retraining — exactly right for frequently changing facts like prices.
Which of the following is a retrieval-specific failure mode that RAG introduces — one that plain prompting does NOT have?
RAG adds a retrieval step, which means a new class of failure: the retriever can surface the wrong chunk, leading the model to give a confident but incorrect answer based on bad context.
A team wants to fine-tune a model so it always writes SQL in their company's specific style. Before they collect training data, what should they do FIRST?
Evals must come before training so you have a measurable definition of success — without them you can't tell whether fine-tuning actually helped.
In one or two sentences, explain the core difference between what fine-tuning changes and what RAG changes in a language model system.
Fine-tuning modifies weights (permanent, requires retraining to update); RAG keeps knowledge external and swappable without touching the model at all.