Fine-tuning, retrieval-augmented generation (RAG), and prompting are three ways to adapt a large language model to your task. Prompting changes only…
Compare prompting, RAG, and fine-tuning across cost, knowledge, and what each changes.
The three differ in what they touch. Prompting changes only the input text — the model is untouched. RAG changes the input too, but fills it automatically with passages retrieved from your data at query time. Fine-tuning changes the model itself, updating its weights on examples you provide.
That single distinction — input versus weights, manual versus retrieved — drives every trade-off that follows.
| Prompting | RAG | Fine-tuning ----------------|---------------|------------------|---------------- Changes | the prompt | prompt + data | model weights Adds knowledge | no | yes (at query) | limited Fresh / private | no | yes | stale after run Setup cost | lowest | medium | highest Best for | behavior | knowledge | style / format
Read down the columns: prompting is cheapest but adds no new knowledge; RAG is the way to bring in fresh, private, or large knowledge without retraining; fine-tuning is the way to bake in a consistent style, format, or skill, but its knowledge freezes at training time.
Recognize the cases prompting alone solves, and its ceiling.
Prompting is the first thing to try because it is instant, free of extra infrastructure, and often enough. Clear instructions, a few examples, and an output format handle classification, summarization, extraction, and rewriting on knowledge the model already has.
Its ceiling is knowledge and reliability: prompting cannot teach the model facts it never learned, and very long instructions can be followed inconsistently. When you keep pasting the same reference documents into every prompt, that is the signal to move to RAG.
See why RAG is the default for grounding answers in your own, changing data.
Retrieval-augmented generation (RAG) retrieves the passages relevant to each question and puts them in the prompt, so the model answers from your data instead of its memory. Update the data and the answers update immediately — no retraining.
RAG shines for private or proprietary knowledge, frequently changing information, large corpora that could never fit in one prompt, and any case where you must cite sources. It also reduces hallucination by grounding answers in retrieved text.
Judge when updating weights beats prompting or RAG, and how to combine approaches.
Fine-tuning is for teaching the model how to behave, not what to know. It excels at locking in a consistent tone or format, a specialized classification skill, or a narrow domain style — cases where hundreds or thousands of examples show a pattern that instructions struggle to convey.
It is the highest-cost option: you need a quality labeled dataset, a training run, and evaluation, and its knowledge is frozen at training time, so it goes stale as your data changes.
The three combine well. A common production stack fine-tunes a small model for a reliable format or style, then wraps it in RAG for fresh knowledge, all steered by a careful prompt. Reach for the cheapest layer that solves the problem and add the next only when you hit its ceiling.
Turn the trade-offs into a one-line rule you can apply immediately.
A practical rule: if the gap is missing knowledge, use RAG; if the gap is missing behavior (style, format, a repeated skill), fine-tune; and always start with prompting to see how far it gets you.
The classic mistake is fine-tuning to add facts. Fine-tuning teaches patterns, not a reliable, updatable knowledge base — that is RAG's job. Confusing the two wastes money and produces a model that still gets facts wrong.
Prompting, RAG, and fine-tuning adapt a language model in different places: prompting edits the input, RAG fills the input with retrieved data at query time, and fine-tuning updates the weights. Prompting is cheapest and adds no knowledge; RAG supplies fresh, private, citable knowledge; fine-tuning bakes in behavior and format but freezes its facts. Rule: knowledge gap → RAG, behavior gap → fine-tune, and start with prompting.
Pick a use case — say a legal assistant that must cite current regulations in a fixed report format. Decide which of prompting, RAG, and fine-tuning handles the citations and which handles the format, and write the one-line rule you used to split them.
What is the difference between fine-tuning, RAG, and prompting?
The three differ by what they touch — input, input-plus-retrieved-data, or weights — which drives their cost and capabilities.
When should I use RAG instead of fine-tuning?
RAG is the tool for supplying knowledge, especially data that changes or must be cited; fine-tuning is for behavior, not an updatable knowledge base.
Does fine-tuning add new knowledge to a model?
Fine-tuning bakes in style, format, and skills; its knowledge is limited and goes stale, so use RAG when the need is current, factual knowledge.
Can I combine RAG and fine-tuning?
The approaches are complementary; teams layer prompting, RAG, and fine-tuning, adding each only when the cheaper layer hits its ceiling.