Turn questions and documents into searchable geometry.
Keyword search matches letters; embeddings match meaning. This module frames the whole topic: text becomes a point in space, and similarity becomes distance.
An embedding is a list of numbers that places a piece of text in a meaning-space, where closeness equals similarity.
Why this matters: It matters because every later choice — model, chunking, search — only makes sense once you picture text as points in space.
Imagine a help centre. A user types "my card got declined", but the right article is titled "payment failed at checkout". Keyword search shares no words, so it finds nothing — even though the meaning is identical.
That is the gap close. An turns a piece of text into a list of numbers — a point in space — positioned so that texts with similar meaning sit close together, even when they share no words.
Hold three ideas together and the rest of this lesson follows.
This is also the engine behind : retrieve the most relevant passages by meaning, then hand them to a language model as evidence before it answers.
A support team has 500 help articles. With keyword search, "my card got declined" returns nothing useful because no article uses those exact words.
They switch to . Each article is embedded once and stored. The query is embedded at search time, and the nearest article — "payment failed at checkout" — comes back first. Same meaning, different words, correct result.
That is the pattern to copy: embed the corpus once, embed each query, return nearest neighbours.
Each point is a short text placed by meaning. Pick a query and watch the nearest points light up — that is exactly what vector search returns.
The smallest useful sequence: text becomes vectors, a query finds the nearest ones.
Now we open the box. Text is chunked, each chunk is embedded into a vector, vectors are stored in an index, and a query is embedded and matched by distance. Knowing each stage tells you where quality is won or lost.
An embedding model converts each chunk to a vector, you store the vectors, and at query time you embed the question and find nearest vectors.
Why this matters: It matters because knowing the pipeline tells you exactly where quality, latency, and cost are decided.
Before reading on, predict from memory: in module 1 you met the core idea. What does an turn a piece of text into, and what does "close together" mean for two texts?
Answer it, then continue. (An embedding turns text into a vector — a point in meaning-space. Two texts close together mean similar things. This module shows the machinery that makes that happen.)
Building a searchable corpus and answering a query is one pipeline with two phases. Indexing happens once; querying happens on every request.
The query must be embedded with the same model used for the chunks — otherwise the two sets of points live in different spaces and distance means nothing.
Two vectors are compared by an angle, not just a straight-line gap. The most common measure is cosine similarity: it looks at the direction each vector points and scores how aligned they are, from -1 (opposite) to 1 (identical direction).
Why direction and not raw distance? Because meaning lives in which way a vector points, not how long it is. Two passages about the same topic point the same way even if one is wordier than the other.
So "nearest neighbour" really means "smallest angle" — the chunks whose direction best matches the query's direction.
A 200-page manual is ingested, split into ~800 of a few paragraphs each, and every chunk is embedded and stored once.
When a user asks "how do I replace the filter?", the question is embedded and the index instantly returns the three chunks whose vectors point most like the query — the filter-replacement section — without scanning all 800 by hand.
"How do I get a refund?" becomes a query vector using the same embedding model the chunks used.
The index computes similarity between the query vector and each stored chunk vector.
The highest-scoring chunks — say the top 5 — are returned as the most relevant passages.
The pipeline has a handful of high-leverage knobs: chunk size, which embedding model, exact vs approximate search, and whether to add a reranker. This module helps you pick the smallest design that meets the need.
The big levers are chunk size, which embedding model, exact vs approximate nearest-neighbour search, and an optional reranker.
Why this matters: It matters because these choices set the quality/cost/latency tradeoff for the whole retrieval system.
Decision this forces: Choose the retrieval design: pure vector search, hybrid with RRF, or vector search plus a reranker.
Before reading on, predict: if you make each very large, what do you gain and what do you lose at search time?
Four choices shape almost every retrieval system.
A common upgrade is hybrid retrieval — run keyword and together and merge the two ranked lists with so neither blind spot dominates.
Slide from tiny to huge chunks and read what you win and lose at each setting.
Write the user-visible need in one sentence, e.g. "return the right help article in the top 3 results."
Start with medium chunks, a standard embedding model, and approximate search. Add a reranker only if the requirement is unmet.
Run a handful of real queries and check whether the right chunk appears in the top results before reaching for heavier machinery.
| Option | Quality | Latency | Build effort | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Pure vector search, approximate | Good for most semantic queries. | Fast even on millions of vectors. | Just embed, store, query. | Start here: meaning-based search where exact wording varies. | Low | Low |
| Hybrid (vector + keyword, RRF merge) | Catches exact terms vector search misses (IDs, codes). | Two searches plus a merge step. | Maintain two indexes and fuse with RRF. | Use when queries mix natural language with exact tokens like part numbers. | Medium | Medium |
| Vector search + reranker | Best top-1 accuracy; reorders the shortlist. | Reranker adds a slower model on the top results. | Add and tune a second model. | Use when the single best answer must be first and latency budget allows. | High | Medium |
Theory becomes concrete in a few lines: embed the chunks, store them, embed the query, return the top matches. We also cover how to check an AI-generated version before trusting it.
A few lines that embed chunks, store them, embed a query, and return the top matches by similarity.
Why this matters: It matters because seeing the smallest runnable shape makes the pipeline concrete and debuggable.
A working has only three moving parts: a function that turns text into a vector, a place to keep the vectors, and a similarity comparison at query time.
Everything heavier — an index for scale, a , monitoring — is added on top of this same shape once you can see it run.
import numpy as np # 1. Embed: each chunk and the query become vectors (model omitted). chunk_vecs = embed(["payment failed at checkout", "reset my password"]) query_vec = embed(["my card got declined"])[0] def cosine(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) # 2. Search: score every stored chunk, return the best match. scores = [cosine(query_vec, c) for c in chunk_vecs] best = int(np.argmax(scores)) print("best chunk index:", best)
np.dot(a, b)np.linalg.normnp.argmaxTreat this as the smallest runnable shape: embed the and the query, score each chunk with cosine similarity, and pick the highest. A real system swaps the list scan for a vector index, but the logic is identical.
It returns the index of the HIGHEST score. Cosine similarity is larger when two vectors point the same way, so the most similar chunk has the maximum score — hence argmax, not argmin.
The same loop scales up almost unchanged: replace the Python list with a vector index, return the top 5 instead of the top 1, and feed those to a language model as evidence — and you have built a basic retriever.
If you ask an AI assistant to generate retrieval code, check these four things before trusting it:
Retrieval fails quietly — wrong chunks look like confident answers. This module names the common failure modes, brings back an earlier concept from memory, and ends with a decision and a checklist so the lesson sticks.
The common failure modes (bad chunking, model mismatch, stale index) and a checklist to catch them.
Why this matters: It matters because retrieval fails quietly — wrong results look like confident answers — so you need a deliberate check.
Decision this forces: Decide whether your retrieval is good enough to ship or needs a chunking fix or index rebuild first.
Before reading on, predict from memory: in module 2 you learned how closeness is measured. What does cosine similarity compare between two vectors, and what range does it run over?
Answer, then check. (Cosine similarity compares the direction the two vectors point, scoring from -1 for opposite to 1 for identical direction. Same direction means same meaning.)
These failures rarely throw an error — the system returns something, just the wrong something. Watch for each.
split mid-sentence or mix several topics. The relevant idea is fragmented or diluted, so the right passage never scores high enough to return.
The query is embedded with a different model than the corpus, or the corpus was re-embedded with a new model but the index was not rebuilt. Distances become noise, and results look random.
Documents change but their vectors are not re-embedded. Search confidently returns the old version — a silent correctness bug nobody sees until a user complains.
The right chunk is in the top 10 but not first, so a system that uses only the single best result answers from the wrong passage. This is exactly the gap a closes.
From memory, reconstruct the path: an turns text into a point in meaning-space; returns the nearest points; size, model choice, and an optional set the quality/cost tradeoff; and most failures come from bad chunking, model mismatch, or a stale index. Can you say each step before reading it?
Build a tiny semantic search over 20 short documents of your own: chunk them, embed with one model, store the vectors, and return the top 3 for five real queries. Confirm the right chunk appears in the top 3 each time. NEXT rung: feed those top chunks to a language model to turn your retriever into a working RAG answer, then add a reranker and measure whether top-1 accuracy improves.
Before choosing an answer, try to recall the idea from memory: in this lesson, what is an embedding?
An embedding turns text into numbers so a computer can compare meaning rather than only exact words.
You are building a retrieval pipeline for product-support docs; which order matches the basic flow?
Vector retrieval usually prepares chunks and their embeddings first, then embeds the query and compares it to stored vectors.
Your search often returns tiny snippets that match the query but lack enough surrounding information to answer well; what design change is most likely to help?
Chunk size is a tradeoff: smaller chunks can be precise, but larger or overlapping chunks can preserve needed context.
When is approximate nearest-neighbour search most worth using?
Approximate nearest-neighbour search trades a little accuracy for speed at larger scale.
In one or two sentences, what should you verify before shipping AI-generated retrieval code?
Retrieval code can look plausible while failing through wrong chunking, mismatched embeddings, reversed scoring, missing metadata, or untested bad results.