Build and deploy a vector search application with Pinecone by mastering index creation, embedding ingestion, similarity querying, metadata filtering…
Create a Pinecone account, generate an API key, and provision a serverless index with the correct dimension and metric for your embedding model. You'll end this module with a live, empty index ready to receive vectors.
Step-by-step guide to creating a Pinecone serverless index with the correct dimension and similarity metric, then verifying it is live via the Python SDK.
Why this matters: Every vector you store and every search you run depends on getting this setup right — a wrong dimension or metric cannot be fixed without deleting the index and starting over.
Decision this forces: Which similarity metric — cosine, dotproduct, or euclidean — fits your embedding model and use case?
Your semantic search or pipeline needs a place to store and query . That place is a . Before writing vectors, you lock in two parameters: (vector length) and (how Pinecone measures closeness).
A removes infrastructure management. Pinecone scales storage and compute automatically. You pay per query and storage, not idle capacity.
You're building document search with OpenAI's text-embedding-3-small model. It outputs 1536-dimensional vectors. The model card confirms: dimension = 1536, vectors are L2-normalized, cosine is the right metric.
You create a serverless index named docs-search with dimension=1536 and metric='cosine'. A colleague suggests dotproduct for speed. But normalized vectors give identical rankings with both metrics. Cosine is safer and more explicit.
Using dimension=768 (a common sentence-transformers size) with a 1536-d model would fail. Every would error on dimension mismatch. The index becomes useless.
from pinecone import Pinecone, ServerlessSpec pc = Pinecone(api_key="YOUR_API_KEY") pc.create_index( name="docs-search", dimension=768, # ← wrong: model outputs 1536-d metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1") )
This creates the index without error — Pinecone accepts any positive integer for dimension. The problem surfaces only when you try to upsert 1536-d vectors later.
Pinecone raises: 'Vector dimension 1536 does not match the dimension of the index 768.' The index itself was created successfully — the mismatch is silent at creation time and only explodes at upsert. You must delete the index and recreate it with dimension=1536.
from pinecone import Pinecone, ServerlessSpec import time pc = Pinecone(api_key="YOUR_API_KEY") pc.create_index( name="docs-search", dimension=1536, # matches text-embedding-3-small metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1") ) # Wait until the index is ready before any operation while not pc.describe_index("docs-search").status["ready"]: time.sleep(1) print(pc.describe_index("docs-search").status)
This is the corrected version: dimension matches the model, and the readiness poll prevents 'index not ready' errors on the very next call. Skipping the poll is a common source of intermittent failures in CI pipelines.
{'ready': True, 'state': 'Ready'} — confirming the index is live and accepting operations. If you print before it's ready you'll see {'ready': False, 'state': 'Initializing'}.
from pinecone import Pinecone pc = Pinecone(api_key="YOUR_API_KEY") # TODO: get a handle to the "docs-search" index # Hint: use pc.Index(...) with the index name index = ___ stats = index.describe_index_stats() print(f"Dimension : {stats.dimension}") print(f"Total vectors: {stats.total_vector_count}")
Stop — attempt the TODO before revealing. You have the client and the index name; supply the one line that returns an index handle.
Changed line: index = pc.Index("docs-search")
Output:
Dimension : 1536
Total vectors: 0
The dimension confirms the index was created correctly. Total vectors = 0 means it's live but empty — exactly the state you want before ingestion begins.
| Option | Best for normalized embeddings | Sensitive to vector magnitude | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| cosine | Yes — measures angle only | No — magnitude is ignored | Use for most text embedding models (OpenAI, sentence-transformers) where direction matters more than magnitude — the safe default for semantic search. | Low | Low |
| dotproduct | Equivalent to cosine when normalized | Yes — larger magnitude = higher score | Use when your model explicitly outputs un-normalized vectors and magnitude carries meaning (e.g. ColBERT-style models, some recommendation embeddings). | Low | Low |
| euclidean | Redundant with cosine when normalized | Yes — penalizes magnitude differences | Use for image or numeric embeddings where absolute distance in space matters — rarely the right choice for text. | Low | Low |
Three failure patterns account for most broken Pinecone setups at this stage.
Vector dimension 1536 does not match the dimension of the index 768. Fix: always check the model card for the exact output dimension before calling create_index.cosine with a model that outputs un-normalized vectors (e.g. some fine-tuned bi-encoders) produces subtly wrong rankings — no error, just bad results. Check whether your model normalizes output; if not, use dotproduct.index.upsert() immediately after create_index raises Index not ready intermittently — it passes locally but fails in fast CI runs. Always poll describe_index(...).status['ready'] before proceeding.If you used an LLM to generate your index creation code, check these three things before running it in production:
create_index and the first or query call.You now have a live, empty with the right dimension and metric. The status shows total_vector_count: 0. An empty index is useless until it holds vectors.
Next, you'll generate embeddings and ingest data. Use OpenAI's text-embedding-3-small to convert text into . Format each as (id, values, metadata) and it into your index.
Use OpenAI's text-embedding-3-small (1536-d) or a sentence-transformers model to convert raw text into dense vectors, then prepare them as (id, values, metadata) tuples for Pinecone. You'll work through a fully annotated ingestion loop and complete the batching step yourself.
How to convert raw text into embedding vectors and structure them as Pinecone-ready (id, values, metadata) records, including batching.
Why this matters: Every vector your SEO/GEO page index stores starts here — get the embedding model, dimensions, and record shape right before a single write happens.
Decision this forces: Which embedding model — OpenAI text-embedding-3-small, text-embedding-3-large, or a local sentence-transformers model — fits your latency and cost constraints?
The dimension is the length of every vector your will accept — for example, 1536 for text-embedding-3-small. Pinecone rejects any upsert whose vectors have a different length with a dimension-mismatch error. That constraint is why you must lock in the embedding model before provisioning the index, not after.
Your pipeline has three steps. First, embed each document chunk into a . Second, attach a unique and . Third, push the tuples to Pinecone.
An model maps a string to a fixed-length float list. text-embedding-3-small produces 1536 numbers. Nearby vectors share meaning, not wording. This enables semantic search.
Each Pinecone record is (id, values, metadata): a string ID you control, the float vector, and a dict of filterable fields like URL or date.
Metadata travels with the vector but doesn't affect similarity calculation. It's used later for to narrow results before or after ranking.
import openai client = openai.OpenAI(api_key="YOUR_API_KEY") text = "Pinecone is a managed vector database." response = client.embeddings.create( model="text-embedding-3-small", input=text ) vector = response.data[0].embedding print(len(vector)) # → 1536
This is the smallest working unit: one string in, one 1536-d float list out. Confirm the length before building the loop — a mismatch here means your index dimension is wrong.
len(vector) prints 1536. Upserting into a 384-d index raises a dimension-mismatch error — Pinecone rejects the record immediately because the vector length doesn't match the index's fixed dimension.
docs = [
{"id": "doc-0", "text": "Pinecone is a managed vector database."},
{"id": "doc-1", "text": "Embeddings map text to dense vectors."},
]
records = []
for doc in docs:
resp = client.embeddings.create(
model="text-embedding-3-small", input=doc["text"]
)
records.append({
"id": doc["id"],
"values": resp.data[0].embedding,
"metadata": {"text": doc["text"]},
})Each iteration produces one Pinecone-ready record with an ID, the float vector, and a metadata dict. Storing the source text in metadata lets you return it alongside search results without a second lookup.
records[1]['values'] is a list of 1536 floats — the dense vector for the second document. It's never a string or integer; the OpenAI SDK always returns a Python list of float values.
BATCH_SIZE = 100 def chunk_records(records, size): # TODO: yield successive slices of `records`, # each of length `size`. # Hint: use range(0, len(records), size) # and list slicing. pass for batch in chunk_records(records, BATCH_SIZE): index.upsert(vectors=batch) # index from module 1
Stop — attempt chunk_records before revealing the answer. The function body is the crux of this module: it controls how many records hit Pinecone per call.
Completed function (changed lines marked ►):
def chunk_records(records, size):
► for i in range(0, len(records), size):
► yield records[i : i + size]
Why it matters: a single upsert call with thousands of records can exceed Pinecone's per-request payload limit (~2 MB) and time out or return a 413 error. Batches of 100 stay well under the limit and let you retry a failed batch without re-sending everything.
You can now embed documents, structure each as (id, values, metadata), and slice records into safe batches.
For an SEO/GEO page, each page title becomes a 1536-d vector. Store it with metadata like {"url": "...", "title": "...", "category": "..."} for semantic lookup.
Batches are prepared but not yet written to . Next, call index.upsert() with batched records. Handle partial failures and confirm success with index.describe_index_stats().
| Option | Output dimension | Retrieval quality | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| text-embedding-3-small | 1536 | Strong (MTEB ~62) | Default choice for most RAG pipelines — low cost, fast, and 1536-d fits a standard Pinecone index. | ~$0.02 / 1M tokens | API call only; no GPU needed |
| text-embedding-3-large | 3072 | Best (MTEB ~64) | When retrieval accuracy is the top priority and you can absorb 2× the cost and latency. | ~$0.13 / 1M tokens | API call only; no GPU needed |
| sentence-transformers (local) | Varies (e.g. 384) | Model-dependent | When data must stay on-premise or API costs are unacceptable at your volume. | Compute only; no per-token fee | Requires GPU or CPU inference setup; model download ~90 MB+ |
Three failure patterns cause most ingestion bugs:
Vector dimension 3072 does not match the dimension of the index 1536. You switched from text-embedding-3-small to text-embedding-3-large without re-provisioning. Fix: create a new 3072-d index or stay on the original model.Invalid metadata value at upsert time, not build time. Fix: serialize non-primitive fields to strings first.len(vector) == index_dimension immediately after the first embed call.id is non-empty, values is a float list, and metadata values are primitives.Call index.upsert() with batched records, handle partial failures, and confirm write success with index.describe_index_stats(). You'll complete a retry-safe upsert loop where the error-handling block is left for you to fill in.
How to write vectors into a Pinecone index using batched upserts, namespaces, and post-write verification.
Why this matters: Your SEO page generator needs to store embeddings reliably before it can search them — this module is the write layer that makes retrieval possible.
Decision this forces: Should records be isolated by namespace (multi-tenant) or by separate indexes (strict isolation)?
Module 2 gave you a list of as (id, values, metadata) tuples. This module writes them into a live and confirms the write landed.
The answer to the recall prompt: Pinecone expects each record as a dict with "id", "values", and optionally "metadata". If values has the wrong , the API rejects the entire batch with a dimension-mismatch error.
index.upsert() inserts a record if its is new, or overwrites it if the ID already exists — hence "upsert" (update + insert).
You pass a list of dicts to vectors= and optionally a string to route records into a logical partition. Pinecone recommends batches of up to 100 vectors per call; larger payloads risk hitting the 2 MB request-size limit.
The call returns an UpsertResponse with an upserted_count field. A count lower than your batch size means some records were silently skipped — always check it.
records = [
{"id": "doc-001", "values": embedding_1, "metadata": {"source": "blog"}},
{"id": "doc-002", "values": embedding_2, "metadata": {"source": "faq"}},
]
response = index.upsert(vectors=records, namespace="seo-corpus")
print(response.upserted_count) # expect 2This is the minimal working upsert: a list of record dicts passed to index.upsert() with a to keep this corpus isolated.
Printing upserted_count is the first guard: if it prints 1 instead of 2, one record was silently rejected and you need to inspect the batch.
Pinecone rejects the entire batch and raises a 400 Bad Request error: 'Vector dimension 768 does not match the dimension of the index 1536'. The upsert_count is never returned — the call throws before any records are written. Fix: re-embed with the correct model (text-embedding-3-small → 1536-d) or recreate the index at 768-d.
import time def upsert_in_batches(index, records, namespace, batch_size=100): for i in range(0, len(records), batch_size): batch = records[i : i + batch_size] for attempt in range(3): resp = index.upsert(vectors=batch, namespace=namespace) if resp.upserted_count == len(batch): break time.sleep(2 ** attempt) # exponential back-off else: raise RuntimeError(f"Batch {i} failed after 3 attempts")
This loop slices records into chunks of up to 100, retries each chunk up to three times with exponential back-off, and raises if a batch never fully lands.
The else clause on a for loop runs only when the loop exhausts all iterations without hitting break — Python's way of expressing "all retries failed".
Exactly 10 calls — one per 100-record chunk. Each call returns upserted_count == 100, so the inner retry loop breaks immediately every time and the outer loop advances to the next chunk.
upsert_in_batches(index, all_records, namespace="seo-corpus") stats = index.describe_index_stats() # TODO: print the vector count for the "seo-corpus" namespace # Hint 1: stats has a 'namespaces' dict keyed by namespace name. # Hint 2: each namespace entry has a 'vector_count' field. expected = len(all_records) assert actual_count == expected, f"Expected {expected}, got {actual_count}"
After upserting, call index.describe_index_stats() to confirm the vector count updated — this is the canonical post-write health check.
Your task: replace the TODO with the two lines that read actual_count from stats.namespaces["seo-corpus"].vector_count and print it. Stop and attempt this before revealing.
actual_count = stats.namespaces["seo-corpus"].vector_count
print(f"seo-corpus vector count: {actual_count}")
Changed lines vs. the worked example: we index into stats.namespaces with the namespace key (not stats.total_vector_count, which aggregates ALL namespaces) and read .vector_count off the result. This is the crux: using the namespace-scoped count rather than the global total, which would pass the assert even if records landed in the wrong namespace.
Your SEO page generator upserted 500 chunk embeddings into "seo-corpus". describe_index_stats() confirms vector_count: 500 — the index is live.
A user types a search query. Your app embeds it with the same model. Then index.query() retrieves top-k similar chunks from that namespace. Similarity scores use the from Module 1 — cosine by default.
Module 4 covers querying and similarity search. It walks through embedding the query, setting , filtering by , and reading scored results. It explains why your Module 1 metric choice determines what those scores mean.
| Option | Isolation strength | Query scope control | Operational overhead | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Namespace (shared index) | Logical only — same index, different partition key | Query targets one namespace at a time; cross-namespace queries require multiple calls | Single index to manage, scale, and back up | Multi-tenant apps or multiple corpora that share the same embedding model and dimension — e.g. per-user document stores in a RAG pipeline. | Lower — shared index capacity; no extra index cost per tenant. | Low — one index to provision and monitor. |
| Separate Index | Physical — separate storage and compute per index | Each index is fully independent; no risk of cross-tenant leakage | N indexes to provision, monitor, and pay for | Strict data-residency, compliance, or security requirements where tenants must never share infrastructure — e.g. regulated healthcare or finance data. | Higher — each serverless index has its own billing and storage footprint. | High — one index per tenant to provision, monitor, and pay for. |
Three failures account for most broken upsert pipelines. Each has a distinct fingerprint.
"Vector dimension 768 does not match the dimension of the index 1536". Cause: embedding model changed or wrong model used. Fix: re-embed with the matching model or recreate the index."Missing required field: id" for blank IDs; silent overwrite for duplicates. Cause: ID generation logic is inconsistent. Fix: assert all IDs are non-empty strings before upserting. Use stable hash (SHA-256 of source URL + chunk index) so re-runs overwrite rather than duplicate."Metadata size exceeds 40KB limit". Cause: storing full document text in instead of a reference. Fix: store only filterable fields (URL, category, date) in metadata. Keep full text in a separate store."id" key. (3) Metadata values are scalars or scalar lists — not nested dicts or text blobs. (4) Batch loop checks upserted_count == len(batch) rather than silently continuing on short count.Embed a query string, call index.query() with top_k and include_metadata=True, and interpret the scored results. You'll revisit the metric choice from Module 1 to explain why a dot-product index returns different rankings than a cosine index for the same vectors.
How to embed a query string, call index.query(), and interpret the scored results to retrieve the most relevant vectors from a Pinecone index.
Why this matters: This is the retrieval step that powers your RAG pipeline — without a well-tuned query, the LLM gets the wrong context and produces wrong answers.
Decision this forces: What top_k value and score threshold should you set to balance recall and precision for your use case?
index.upsert() call, which method confirms vectors landed in the index?Answer: index.describe_index_stats() returns the total vector count per . If the count didn't increase, the silently failed — usually a batch exceeding 4 MB or a dimension mismatch.
Now those vectors are stored. This module covers the other side: sending a vector and getting closest matches back. How does Pinecone decide what's 'close'?
At query time, embed your search string with the same model used at ingest. Pass that vector to index.query(). Pinecone computes similarity between your query vector and every stored using the you chose when creating the .
It returns the top matches ranked by score. Each carries a , a score, and optionally the you attached at upsert time.
For , scores range from –1 to 1. Semantically related text lands between 0.75 and 0.95; unrelated text sits below 0.5. Dot-product scores are unbounded — higher still means 'closer', but absolute value depends on vector magnitude.
Cosine similarity measures the angle between vectors, ignoring length. Dot-product measures both angle and magnitude. A longer vector scores higher even if pointing the same direction.
If you chose dot-product and your embedding model doesn't normalize, high-frequency documents rank above semantically closer but shorter ones. text-embedding-3-small outputs unit-normalized vectors, so cosine and dot-product rank identically. Sentence-transformers models often don't normalize by default.
Imagine you've built a assistant over a product documentation corpus. A user types: "How do I reset my API key?"
Your app embeds that question with text-embedding-3-small (the same model used at ingest), then calls index.query() with top_k=5 and include_metadata=True. Pinecone returns five doc chunks, each with a cosine score.
The top result scores 0.91 — the exact "Reset API Key" section. Results 4 and 5 score 0.61 and 0.58 — generic account pages that mention keys but aren't about resetting them. You pass only the chunks above 0.70 to the , cutting noise without losing the relevant context.
This is the core loop: embed → query → threshold → generate. The threshold (here 0.70) is the judgment call this module teaches you to make.
import openai, pinecone client = openai.OpenAI(api_key="OPENAI_KEY") pc = pinecone.Pinecone(api_key="PINECONE_KEY") index = pc.Index("docs-index") query_text = "How do I reset my API key?" resp = client.embeddings.create(model="text-embedding-3-small", input=query_text) query_vec = resp.data[0].embedding results = index.query(vector=query_vec, top_k=5, include_metadata=True) for match in results["matches"]: print(match["id"], round(match["score"], 3), match["metadata"].get("source"))
This embeds the query string with the same model used at ingest, then calls index.query() to retrieve the five nearest vectors.
Each match in results["matches"] carries an id, a score (cosine similarity, 0–1 for normalized vectors), and the dict you stored at upsert time.
doc-42 0.873 reset-key.md
The three fields are printed in order: id, score rounded to 3 decimal places, and the 'source' metadata value. If metadata is missing the 'source' key, .get() returns None instead of raising a KeyError.
MIN_SCORE = 0.70 results = index.query(vector=query_vec, top_k=10, include_metadata=True) relevant = [ m for m in results["matches"] if m["score"] >= MIN_SCORE ] context_chunks = [m["metadata"]["text"] for m in relevant] print(f"{len(relevant)} chunks passed threshold (of 10 retrieved)")
Retrieve more candidates than you need (top_k=10), then filter in Python to keep only high-confidence matches.
This two-step pattern — over-fetch then threshold — gives you control without a second round-trip to Pinecone. The MIN_SCORE constant is the tunable knob: raise it for precision, lower it for recall.
relevant = [
m for m in results["matches"]
if m["score"] >= 0.75
and m["metadata"].get("doc_type") == "faq"
]
# Changed lines vs Stage 2:
# - top_k=15 in the query call (over-fetch more)
# - score threshold raised to 0.75
# - added 'and m["metadata"].get("doc_type") == "faq"' to the filter
# The metadata check is the crux: it narrows by document type before the LLM sees any context.
You upserted with text-embedding-3-small (1536-d) but query with sentence-transformers (768-d). Pinecone raises:
PineconeApiException: Vector dimension 768 does not match index dimension 1536
Fix: embed queries with the exact same model and version used at ingest. Store the model name in your config.
If you created the index with metric='dotproduct' but your model doesn't normalize, high-magnitude docs rank above semantically closer ones. No error — scores look plausible but the top result is wrong. Verify by checking index.describe_index_stats() for the metric field.
With top_k=3 and a large index, the correct chunk may rank 4th or 5th. The LLM answers from incomplete context — no error, just wrong answers. Start with top_k=10 and tighten after measuring recall.
include_metadata=True is present if needed, (3) score threshold is applied after fetching, (4) index name matches your provisioned index.Attach metadata fields (strings, numbers, booleans, lists) to vectors at upsert time, then pass a filter dict to index.query() to restrict results — for example, returning only documents where category == 'legal' and year >= 2023. You'll complete a filter expression for a multi-condition query.
How to attach metadata fields to Pinecone vectors at upsert time and write filter expressions that restrict query results by field equality and range conditions.
Why this matters: Metadata filtering lets your SEO/GEO page retrieval return only the right content category and date range — preventing irrelevant results from polluting the LLM's context.
Decision this forces: Which fields belong in metadata (filterable) versus in the returned payload only, given Pinecone's metadata size limits?
index.query() with include_metadata=True. What does Pinecone return for each match — and what does it not filter on by default? Answer from memory, then read on.By default, index.query() returns the top-k nearest vectors across your entire index — no restriction by document type, date, or any other field.
That's fine for a single corpus, but real applications mix content: legal docs alongside marketing copy, 2019 filings alongside 2024 ones.
This module adds the missing piece: a filter dict that restricts which vectors are even considered before the similarity ranking runs.
Every record you into can carry a dict alongside its vector values and ID.
Supported field types are strings, numbers, booleans, and lists of strings — Pinecone indexes these for fast filtering.
There is a 40 KB per-vector metadata size limit. Fields that exceed it cause the upsert to fail, so large blobs (full document text, base64 images) belong in external storage, not here.
The key design question is: which fields need to filter results, and which are just payload you want returned with the match? Only filterable fields need to live in metadata.
| Option | Filter support | Metadata size impact | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Metadata field (filterable) | Fully supported — equality, range, $in, $nin operators | Each field consumes metadata budget; many large fields risk hitting 40 KB cap | When you need to restrict query results by this field — e.g. category, year, language, is_published. | Counts toward the 40 KB per-vector limit; high-cardinality string fields bloat index size. | Low — add key/value to the metadata dict at upsert time. |
| External store (payload-only) | Not filterable in Pinecone — must post-process in application code | Zero impact on Pinecone metadata budget | When the field is large (full text, HTML, base64) or never used as a filter — store the vector ID as a foreign key and fetch the payload separately. | No metadata budget consumed; adds a round-trip for payload retrieval. | Medium — requires a second lookup against your own DB or object store. |
Pinecone uses a approach: the metadata filter runs before the approximate nearest-neighbor search, not after.
Only vectors satisfying the filter are candidates for ranking. You always get up to results from the matching subset.
Post-filtering (used elsewhere) runs ANN first, then discards non-matches. It can return fewer than top_k results if many top candidates fail the filter.
Pinecone's pre-filter guarantees result count but slows down when the filter matches a tiny fraction of the index.
records = [
{
"id": "doc-001",
"values": embedding_doc1, # 1536-d float list
"metadata": {
"category": "legal",
"year": 2024,
"is_published": True,
},
},
]
index.upsert(vectors=records, namespace="contracts")Each record passes a metadata dict alongside its id and values — this is the only moment you can attach filterable fields.
Note that year is an integer, not a string — matching the numeric range operator you'll use in the query.
Pinecone returns 0 results and raises no error. The filter operator $gte expects a number; the stored string type never matches, so every record is silently excluded from the candidate set.
results = index.query( vector=query_embedding, top_k=5, namespace="contracts", include_metadata=True, filter={ "category": {"$eq": "legal"}, # TODO: add a range condition so only docs from 2023 onward are returned }, ) for match in results["matches"]: print(match["id"], match["score"], match["metadata"])
The filter dict is passed directly to index.query() — Pinecone applies it as a pre-filter before the ANN search runs.
The category equality condition is already in place; your job is to add the year range condition on the TODO line.
Replace the TODO with:
"year": {"$gte": 2023},
Changed line: the year field uses the $gte operator with an integer value.
Why it matters: using "2023" (string) instead of 2023 (int) would silently return 0 results — the type must match what was stored at upsert time.
Full filter:
{
"category": {"$eq": "legal"},
"year": {"$gte": 2023},
}
Three failure patterns cause most production bugs with :
year as a string ("2023") but filter with $gte: 2023 (integer). Pinecone returns 0 results silently. Fix: enforce numeric types at ingest.category field and you filter category == 'legal', those vectors are silently excluded.Before trusting AI-generated filter code, run these four checks:
index.fetch(["doc-001"]) and compare.$eq, $gte, $in — not SQL-style >= or AND. Misspelled operators silently match nothing.With reliable filtering in place, wire this retrieval step into a full loop. Pass filtered top-k chunks directly into an completion call to build a grounded, source-cited answer.
Connect Pinecone retrieval to an OpenAI chat completion call to build a working RAG loop: embed the user question, query Pinecone for top-k chunks, inject them into the prompt, and return a grounded answer. You'll also handle index scaling decisions — when to add replicas, switch pod types, or use serverless auto-scaling — and audit the pipeline for the failure modes introduced in earlier modules.
Wires Pinecone retrieval into a full RAG loop with an LLM, covers index scaling choices, and audits the pipeline for the top failure modes.
Why this matters: This is the capstone integration step — it turns everything built in earlier modules into a working, production-aware RAG chatbot.
Decision this forces: Serverless auto-scaling vs. pod-based index with fixed replicas — which fits your query volume and latency budget?
index.query() to restrict results to a specific category, and what does Pinecone evaluate it against?Answer: you pass a filter dict (e.g. {"category": {"$eq": "legal"}}). Pinecone evaluates it against the stored with each vector at time. Only matching vectors enter the similarity ranking.
Filtered retrieval is the last piece before the generator. This module wires it into a full loop: embed the question, query , inject top chunks into a prompt, and call an . Then decide how to scale the index when traffic grows.
A complete pipeline has four steps per user turn: embed the question, retrieve nearest chunks from , inject chunks as context into a prompt, and call the to generate a grounded answer.
The retrieval step returns scored matches. Concatenate their text fields and place them in the system or user message before the question. The LLM is instructed to answer only from supplied context and cite sources. This prevents hallucination.
Two constraints govern design: the model at query time must match the ingest model (same and ). Total token length of injected chunks must stay inside the LLM's context window.
import openai def embed_query(text: str) -> list[float]: resp = openai.embeddings.create( model="text-embedding-3-small", input=text ) return resp.data[0].embedding query_vec = embed_query("What is Pinecone's serverless pricing model?")
This produces the 1536-dimensional query vector using the same model used during ingest — dimension and metric alignment is non-negotiable. Switching models between ingest and query is the single most common silent failure in RAG pipelines.
embed_query returns a list of 1536 floats. If ingest used ada-002 (also 1536-d) the dimension matches but the vector space differs — Pinecone returns results with plausible-looking scores that are semantically meaningless. There is no error; the pipeline runs and returns wrong answers. If ingest used a 768-d model, Pinecone raises a dimension mismatch error on query.
def rag_answer(question: str, index, top_k: int = 5) -> str: q_vec = embed_query(question) results = index.query( vector=q_vec, top_k=top_k, include_metadata=True, filter={"status": {"$eq": "published"}} ) chunks = "\n---\n".join( m["metadata"]["text"] for m in results["matches"] ) resp = openai.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": f"Answer using only this context:\n{chunks}"}, {"role": "user", "content": question} ] ) return resp.choices[0].message.content
This is the full RAG loop in one function: embed → query Pinecone with a → concatenate chunk text → call the LLM with context in the system message. The filter here restricts to published documents — the same pattern from Module 5, now wired into the generator.
5 × 400 = ~2 000 tokens for context, well within gpt-4o-mini's 128 k window. At top_k=50 that's ~20 000 tokens — still fits, but cost scales linearly with input tokens and latency rises. Beyond a certain point, LLMs lose focus on middle-of-context chunks (the 'lost in the middle' effect), so more chunks does not always mean better answers.
These are the failure modes that survive into production — each has a distinct symptom that tells you which layer broke.
index.upsert() with the same to overwrite, or index.delete() first if the chunk count changed.openai.BadRequestError: maximum context length exceeded. Cause: top_k × chunk_tokens exceeds the model's limit. Fix: cap injected tokens explicitly (count with tiktoken before building the prompt) or reduce top_k; never rely on the model to truncate gracefully.def rag_answer_tenant( question: str, index, tenant_id: str, top_k: int = 5 ) -> str: q_vec = embed_query(question) results = index.query( vector=q_vec, top_k=top_k, include_metadata=True, # TODO: add the two arguments that isolate this tenant # and restrict results to status == "published" ) chunks = "\n---\n".join( m["metadata"]["text"] for m in results["matches"] ) resp = openai.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": f"Answer using only:\n{chunks}"}, {"role": "user", "content": question} ] ) return resp.choices[0].message.content
This is a variation of the Stage 2 function — the only change is multi-tenant isolation. Fill in the TODO before revealing the answer.
Replace the TODO with:
namespace=tenant_id,
filter={"status": {"$eq": "published"}}
Changed lines vs Stage 2: (1) namespace=tenant_id is NEW — it scopes the search to only this tenant's vectors, preventing namespace bleed. (2) filter is unchanged but must still be present; omitting it would return unpublished chunks within the tenant's namespace. Together these two arguments are the crux of multi-tenant RAG safety.
With the full pipeline working and the index scaling decision made, you're ready for the capstone: build a complete RAG chatbot end-to-end — ingest a real corpus, wire the query loop, add tenant namespaces, and stress-test it against the failure modes from every module.
| Option | Latency predictability | Traffic pattern fit | Operational control | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Serverless auto-scaling | Variable; cold starts add latency spikes under sudden load | Excellent for bursty or low-and-growing traffic | Minimal — Pinecone manages all scaling decisions | Spiky or unpredictable query volume; early-stage products where traffic is unknown; cost-sensitive workloads that can tolerate occasional cold-start latency. | Pay-per-read-unit; near-zero cost at idle. | Low — no replica or pod-type configuration needed. |
| Pod-based with replicas | Consistent; replicas distribute load and eliminate cold starts | Best for steady, high-throughput workloads | Full control over pod type, replicas, and shards | Sustained high query volume with a strict p99 latency SLA; production systems where cold-start variance is unacceptable. | Fixed hourly per pod; replicas multiply cost linearly. | Medium — choose pod type (s1, p1, p2) and replica count; right-sizing requires load testing. |
Before trusting AI-generated Pinecone + LLM integration code, run through this checklist. Each item maps to a failure mode above.
index.describe_index_stats() and assert it equals the model's output size.namespace= in both upsert and query calls. Flag any call that omits it.Before reading the summary: from memory, list the six build steps in order — from provisioning the index to serving a RAG response — and name the one parameter you must lock in at index creation that cannot be changed later. Then check your answer against the buildOrder below.
Apply what you learned to SEO/GEO page — the page title is the H1 and the exact search query. OPEN with a 40-60 word self-contained, quotable answer capsule that names the entity explicitly in sentence one (no "in this lesson…" preamble) — this is the sentence an LLM lifts. Name the entity by its full name every time (not pronoun-only). Structure each section as ONE sub-question (How it works · X vs Y · When to use it · Example · Common mistakes), each opening with its own 1-2 sentence direct answer then depth. End with a genuine 3-5 question FAQ phrased as real user questions ending in "?" (these become FAQPage schema + the knowledge check). Prefer quotable specifics — concrete numbers, defaults, version names, one short snippet — over vague prose. Self-contained for a reader who arrived cold from a search engine or an LLM. Intent = tutorial: numbered steps + one short runnable code snippet + a common-mistakes section. Target query: "pinecone vector database"..
You are creating a Pinecone serverless index for OpenAI text-embedding-3-small, which produces 1536-dimensional vectors and is trained with cosine similarity. Which index configuration is correct?
A) dimension=768, metric="euclidean"
B) dimension=1536, metric="cosine"
C) dimension=1536, metric="dotproduct"
D) dimension=3072, metric="cosine"
text-embedding-3-small outputs 1536 dimensions and is optimized for cosine similarity, so dimension=1536 and metric="cosine" is the only correct pairing. 768 is the dimension for smaller models like all-MiniLM-L6-v2, not this model. dotproduct is valid for normalized vectors but is not the recommended metric for text-embedding-3-small and can produce unexpected ranking if vectors are not unit-normalized. 3072 is the dimension for text-embedding-3-large, a different model entirely.
You have 50,000 text chunks to upsert into Pinecone. Your code sends all 50,000 vectors in a single upsert call and you receive a payload size error. What is the correct fix, and why does the original approach fail?
Pinecone enforces a per-request payload size limit (roughly 2 MB or ~100 vectors per call for typical 1536-dim vectors). Batching into loops of ~100 records each is the standard fix. The index type (serverless vs. pod) does not change per-request limits. Reducing dimension changes the embedding model's output and breaks dimension alignment with the index — it does not bypass payload limits. Namespaces isolate data within an index but have no effect on payload size constraints.
Read this short query code:
results = index.query(vector=query_embedding, top_k=5)
for r in results["matches"]:
print(r["id"], r["score"])
The scores returned are all between 0.91 and 0.95, but the retrieved documents are clearly off-topic. What is the most likely root cause?
When the query embedding and the stored embeddings come from different models, the vector spaces are incompatible. Cosine similarity measures geometric angle, not meaning, so scores can appear high even when documents are semantically unrelated — this is the embedding model mismatch failure mode. Increasing top_k retrieves more candidates but does not fix a model mismatch. High cosine scores are only meaningful when both sides use the same model. Metadata filters narrow results by field values but cannot compensate for a broken embedding space.
Your SaaS product stores vectors for multiple customers in one Pinecone index. You need to ensure Customer A's queries never return Customer B's vectors. Which approach does Pinecone provide for this, and when would you choose a separate index instead?
Pinecone namespaces provide logical isolation within a single index: a query scoped to namespace "customer-A" will never return vectors from namespace "customer-B". This is the recommended multi-tenant pattern and is cost-efficient. Separate indexes are warranted only when tenants need different dimensions, metrics, or strict infrastructure separation (e.g., compliance requirements). Metadata filters on customer_id can work but are slower than namespace scoping and are not the primary isolation mechanism. top_k controls result count and has nothing to do with tenant isolation.
In a RAG pipeline using Pinecone, you retrieve the top 20 chunks (each ~500 tokens) and pass all of them as context to an LLM with an 8,192-token context window. Describe the failure mode this causes and state two concrete ways to fix it.
Context window overflow is one of the top RAG failure modes covered in the final module. The fix requires either reducing top_k, applying a score threshold to filter low-relevance chunks, or truncating/summarizing chunks before insertion. Any two of these concrete strategies constitute a correct answer. Vague answers like 'use a smaller model' do not address the retrieval design problem.