Qdrant is a purpose-built vector database that stores embeddings as points inside collections and retrieves the nearest neighbors fast — with payload…
Trace why a dedicated vector database outperforms bolt-on solutions, and map Qdrant's four-layer architecture (storage → index → filter → API) using a semantic search scenario as the running example throughout this lesson.
Explains what Qdrant is, why a dedicated vector database outperforms bolt-on solutions, and maps its four-layer architecture.
Why this matters: Gives you the foundation to decide when Qdrant is the right tool and how its internals make filtered semantic search fast — essential before building any collection or query.
Decision this forces: Qdrant vs. pgvector vs. Milvus — which dedicated vector service fits the workload?
Your semantic search pipeline embeds queries and needs the 10 most similar documents from millions — fast, with metadata filters applied simultaneously. General-purpose databases weren't built for this workload.
is an open-source built for retrieval over high-dimensional . It stores vectors as in with JSON metadata called . It applies inside the ANN search, not after.
The key difference: filtering and indexing are co-designed. Qdrant prunes the candidate graph using payload conditions before scoring. You get low latency even with selective filters.
Qdrant has four layers, each with a distinct job.
The filter layer separates Qdrant from plain HNSW: filtering is a first-class citizen of the search graph, not a post-filter on results.
You're building semantic search for a documentation site. Users type questions; you need the top-5 most relevant docs filtered to a product category — all under 50 ms.
Each document embeds into a (e.g., 1536 dimensions from OpenAI's text-embedding-3-small). Vectors plus a payload like {"category": "billing", "url": "...", "ts": 1718000000} upsert into a Qdrant .
At query time, Qdrant traverses the HNSW graph while checking the category payload index. Only nodes passing the filter are scored by . Result: filtered ANN search in one round-trip.
This scenario threads through all six modules. You'll build it piece by piece, starting with the collection and payload schema next.
import psycopg2 conn = psycopg2.connect("dbname=docs") cur = conn.cursor() # Exact scan — no vector index, no extension cur.execute(""" SELECT id, content FROM documents ORDER BY embedding <-> %s -- operator not found! LIMIT 5 """, (query_vector,))
This is the obvious first attempt: store embeddings in Postgres and query with a distance operator. Predict what happens before revealing the result.
ERROR: operator does not exist: double precision[] <-> double precision[]
HINT: No operator matches the given name and argument types.
Even with pgvector installed and an HNSW index added, a corpus of 1M rows with a selective WHERE clause forces Postgres to post-filter ANN results — rows that fail the filter are discarded after scoring, so you waste work and may return fewer than 5 results. Qdrant's filter-inside-HNSW design avoids both problems.
| Option | Filtered ANN (filter inside graph) | Operational simplicity | Hybrid dense + sparse search | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Qdrant | Native — payload index prunes HNSW graph during traversal | Single binary, no Zookeeper or external metadata store | Supported — dense + sparse vectors in one collection | Vector search is the primary workload; you need fast filtered ANN, multi-tenancy, or hybrid dense+sparse search with a dedicated service boundary. | Free OSS; Qdrant Cloud from ~$0.014/hr for managed | Low-medium — single binary or Docker image; no external dependencies |
| pgvector | Post-filter only — WHERE applied after ANN candidates are scored | Lowest — lives inside existing Postgres; no new service | Not natively supported; requires workarounds | Postgres is already your primary store and the vector corpus is not the dominant scaling concern; you want one fewer service to operate. | Free; runs on existing Postgres instance | Very low — just a Postgres extension; no new infra |
| Milvus | Supported but requires explicit index configuration | High complexity — distributed components, external metadata store | Supported via sparse vector fields | Billion-scale vector corpora with distributed horizontal scaling; you have the infra team to manage Kubernetes and etcd dependencies. | Free OSS; significant infra overhead | High — requires etcd, MinIO/S3, and Kubernetes for production |
Three failure patterns catch teams early — each with a concrete symptom to watch for.
You create a collection with Distance.EUCLID but your embedding model (e.g. OpenAI text-embedding-3-small) expects cosine similarity. Symptom: search returns results, but ranking is wrong — the top hit is semantically unrelated to the query. No error is raised; you only catch it by inspecting scores.You filter on category without creating a payload index for it. Qdrant falls back to a full payload scan on every search. Symptom: p99 latency climbs linearly with collection size; the query still returns correct results, so it's easy to miss until load testing. a point whose vector has a different dimension than the collection was created with. Qdrant returns 400 Bad Request: Wrong input: Vector dimension error. Fix: always pin the embedding model version and validate vector length before upserting.When you use an AI assistant to scaffold Qdrant collection or search code, check these four things before trusting it.
AI-generated code often omits create_payload_index() — check that each field used in a filter has an explicit index call.You now have the mental model: Qdrant's four layers (storage → index → filter → API) make filtered ANN search a single, fast operation.
Reach for Qdrant when vector search is the primary workload and you need filter-inside-graph performance or hybrid search.
Next, you'll create a collection tied to an embedding model and distance metric. Then upsert points with stable IDs and JSON payloads carrying source URL, category, and timestamp — the schema the semantic search scenario will query throughout this lesson.
Create a collection tied to a specific embedding model and distance metric, upsert points with stable IDs and JSON payloads (source URL, category, timestamp), and add payload indexes so filters stay fast at scale.
How to create a Qdrant collection, upsert points with structured payloads, and add payload indexes for fast filtered search.
Why this matters: Every SEO page you index is a point in a collection — getting the vector size, distance metric, and payload indexes right determines whether your semantic search is accurate and fast at scale.
Module 1 traced the stack: storage → index → filter → API. Every operation lands in one of those layers.
This module zooms into the storage and filter layers: how you shape incoming data and make filters fast at scale.
A locks in two things at creation: vector dimension and . You cannot change either without recreating the collection.
Each has three parts: a stable ID, a from your model, and JSON (metadata like URL, category, timestamp).
The payload travels with the point through every search result. You get metadata back without a second lookup.
Without a , Qdrant evaluates a filter by scanning every candidate the graph returns — fine for thousands of points, painful for millions.
A payload index on a field (e.g. category or timestamp) lets Qdrant pre-filter the candidate set before scoring, so the ANN search only touches points that can pass the filter.
Index the fields you filter on in queries; skip fields you only read back in results.
You're building a semantic search layer for an SEO/GEO page generator. Each page is embedded and stored as a point so the system can retrieve the most relevant pages for a given query.
The payload for each point carries: source_url (the canonical URL), category (e.g. "product", "blog"), and published_at (Unix timestamp for recency filtering).
At query time you filter by category and a published_at range, so both fields need payload indexes. source_url is only read back in results — no index needed there.
from qdrant_client import QdrantClient from qdrant_client.models import Distance, VectorParams client = QdrantClient(url="http://localhost:6333") client.recreate_collection( collection_name="seo_pages", vectors_config=VectorParams(size=1536, distance=Distance.COSINE), )
This creates a collection sized for OpenAI's text-embedding-3-small (1536 dimensions) with cosine distance — the right metric for unit-normalized embeddings.
recreate_collection drops and rebuilds if the collection already exists, which is safe during development but destructive in production — prefer create_collection with an existence check there.
The second call silently drops the existing collection and creates a fresh one — all points are lost. No error is raised. In production, use create_collection and catch CollectionAlreadyExistsError instead.
from qdrant_client.models import PointStruct points = [ PointStruct( id=1, vector=embed("Best running shoes for flat feet"), # returns list[float] payload={"source_url": "https://example.com/shoes", "category": "product", "published_at": 1718000000}, ), ] client.upsert(collection_name="seo_pages", points=points)
inserts a new point or overwrites an existing one with the same ID — safe to call repeatedly without duplicating data.
The ID is a stable integer here; UUIDs also work. Stable IDs let you overwrite a point when a page is re-crawled without scanning for it first.
Only the second version. Qdrant replaces the entire point (vector + payload) on a matching ID — the first payload is gone. This is the intended behavior for re-indexing updated pages.
from qdrant_client.models import PayloadSchemaType # Index the category field for keyword filtering client.create_payload_index( collection_name="seo_pages", field_name="category", field_schema=PayloadSchemaType.KEYWORD, ) # TODO: index published_at for range filtering # Hint: use field_schema=PayloadSchemaType.??? — check which type supports gt/lt range queries
The category index is done. Your task: add the matching call for published_at so range filters on timestamps stay fast at scale.
Stop — attempt this before revealing. Which PayloadSchemaType supports gt/lt range queries on a numeric Unix timestamp?
Use field_schema=PayloadSchemaType.INTEGER (or FLOAT for fractional timestamps). KEYWORD indexes store values as strings and only support exact-match filters — a gt/lt range query on a KEYWORD-indexed field falls back to a full scan, silently returning correct results but with no speed benefit. The changed lines: field_name="published_at", field_schema=PayloadSchemaType.INTEGER.
Bad input: wrong vector size error. Cause: collection size=768 but model outputs 1536. Fix: check model_card.embedding_dim before recreate_collection.Distance.EUCLID with normalized vectors. Fix: match metric to model's training objective.client.get_collection("seo_pages") and inspect payload_schema.size matches the model's actual output dimension.distance matches the model card's recommended metric.create_payload_index call with the right schema type.client.retrieve() to confirm payload round-trips before batch ingestion.The next module covers Vector Storage and HNSW Indexing. It shows how Qdrant writes points to disk and builds the graph for fast nearest-neighbor search. Understanding that layer explains why collection design choices have lasting performance consequences.
Trace how Qdrant writes vectors to segments on disk, builds an HNSW graph for approximate nearest-neighbor search, and applies quantization to cut memory use — using the same semantic search collection from Module 2.
Explains how Qdrant writes vectors to disk as segments, builds an HNSW graph index, and applies quantization to cut memory use.
Why this matters: Getting indexing and quantization right determines whether your semantic search collection is fast and affordable at scale — or slow and expensive.
Decision this forces: Full precision vs. quantization — and which HNSW parameters to tune for the target recall/latency budget?
Answer: a stable ID, one or more , and a JSON (metadata like source URL, category, timestamp). Module 2 built exactly this structure — now Module 3 asks: once those vectors land on disk, how does Qdrant make searching them fast?
Qdrant stores vectors inside — immutable, self-contained disk files holding collection points.
(Write-Ahead Log), protecting against crash data loss. New points accumulate in a small in-memory segment. The optimizer later merges small segments into larger, indexed ones.
index. It also runs quantization if configured. Until it finishes, segments are searchable but slow — brute-force scan is the most common cause of unexpectedly slow queries after bulk upload.
index that organizes vectors into a multi-layer graph. Upper layers are sparse long-range connections; lower layers are dense short-range ones. A query enters at the top, greedily descends toward the nearest cluster, and refines at the bottom.
m — edges per node in the graph. Higher m means better recall and faster search, but the index takes more RAM. Default: 16.ef_construct — candidate list size during index build. Higher values produce a denser, more accurate graph but slow down ingestion. Default: 100.At query time, ef (search beam width) controls the recall/latency tradeoff independently of the build parameters. Raising ef without rebuilding the index.
from qdrant_client import QdrantClient from qdrant_client.models import Distance, VectorParams, HnswConfigDiff client = QdrantClient(url="http://localhost:6333") client.recreate_collection( collection_name="articles", vectors_config=VectorParams(size=1536, distance=Distance.COSINE), hnsw_config=HnswConfigDiff( m=16, # edges per node — controls graph density ef_construct=128, # build-time beam width — higher = better recall ), )
This recreates the articles collection from Module 2 with explicit HNSW parameters. Passing HnswConfigDiff at creation time means the optimizer uses these values when it builds the index on each segment.
Qdrant falls back to its defaults: m=16 and ef_construct=100. The collection is still indexed — you only need HnswConfigDiff when you want to override those defaults.
Qdrant supports two quantization modes that compress stored vectors to cut RAM and speed up distance calculations.
Both modes store the original full-precision vectors on disk for optional rescoring. Enabling rescore=True at a small latency cost.
from qdrant_client.models import ScalarQuantizationConfig, ScalarType, QuantizationConfig # TODO: pass quantization_config to recreate_collection # Hint: ScalarQuantizationConfig takes type=ScalarType.INT8 and quantile=0.99 client.recreate_collection( collection_name="articles", vectors_config=VectorParams(size=1536, distance=Distance.COSINE), hnsw_config=HnswConfigDiff(m=16, ef_construct=128), quantization_config=??? # <-- your line here )
This is the completion rung — Stage 1 is already wired; your job is to supply the quantization_config argument. Stop and write the missing line before revealing the answer.
quantization_config=QuantizationConfig(
scalar=ScalarQuantizationConfig(
type=ScalarType.INT8,
quantile=0.99,
always_ram=True, # keep quantized vectors in RAM for fast ANN
)
)
# Changed lines vs Stage 1: added quantization_config= block.
# always_ram=True is the key production flag — without it, quantized
# vectors may be paged out, defeating the latency benefit.
| Option | Memory use | Recall at default settings | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| Full Precision (float32) | 4 bytes per dimension — baseline | Exact — no approximation loss | When recall must be exact and RAM is not a constraint — small-to-medium collections or high-stakes retrieval. | Highest RAM (4 bytes/dim) | None — default, zero config |
| Scalar Quantization (SQ8) | ~4× reduction (float32 → uint8) | ~97–99% of full-precision recall | When you need ~4× memory savings with minimal recall loss — the right default for most production collections. | ~1 byte/dim after compression | Low — one config block |
| Product Quantization (PQ) | 8–32× reduction depending on config | Larger drop; rescore=True recommended | When memory is severely constrained (millions of high-dim vectors) and you can tolerate a larger recall drop or will always rescore. | Lowest RAM (8–32× reduction) | Medium — requires tuning m and training data |
client.get_collection('articles').optimizer_status and wait for status ok before benchmarking.m to 32 or 64 and rebuild (requires recreating the collection).rescore=True in search, or disable quantization.m and ef_construct are in the config. LLMs often omit them, leaving silent defaults.always_ram=True on the quantization block if low latency matters. Its absence is a common omission.optimizer_status is ok before treating latency measurement as representative.With storage, indexing, and quantization in place, the next module runs your first real nearest-neighbor query. It layers payload filters on top of the collection.
Run a nearest-neighbor query against the lesson's collection, layer a payload filter to restrict candidates, and compare the three retrieval modes (dense, sparse, hybrid) — completing a partially written search call as the hands-on step.
How to run filtered nearest-neighbor searches in Qdrant and choose between dense, sparse, and hybrid retrieval modes.
Why this matters: Retrieval quality is the core of any SEO/GEO semantic search page — this module gives you the exact query pattern and the decision framework to pick the right search mode for your corpus.
Decision this forces: Dense vs. sparse vs. hybrid retrieval — which mode fits the query type and corpus?
Answer: is an approximate index. It skips exhaustive vector comparison to return results faster. It trades exactness for speed. The metric is — the fraction of true nearest neighbors actually returned. Module 3 showed how to tune it. This module shows what you're searching for and how to narrow the candidate set before HNSW runs.
Your agent embedded a user question and wants the top-5 matching chunks. But only from documents published after a certain date and tagged as 'legal'. How do you stop Qdrant from returning irrelevant results that happen to be geometrically close?
A in accepts both a query and a object in the same request. Qdrant evaluates the filter against fields. It restricts the candidate set before scoring vectors.
This works efficiently because of — optional indexes you created in Module 2. Without them, Qdrant post-filters: it scores all vectors first, then discards failures. This wastes work and can silently return fewer than top_k results.
With a payload index on the filtered field, Qdrant uses pre-filtering. It narrows the candidate set before HNSW traversal. The graph only visits points that can pass the condition.
You're building a semantic search layer for an SEO/GEO page corpus. Each in the carries a payload with category, published_ts, and source_url — the same fields upserted in Module 2.
A user searches for 'best on-page SEO tactics'. You embed the query and want the top-5 chunks from the 'seo' category published after January 2024. Without a filter, a chunk about 'PPC bidding strategies' from 2019 might rank second — geometrically close but contextually wrong.
Adding a must condition on category == 'seo' and published_ts >= 1704067200 (Unix epoch for 2024-01-01) restricts the HNSW graph walk to only qualifying points. The payload index on category makes this pre-filter fast even across millions of points.
results = client.search( collection_name="seo_pages", query_vector=query_embedding, # list[float] from your embedder limit=5, ) for r in results: print(r.id, r.score, r.payload["source_url"])
This is the minimal search call against the lesson's seo_pages collection — no filter yet, so Qdrant scores all indexed points by cosine similarity to query_embedding and returns the top 5.
Each result carries id, score (0–1 for cosine), and the full . Notice there is no payload index benefit here — the next stage adds the filter that makes the index matter.
Qdrant accepts it and returns results, but the cosine similarity of a zero vector is undefined (division by zero). In practice Qdrant may return arbitrary ordering or raise a server-side error depending on version — never pass a zero vector. Always assert len(query_embedding) > 0 and that the norm is non-zero before calling search.
from qdrant_client.models import Filter, FieldCondition, MatchValue, Range results = client.search( collection_name="seo_pages", query_vector=query_embedding, query_filter=Filter( must=[ FieldCondition(key="category", match=MatchValue(value="seo")), FieldCondition(key="published_ts", range=Range(gte=1704067200)), ] ), limit=5, )
The Filter object is passed as query_filter — Qdrant evaluates both must conditions against the before the HNSW walk, so only qualifying points are ever scored.
This is the delta from Stage 1: the query_filter argument and the two FieldCondition entries. Everything else stays the same.
Results are still correct — Qdrant falls back to post-filtering (score all vectors, then discard). But performance degrades: Qdrant may scan far more candidates than limit=5 requires, and if the filter is very selective you can silently receive fewer than 5 results because the post-filter discards most of the top-k pool.
# Variation: hybrid search — dense + sparse legs, filtered to category='geo' # The collection has a named sparse vector field: "text_sparse" results = client.search_batch( collection_name="seo_pages", requests=[ models.SearchRequest( vector=models.NamedVector(name="text_dense", vector=query_embedding), filter=Filter(must=[FieldCondition(key="category", match=MatchValue(value="geo"))]), limit=5, ), models.SearchRequest( vector=models.NamedSparseVector(name="text_sparse", vector=# TODO: sparse_vector), filter=Filter(must=[FieldCondition(key="category", match=MatchValue(value="geo"))]), limit=5, ), ], )
Stop — attempt this before revealing. The dense leg is complete; your job is to fill in the TODO for the sparse leg.
Hints: (1) NamedSparseVector takes a vector argument that is a SparseVector object with indices and values. (2) Your sparse encoder returns exactly those two lists.
# CHANGED LINE — this is the crux:
vector=models.SparseVector(indices=sparse_indices, values=sparse_values)
# Full corrected sparse request:
models.SearchRequest(
vector=models.NamedSparseVector(
name="text_sparse",
vector=models.SparseVector(indices=sparse_indices, values=sparse_values),
),
filter=Filter(must=[FieldCondition(key="category", match=MatchValue(value="geo"))]),
limit=5,
)
# Why: NamedSparseVector wraps a SparseVector (not a plain list) because sparse
# vectors store only non-zero positions. The filter is identical to the dense leg —
# both legs must agree on the candidate set before fusion.
| Option | Query type fit | Exact keyword match | Recall on long-tail terms | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Dense vector search | Natural language, paraphrases, semantic intent | Weak — synonyms score high, exact terms may not | Poor for rare proper nouns or codes | Use when queries are natural-language or paraphrased — semantic similarity matters more than exact wording. | Higher memory: full-dimension float vectors (e.g. 1536 dims). | Low — one vector per point, standard HNSW index. |
| Sparse vector search | Keyword, boolean, exact-match queries | Strong — term weights map directly to BM25-style scoring | Good for rare terms present in the corpus | Use when queries contain exact keywords, product codes, or domain jargon that must match literally. | Lower memory per vector; index size grows with vocabulary. | Medium — requires a sparse encoder (e.g. BM25/SPLADE) and a named sparse vector field. |
| Hybrid search (dense + sparse) | Mixed queries: semantic intent + exact terms | Good — sparse leg handles literals | Best overall — both legs contribute | Use when queries mix natural language with specific terms — the default choice for production RAG and SEO retrieval. | Highest: stores both dense and sparse vectors; doubles index overhead. | Higher — two vectors per point, two index passes, a fusion/rerank step. |
limit=10 but get 3. No error is raised. Qdrant post-filters a fixed candidate pool and runs out. Observable symptom: len(results) < limit even though matching points exist. Fix: add the payload index and re-run.NamedVector(name="dense") when the field is "text_dense" raises: Not existing vector name: dense. Match the name string to the collection schema exactly.assert any(v != 0 for v in query_embedding) before calling search.query_filter is passed as a keyword argument, not positional. SDK signatures changed across versions.FieldCondition key matches a field with a payload index. Run client.get_collection("seo_pages").payload_schema to list indexed fields.NamedVector and NamedSparseVector names match the collection's vector config. Mismatch is a runtime error, not a type error.limit=1 and print r.score. A score of 0.0 or NaN signals a bad query vector.The next module covers API, Client Libraries, and the Upsert-Search Pattern. It wires this search call into a full Python application using . You'll learn to initialize the client, batch-upsert points, and read results end-to-end.
Wire the lesson's collection into a Python application using the qdrant-client library: initialize the client, upsert a batch, run a filtered search, and read back results — with a completion exercise that asks you to add a second filter condition.
Wire a Qdrant collection into Python: initialize the client, upsert a batch of points, run a filtered search, and read back IDs, scores, and payload fields.
Why this matters: This is the hands-on coding layer — the step where your SEO/GEO page application actually talks to Qdrant and retrieves ranked results.
Decision this forces: REST vs. gRPC transport — and local Docker vs. Qdrant Cloud connection string?
Answer: a pre-screens candidates by metadata before vector scoring. The ANN graph only visits points that pass the filter, keeping search fast at scale.
Module 4 showed the conceptual search. This module wires it into Python code using — so you can upsert, query, and read results from your application.
The library exposes QdrantClient — one class handling local and cloud connections with identical method signatures.
For local Docker, pass host and port. For , pass url and api_key. The transport layer — or — is controlled by prefer_grpc.
REST is the safe default: it works everywhere and is easy to inspect with curl. gRPC cuts latency on batch upserts using binary framing instead of JSON. It requires port 6334 and adds a protobuf dependency.
Pick gRPC only after profiling confirms serialization is the bottleneck, not your embedding model or network round-trips.
from qdrant_client import QdrantClient from qdrant_client.models import PointStruct # Local Docker (port 6333) — swap url+api_key for Qdrant Cloud client = QdrantClient(host="localhost", port=6333) points = [ PointStruct(id=1, vector=[0.12, 0.45, 0.78], payload={"category": "seo", "source_url": "https://example.com/a"}), PointStruct(id=2, vector=[0.88, 0.21, 0.34], payload={"category": "geo", "source_url": "https://example.com/b"}), ] client.upsert(collection_name="pages", points=points)
This stage initializes the client and two into the pages collection — the same collection built in Module 2.
Each PointStruct carries a stable integer ID, a vector, and a JSON . Upsert is idempotent: re-running with the same IDs overwrites rather than duplicates.
Qdrant returns an UpdateResult object with status='completed' and an operation_id. Calling upsert again with the same IDs overwrites the existing points in place — no duplicates are created. This is the 'upsert' (update-or-insert) guarantee.
from qdrant_client.models import Filter, FieldCondition, MatchValue results = client.search( collection_name="pages", query_vector=[0.10, 0.44, 0.80], query_filter=Filter( must=[FieldCondition(key="category", match=MatchValue(value="seo"))] ), limit=5, ) for r in results: print(r.id, r.score, r.payload["source_url"])
This stage adds a to the search so only points where category == "seo" are considered — the from Module 2 makes this fast.
Each result in results is a ScoredPoint with three fields you'll use constantly: .id (the stable point ID), .score (cosine or dot-product similarity), and .payload (the full JSON metadata dict).
Only point id=1 passes the filter (category='seo'). Output: '1 <score near 1.0> https://example.com/a'. Point id=2 is excluded before scoring because its category is 'geo'. The exact score depends on the distance metric set when the collection was created.
results = client.search( collection_name="pages", query_vector=[0.10, 0.44, 0.80], query_filter=Filter( must=[ FieldCondition(key="category", match=MatchValue(value="seo")), # TODO: also require source_url == "https://example.com/a" ] ), limit=5, ) for r in results: print(r.id, r.score, r.payload["source_url"])
This is a completion exercise: the first filter condition is already in place; your job is to add the second one inside the must list so only the point with source_url == "https://example.com/a" can be returned.
Changed lines — the second FieldCondition is the new addition:
must=[
FieldCondition(key="category", match=MatchValue(value="seo")),
FieldCondition(key="source_url", match=MatchValue(value="https://example.com/a")), # ADDED
]
Both conditions are ANDed together (that's what 'must' means in Qdrant's filter model). Only point id=1 satisfies both, so the output is:
1 <score> https://example.com/a
If you used 'should' instead of 'must', the conditions would be ORed — a common mistake that returns more results than intended.
With upsert-search in place, your SEO/GEO page application can embed a query, call client.search(), and surface top matching pages — filtered by category, tenant, or any indexed field — in one round-trip.
A realistic production loop: embed the query → call client.search() with a category filter → extract .id, .score, and .payload["source_url"] → rank or rerank → return to user.
Your client code assumes Qdrant is running and configured correctly. The final module covers: starting Qdrant via Docker, setting storage path, tuning WAL and memory-map settings, and diagnosing five common configuration mistakes.
| Option | Latency on large batches | Ease of debugging | Firewall / port requirements | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| REST (default) | Higher — JSON parse overhead | Easy — human-readable JSON | Port 6333 only | Use REST for local dev, small teams, or any environment where you can't guarantee port 6334 is open. | Negligible for typical workloads | Low — no extra deps |
| gRPC (prefer_grpc=True) | Lower — binary framing | Hard — binary protocol | Needs port 6334 open | Use gRPC when batch upsert throughput is the bottleneck and port 6334 is accessible. | Same infra; saves CPU at high volume | Medium — protobuf dependency |
| Local Docker | Lowest — no network hop | Full log access | Localhost only | Use for development, CI, and offline testing — no account needed. | Free | Low — one docker run command |
| Qdrant Cloud | Network-bound | Cloud dashboard + logs | Outbound HTTPS only | Use for production or shared team environments where you need managed infra and an API key. | Paid tiers; free tier available | Low — managed service |
These three failures waste the most debugging time when wiring into an application.
size=1536 but your code sends 768-dim vectors. Qdrant rejects it: ValueError: Wrong input: vector size 768 is not equal to expected 1536. Fix: assert len(vector) == expected_dim before every upsert.collection_info.payload_schema to confirm the index exists.prefer_grpc=True when port 6334 is firewalled produces grpc._channel._InactiveRpcError: StatusCode.UNAVAILABLE. Fix: open the port or drop back to REST — don't retry in a loop.Verifying AI-generated client code: check that collection_name matches exactly (case-sensitive), vector dims match the collection's size, every filtered field has a payload index, and the metric matches your embedding model's training metric.
Run Qdrant via Docker, set the key config options (storage path, memory map threshold, WAL settings), and diagnose the five most common mistakes — wrong vector size, missing payload index, unbuilt index, dimension mismatch, and silent score degradation from quantization.
How to run Qdrant via Docker, set the key production config options, and diagnose the five most common failures.
Why this matters: Deploying Qdrant correctly is the operational foundation your SEO/GEO semantic search pipeline depends on — misconfiguration here silently breaks search quality.
Decision this forces: Local Docker vs. Qdrant Cloud — and which config knobs to change before going to production?
builds a layered graph for fast approximate search. . An unbuilt HNSW index silently falls back to brute force. Aggressive quantization degrades result quality without error messages.
instance and set critical config knobs. Learn to catch the five failures that trip up most production deployments.
| Option | Operational overhead | Data control & privacy | Scalability ceiling | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Local Docker | You manage upgrades, snapshots, and restarts. | Vectors never leave your machine. | Single node unless you add sharding manually. | Choose Local Docker for development, CI pipelines, air-gapped environments, or when you need full control over storage and network. | Infrastructure cost only; no per-vector pricing. | Low to start; you own upgrades, backups, and volume mounts. |
| Qdrant Cloud | Fully managed: upgrades, HA, and backups included. | Vectors stored on managed cloud; check data-residency requirements. | Horizontal scaling and replication handled for you. | Choose Qdrant Cloud when you want managed scaling, built-in backups, and zero ops burden — and data residency rules allow it. | Per-cluster pricing; free tier available for prototyping. | Minimal; cluster creation via dashboard or API. |
Mount a host directory to /qdrant/storage so data survives container restarts. Without this mount, every restart wipes your and .
docker run -d --name qdrant -p 6333:6333 -p 6334:6334 \
-v $(pwd)/qdrant_storage:/qdrant/storage \
qdrant/qdrant:v1.18.2
Pass a config.yaml via -v or environment variables. The three knobs with the biggest production impact are: storage.memmap_threshold_kb (when Qdrant switches from RAM to memory-mapped files), storage.wal.wal_capacity_mb (write-ahead log size before a flush), and storage.optimizers.indexing_threshold_kb (segment size that triggers HNSW index building).
Hit the health endpoint: curl http://localhost:6333/healthz. A {"title":"qdrant - vector search engine","version":"1.18.2"} response confirms the REST API is live. Port 6334 serves if you need it.
. Each one has a distinct symptom — know the symptom and you can fix it in minutes.
Symptom: Wrong input: VectorError { ... expected dim: 768, got: 1536 }. Cause: the was created with one model's dimension and you switched embedding models. Fix: delete and recreate the collection with the correct size parameter — there is no in-place resize.
Symptom: filtered queries that were fast in dev become 10–100× slower under load. Cause: no exists, so Qdrant scans every . Fix: call create_payload_index() for every field used in a .
Symptom: search latency is fine in dev (small dataset) but spikes in production. Cause: indexing_threshold_kb is set too high, so the graph was never triggered and Qdrant falls back to brute-force scan. Fix: check GET /collections/{name} and look for "optimizer_status": "ok" and "indexed_vectors_count" matching "vectors_count".
Symptom: Bad request: wrong vector dimension on . This is distinct from failure #1 — the collection is correct but the embedding pipeline is producing the wrong size (e.g. a truncated batch). Fix: assert len(vector) == expected_dim before every upsert call.
Symptom: no error, but top-k results feel wrong — relevant documents drop out of the result set. Cause: compresses vectors and introduces approximation error. Fix: measure by comparing quantized results against exact search on a sample; if recall < 0.95, raise hnsw_config.ef or disable quantization for that collection.
from qdrant_client import QdrantClient from qdrant_client.models import Distance, VectorParams client = QdrantClient(host="localhost", port=6333) client.recreate_collection( collection_name="seo_pages", vectors_config=VectorParams(size=1536, distance=Distance.COSINE), ) info = client.get_collection("seo_pages") print(info.config.params.vectors) # confirm size=1536
This connects to the local Docker instance and creates the lesson's seo_pages collection with the correct vector size. The final get_collection() call is your first verification step — always confirm size matches your embedding model before upserting any data.
VectorsConfig(size=1536, distance=<Distance.COSINE: 'Cosine'>, ...) — the size field confirms 1536 dimensions. If you see a VectorError instead, the collection already exists with a different size; call delete_collection() first.
info = client.get_collection("seo_pages") status = info.optimizer_status vectors_total = info.vectors_count indexed = info.indexed_vectors_count print(f"Optimizer: {status}") # expect 'ok' print(f"Indexed: {indexed}/{vectors_total}") # expect equal # Spot-check recall: compare quantized vs exact on one query exact = client.search("seo_pages", query_vector=q, limit=10, exact=True) approx = client.search("seo_pages", query_vector=q, limit=10) overlap = len(set(p.id for p in exact) & set(p.id for p in approx)) print(f"Recall@10: {overlap / 10:.2f}") # flag if < 0.95
This two-part check catches failures #3 and #5 from the failure-modes block. First it confirms the optimizer finished; then it measures by comparing approximate results against exact search on the same query vector q. A recall below 0.95 is your signal to tune hnsw_config.ef or revisit quantization settings.
The HNSW index has not been built yet. Qdrant only triggers indexing when a segment exceeds indexing_threshold_kb. Check that value in your config — if it's set higher than your current data size (e.g. 200 MB threshold but you only have 50 k vectors), lower it so the optimizer runs.
from qdrant_client.models import PointStruct EXPECTED_DIM = 1536 def safe_upsert(client, vectors: list[list[float]], ids: list[int], payloads: list[dict]): # TODO: assert every vector in `vectors` has length == EXPECTED_DIM # raise ValueError with the offending index if not points = [ PointStruct(id=i, vector=v, payload=p) for i, v, p in zip(ids, vectors, payloads) ] client.upsert(collection_name="seo_pages", points=points)
This is a near-complete helper for the seo_pages collection. The guard is the crux of failure mode #4 — supply it before revealing the answer.
enumerate(vectors); (2) the error message should name the index and the actual length found.CHANGED LINES (the guard body):
for idx, v in enumerate(vectors):
if len(v) != EXPECTED_DIM:
raise ValueError(f"Vector at index {idx} has dim {len(v)}, expected {EXPECTED_DIM}")
Why: iterating with enumerate lets you name the offending position in the error, making debugging fast. The check runs before any network call, so a bad batch fails immediately rather than after a partial upsert.
Before trusting AI-generated Docker or collection setup in production, run these four checks.
-v ... :/qdrant/storage must appear in the Docker command. AI often omits it, leaving ephemeral storage.size= in VectorParams equals your model's output dimension (e.g., 1536 for text-embedding-3-small, 768 for many open-source models).create_payload_index() is called for every field you filter on. AI-generated code frequently skips this.instance. The capstone challenge wires all six modules together: collection design, indexing, search, and failure guards. That's where the pieces become a system.
Before reading the summary: from memory, name the five core concepts that form Qdrant's data model (collection → ? → ? → ? → ?), then describe in one sentence how a filtered nearest-neighbor query uses the HNSW index and payload indexes together. Write it out, then check against the buildOrder below.
Apply what you learned to SEO/GEO page — the page title is the H1 and the exact search query. OPEN with a 40-60 word self-contained, quotable answer capsule that names the entity explicitly in sentence one (no "in this lesson…" preamble) — this is the sentence an LLM lifts. Name the entity by its full name every time (not pronoun-only). Structure each section as ONE sub-question (How it works · X vs Y · When to use it · Example · Common mistakes), each opening with its own 1-2 sentence direct answer then depth. End with a genuine 3-5 question FAQ phrased as real user questions ending in "?" (these become FAQPage schema + the knowledge check). Prefer quotable specifics — concrete numbers, defaults, version names, one short snippet — over vague prose. Self-contained for a reader who arrived cold from a search engine or an LLM. Intent = explainer: mechanism + example + when-to-use. Target query: "qdrant vector database"..
A startup already runs PostgreSQL for all its application data and wants to add semantic search over ~500 000 product descriptions. They have one engineer, no dedicated ML infra, and recall accuracy is more important than sub-millisecond latency. Which choice fits best?
A. Qdrant running in Docker on a dedicated node
B. pgvector extension inside their existing PostgreSQL instance
C. Milvus with a distributed cluster deployment
D. A plain B-tree index on a text column
pgvector is the right call here: the team already operates PostgreSQL, the dataset is modest (500 k rows), and they want to avoid new infrastructure. Qdrant is a better fit when the vector workload is large enough to justify a dedicated service or when advanced filtering and quantization are needed. Milvus targets massive, distributed deployments — overkill for one engineer. A B-tree index cannot perform vector similarity search at all.
A Qdrant collection is created with vector size 768. A developer then runs:
client.upsert(collection_name="docs", points=[PointStruct(id=1, vector=[0.1]*512, payload={"lang": "en"})])
What happens?
Qdrant enforces that every vector inserted into a collection must match the declared vector size exactly. Passing a 512-element list into a 768-dimension collection triggers a dimension-mismatch error and the upsert is rejected — no silent truncation or auto-resize occurs. This is one of the most common failure modes when switching embedding models without recreating the collection.
In Qdrant's HNSW index, what does the ef_construct parameter control?
ef_construct sets how many candidates HNSW considers when inserting each new node during index construction. A larger value produces a denser, higher-recall graph but takes longer to build. The m parameter (not ef_construct) controls the number of bi-directional links per node. Quantization compression and segment sizing are separate configuration concerns.
A retrieval system must handle queries like 'invoice AND refund NOT shipping' over a legal document corpus where exact keyword matching matters as much as semantic similarity. Which Qdrant search mode is the best fit?
Hybrid search fuses dense (semantic) and sparse (keyword-weighted) scores, which is ideal when both exact term matching and semantic understanding matter. Dense-only search misses precise keyword requirements. Sparse-only search loses semantic generalization. A payload filter with full-text index can match keywords but cannot rank by semantic relevance — it is not a vector retrieval mode.
You start Qdrant with Docker but forget to mount a volume. You upsert 10 000 points, stop the container, and restart it. What do you observe, and what is the one-line Docker flag that would have prevented it?
Without a volume mount, Qdrant writes its segment files to the container's ephemeral layer. When the container stops and is removed, that layer is deleted and all data is lost. The fix is to bind-mount a host path to /qdrant/storage at docker run time. This is the most common production-readiness mistake when first deploying Qdrant locally.