QDrant

The part number that vanished

You run support search for an appliance maker. The help-center articles are split into chunks, embedded with the small open model all-MiniLM-L6-v2, and stored in Qdrant. A customer types:

replacement filter for XK-4471

Qdrant returns these three chunks (a real run, dense vectors only):

The right chunk comes third. The winner describes a different product whose number has two digits swapped. An assistant that reads the top result will cheerfully explain how to pull out a cartridge that the XK-4471 does not have: its filter is washable.

Nothing is broken. The embedding model does exactly what it was trained to do and maps text to meaning. "Filter replacement for the XK-4417" and "replacement filter for XK-4471" mean almost the same thing, and the four digits barely move the vector. The fix is to give Qdrant more than one kind of evidence: a lexical signal that treats 4471 as a rare, exact token, and a hard filter for things that are facts rather than similarities.

This lesson builds that system in Qdrant one move at a time. Each move starts with a symptom you will see in real results:

Every number below comes from running the code against a local Qdrant 1.19.1 server with qdrant-client 1.19.1 (September 2026). The lesson assumes you know what embeddings and cosine similarity are (Vector Embeddings, Cosine Similarity & Distance Metrics) and what a vector database adds over a numpy array (Vector Database Architecture). How HNSW works inside belongs to ANN Algorithms; whether Qdrant is the right product for you belongs to Vector DB Selection; sharding, replication, multitenancy, snapshots and production tuning belong to Advanced Qdrant.

Before any code: a server, a client and five nouns

Start a server

docker run -p 6333:6333 -p 6334:6334 \
    -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
    qdrant/qdrant

That is the command from the Qdrant quickstart. Port 6333 serves REST and the web UI (http://localhost:6333/dashboard), and 6334 serves gRPC. The -v mount decides whether your data outlives the container. Without it, docker stop followed by docker start still keeps your collections (the container's own filesystem survives), but docker rm followed by a new docker run starts empty. For anything you depend on, pin a version tag such as qdrant/qdrant:v1.19.1 instead of the moving latest.

Then connect from Python and load the embedding model:

from qdrant_client import QdrantClient, models
from sentence_transformers import SentenceTransformer

client = QdrantClient(url="http://localhost:6333")   # or QdrantClient(":memory:") in unit tests
dense_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

print(client.info().version)                 # 1.19.1
print(dense_model.get_embedding_dimension()) # 384

QdrantClient(":memory:") runs a pure-Python local mode inside your process. It is great for unit tests, but it searches by brute force and ignores payload indexes (the client warns: "Payload indexes have no effect in the local Qdrant"). So never measure speed with it. prefer_grpc=True switches the client to gRPC, and a managed Qdrant Cloud cluster uses the same client with url= and api_key=.

The data model

Qdrant server
└── collection "support_kb"        fixed vector schema: names, sizes, distances
    ├── point
    │     id:      UUID or unsigned 64-bit integer
    │     vectors: "dense" -> 384 floats
    │              "bm25"  -> sparse (index, weight) pairs          (added in Move 4)
    │     payload: {"doc_id": "kb-102", "chunk_no": 0, "product": "XK-4471",
    │               "updated": "2025-03-10", "text": "The XK-4471 dehumidifier ..."}
    └── point ...

• A collection is like a table with a fixed vector schema. Each named vector declares its size and distance. The payload has no schema.
• A point is one retrievable unit, which in RAG usually means one chunk. Its id must be an unsigned integer or a UUID. A string like "doc-1#0" is rejected with 400 … is not a valid point ID.
• A point can hold several vectors by name: dense (fixed length), sparse (index/value pairs) or multivector (a small matrix, Move 5).
• The payload is JSON that you filter on and read back.
• A payload index is an index on one payload field. It never appears on its own; you create it.

Where writes go

Every write is appended to a write-ahead log (WAL) and then applied to a small appendable segment. From that moment it is searchable. A background optimizer merges segments and builds the HNSW graph for a segment only once the segment holds more than indexing_threshold = 10,000 KB of vectors. Qdrant counts 1 KB as 1,024 bytes, so that is about 6,667 vectors at 384 dimensions. Smaller segments are simply searched exactly. You will meet two consequences later:

• The six-chunk demo collection reports indexed_vectors_count = 0. Every query on it is brute force, so hnsw_ef does nothing there.
• In one run, a 48,148-point test collection turned green with only 42,000 vectors indexed. The rest sat in a segment below the threshold and were searched exactly.

The corpus

Six chunks from five help-center documents. Chunks without a product field apply to every product.

chunks = [
    {"doc_id": "kb-101", "chunk_no": 0, "product": "XK-4417", "updated": "2024-11-02",
     "text": "Filter replacement for the XK-4417 air purifier: twist the cover "
             "counter-clockwise and pull the cartridge out."},
    {"doc_id": "kb-102", "chunk_no": 0, "product": "XK-4471", "updated": "2025-03-10",
     "text": "The XK-4471 dehumidifier uses a washable mesh filter. "
             "Rinse it under warm water every two weeks."},
    {"doc_id": "kb-102", "chunk_no": 1, "product": "XK-4471", "updated": "2025-03-10",
     "text": "Error E-217 means the water tank is full. Empty the tank and press reset."},
    {"doc_id": "kb-103", "chunk_no": 0, "updated": "2023-06-20",
     "text": "How to replace the filter in your air purifier: switch it off, "
             "open the rear panel and slide in a new cartridge."},
    {"doc_id": "kb-105", "chunk_no": 0, "updated": "2025-01-15",
     "text": "Warranty: filters are consumables and are not covered by the two-year warranty."},
    {"doc_id": "kb-106", "chunk_no": 0, "updated": "2024-08-01",
     "text": "Order genuine replacement filters from the parts store to keep the warranty valid."},
]

Move 1: Make the collection match the model

Symptom: scores that look odd, errors on insert, or a model upgrade that "works" while results quietly get worse.

Two decisions are fixed when you create a collection: each vector's size and its distance. Here is the collection this lesson uses. The helper also creates payload indexes before any data arrives; Move 3 explains why that order matters.

COLLECTION = "support_kb"

def fresh_collection(name, distance=models.Distance.COSINE):
    """Demo helper: drop and recreate. Never point this at a live collection."""
    if client.collection_exists(name):
        client.delete_collection(name)
    client.create_collection(
        collection_name=name,
        vectors_config={"dense": models.VectorParams(size=384, distance=distance)},
    )
    # Index the fields you will filter on BEFORE the first upsert
    client.create_payload_index(name, "doc_id", models.PayloadSchemaType.KEYWORD)
    client.create_payload_index(name, "product", models.PayloadSchemaType.KEYWORD)
    client.create_payload_index(name, "updated", models.PayloadSchemaType.DATETIME)

fresh_collection(COLLECTION)
print(client.get_collection(COLLECTION).config.params.vectors)
# {'dense': VectorParams(size=384, distance=<Distance.COSINE: 'Cosine'>, ...)}

Older tutorials use recreate_collection. It still exists but prints a deprecation warning telling you to use collection_exists plus create_collection, which is what the helper does.

Now load the chunks. Each point gets an id computed from its document and chunk number (Move 2 explains why), the dense vector under the name "dense", and the whole chunk dict as payload:

import uuid

def point_id(chunk: dict) -> str:
    # same document + same chunk number -> same id, on every run
    return str(uuid.uuid5(uuid.NAMESPACE_URL, f"kb://{chunk['doc_id']}#{chunk['chunk_no']}"))

def ingest(name, chunks, make_id=point_id):
    vectors = dense_model.encode([c["text"] for c in chunks], batch_size=64)
    client.upsert(name, points=[
        models.PointStruct(id=make_id(c), vector={"dense": v.tolist()}, payload=c)
        for c, v in zip(chunks, vectors)
    ])  # wait=True is the Python client's default

ingest(COLLECTION, chunks)
print(client.count(COLLECTION).count)    # 6

And this query reproduces the opening table:

def dense_search(question, limit=3, flt=None):
    hits = client.query_points(
        COLLECTION,
        query=dense_model.encode(question).tolist(),
        using="dense",
        query_filter=flt,
        limit=limit,
    ).points
    for h in hits:
        p = h.payload
        print(f"{h.score:.3f}  {p['doc_id']}#{p['chunk_no']}  {p['text'][:48]}")

dense_search("replacement filter for XK-4471")
# 0.686  kb-101#0  Filter replacement for the XK-4417 air purifier:
# 0.552  kb-106#0  Order genuine replacement filters from the parts
# 0.455  kb-102#0  The XK-4471 dehumidifier uses a washable mesh fi

Size: loud when wrong, silent when "right"

size must equal the model's output dimension (384 for MiniLM). Get it wrong and Qdrant refuses the write:

client.create_collection("shapes", vectors_config=models.VectorParams(size=4, distance=models.Distance.COSINE))
client.upsert("shapes", points=[models.PointStruct(id=1, vector=[3.0, 4.0, 0.0, 0.0])])
print(client.retrieve("shapes", ids=[1], with_vectors=True)[0].vector)   # [0.6, 0.8, 0.0, 0.0]

try:
    client.upsert("shapes", points=[models.PointStruct(id=2, vector=[1.0, 0.0, 0.0])])
except Exception as e:
    print(str(e).splitlines()[-1])
# b'{"status":{"error":"Wrong input: Vector dimension error: expected dim: 4, got 3"},...}'
client.delete_collection("shapes")

That error is the good failure. The dangerous case is a different model with the same dimension: every query still returns results with plausible scores, just from a meaningless comparison. Move 5 shows it happening.

The first print also shows something useful: with Distance.COSINE, Qdrant normalizes vectors on insert, so [3, 4, 0, 0] is stored as [0.6, 0.8, 0, 0]. At query time a cosine collection is scored with a plain dot product. It is not slower than DOT.

Distance: pick what the model was trained with

A model trained for dot product really does produce vectors of different lengths. multi-qa-MiniLM-L6-dot-v1 gave norms between 6.97 and 7.53 on three test sentences. Put that model in a COSINE collection and the normalization on insert throws away the length the model learned to use. Metric theory lives in Cosine Similarity & Distance Metrics; the practical rule here is read the model card and copy its metric.

Predict first: a teammate created the collection with Distance.EUCLID instead of COSINE. MiniLM outputs unit-length vectors. The application drops results below score_threshold=0.5, a cut-off someone once picked for cosine. What changes: the ranking, the scores, or what survives the threshold?

fresh_collection("support_kb_l2", distance=models.Distance.EUCLID)
ingest("support_kb_l2", chunks)

q = dense_model.encode("replacement filter for XK-4471").tolist()
for name in [COLLECTION, "support_kb_l2"]:
    hits = client.query_points(name, query=q, using="dense", limit=3).points
    print(name, [(h.payload["doc_id"], round(h.score, 3)) for h in hits])
    kept = client.query_points(name, query=q, using="dense", limit=3, score_threshold=0.5).points
    print("   score_threshold=0.5 keeps", len(kept))
client.delete_collection("support_kb_l2")

support_kb [('kb-101', 0.686), ('kb-106', 0.552), ('kb-102', 0.455)]
   score_threshold=0.5 keeps 2
support_kb_l2 [('kb-101', 0.793), ('kb-106', 0.946), ('kb-102', 1.044)]
   score_threshold=0.5 keeps 0

⚠️ Score cut-offs are model-specific. The relevant chunk above scored 0.455. A fixed rule such as "good matches score above 0.75" would have thrown away the correct answer. Scores are not probabilities and cannot be compared across models. If you need a threshold (for example, to say "I don't know"), tune it on labeled queries for this model.

Measure it: before you create the collection, encode 20 of your own texts, print their norms and a few pairwise scores, and check the model card's metric. After ingestion, run five queries whose right answer you know and confirm those chunks appear.

Move 2: Write points you can find again

Symptom: the nightly ingestion job runs twice and every chunk now appears twice in the top-k; a revised document still returns its old paragraphs.

Ids that survive a re-run

An upsert overwrites a point with the same id. That makes ingestion re-runnable only if the same chunk always gets the same id. The point_id function in Move 1 builds a name-based UUID (version 5) from kb://<doc_id>#<chunk_no>, so the same chunk maps to the same id on every run and on every machine:

ingest(COLLECTION, chunks)                # the nightly job runs again
print(client.count(COLLECTION).count)    # still 6

Your turn: many tutorials write id=str(uuid.uuid4()). Predict the count after running that version twice on a fresh collection.

fresh_collection("scratch")
ingest("scratch", chunks, make_id=lambda c: str(uuid.uuid4()))   # the old script
ingest("scratch", chunks, make_id=lambda c: str(uuid.uuid4()))   # ...run again
print(client.count("scratch").count)

When a document shrinks

Deterministic ids solve re-runs. They do not solve re-chunking. Suppose kb-102 is revised and now fits in one chunk:

fresh_collection("scratch")
ingest("scratch", chunks)                       # kb-102 is stored as chunks #0 and #1

kb102_v2 = [{"doc_id": "kb-102", "chunk_no": 0, "product": "XK-4471", "updated": "2025-09-01",
             "text": "XK-4471 manual, revised: rinse the washable mesh filter monthly. "
                     "Error E-217 means the tank is full."}]
ingest("scratch", kb102_v2)                     # re-chunked into ONE chunk

def by_doc(doc_id):
    return models.Filter(must=[models.FieldCondition(key="doc_id", match=models.MatchValue(value=doc_id))])

print(client.count("scratch", count_filter=by_doc("kb-102")).count)   # 2

Chunk #0 was overwritten, but the old chunk #1 is an orphan. It still answers E-217 questions with the old version's text, and it will keep doing so after the manual changes again. The fix is to treat a document as the unit of replacement:

def replace_document(name, doc_id, new_chunks):
    client.delete(name, points_selector=models.FilterSelector(filter=by_doc(doc_id)))
    ingest(name, new_chunks)

replace_document("scratch", "kb-102", kb102_v2)
print(client.count("scratch", count_filter=by_doc("kb-102")).count)   # 1

Cost: between the delete and the upsert, the document is briefly missing from search. If that gap matters, write the new chunks first with a version payload field (with version-aware ids, or position ids carrying the version in the payload), then delete the old version by filter. Keeping a whole corpus fresh is the job of Data Freshness & Lifecycle.

What belongs in the payload

• The chunk text: yes. RAG needs it at query time, and a second lookup to another store costs a round trip per query. Payloads live on disk by default (in Qdrant 1.19 the default memory tier for payloads is cold, and the in-RAM tier pinned is rejected for them; the collection config also shows the deprecated on_disk_payload: true). So a 1–2 KB chunk costs disk and network, not RAM. Ask only for the fields you need with with_payload=["text", "doc_id"].
• Whole 50 KB documents: no. Store the document id or URL and fetch the full text from the source when you need it.
• Fields you will filter or cite by: doc_id, product, updated (RFC 3339 dates), language, tenant and access-control fields (designing those is the topic of Metadata & Filtering).
• One type per field. Predict what this prints:

client.upsert("scratch", points=[
    models.PointStruct(id=101, vector={"dense": [0.1] * 384}, payload={"year": 2024}),
    models.PointStruct(id=102, vector={"dense": [0.1] * 384}, payload={"year": "2024"}),
])
year_is_2024 = models.Filter(must=[models.FieldCondition(key="year", match=models.MatchValue(value=2024))])
print(client.count("scratch", count_filter=year_is_2024).count)
client.delete_collection("scratch")   # done with the scratch collection

If a framework will read the collection, match its payload layout. LangChain's QdrantVectorStore, for example, looks for the text under page_content and metadata nested under metadata unless you set content_payload_key and metadata_payload_key. Point it at a flat payload like ours and it returns empty documents without an error.

wait and batching

client.upsert waits by default (wait=True): when it returns, the points are searchable, by exact scan if they are not in the HNSW graph yet. client.upload_points batches large loads (64 points per request unless you set batch_size) and defaults to wait=False, so pass wait=True if the next line queries. The REST API defaults to wait=false. Embed in batches too: calling the model (or an embedding API) once per text multiplies the per-call overhead.

Move 3: Filter inside the search

Symptom: the answer cites a policy for the wrong product, region or year.

The opening failure ranked the XK-4417 chunk first. A product code is not a similarity; it is a fact about the chunk, and facts belong in filters. When the question names a product, restrict the search to that product's chunks plus the chunks that apply to every product:

def product_scope(code):
    # this product's chunks, plus chunks that apply to every product (no "product" field)
    return models.Filter(should=[
        models.FieldCondition(key="product", match=models.MatchValue(value=code)),
        models.IsEmptyCondition(is_empty=models.PayloadField(key="product")),
    ])

print(client.count(COLLECTION, count_filter=product_scope("XK-4471")).count)   # 5
dense_search("replacement filter for XK-4471", flt=product_scope("XK-4471"))
# 0.552  kb-106#0  Order genuine replacement filters from the parts
# 0.455  kb-102#0  The XK-4471 dehumidifier uses a washable mesh fi
# 0.452  kb-103#0  How to replace the filter in your air purifier:

The wrong product is gone. The right chunk is second, because dense scoring still prefers the generic "order filters" chunk. Move 4 works on the order.

The filter vocabulary

Clauses nest, so must can contain a Filter with its own should.

Predict first: how many of the six chunks survive each filter?

recent = models.Filter(must=[
    models.FieldCondition(key="updated", range=models.DatetimeRange(gte="2025-01-01")),
])
not_xk4417 = models.Filter(must_not=[
    models.FieldCondition(key="product", match=models.MatchValue(value="XK-4417")),
])
print(client.count(COLLECTION, count_filter=recent).count,
      client.count(COLLECTION, count_filter=not_xk4417).count)

⚠️ models.Range(gte="2024-05-01") does not filter dates. It raises a pydantic ValidationError ("Input should be a valid number") before anything reaches the server. Use DatetimeRange on a field indexed as datetime.

Index before you ingest

A payload index does three jobs:

1. It finds the matching point ids quickly instead of reading every payload.
2. It gives the query planner a cardinality estimate: roughly how many points match. (In Qdrant's docs, cardinality means this number of matching points, not a field's number of distinct values.)
3. It lets the HNSW builder add extra links based on the indexed values, so that a walk through the graph restricted to one value stays connected.

The third job happens only if the index exists when the graph is built. The indexing docs are explicit: "Payload indexes should be created before ingesting data… If you create a payload index after data has already been ingested, you need to rebuild the HNSW index to take advantage of the new payload indexes." That is why fresh_collection creates the indexes first; Challenge 3 at the end shows what happens when you don't, and how to recover. Index types: keyword, integer, float, bool, datetime, uuid, text and geo. Filtering on a field with no index at all still runs, but Qdrant has to read payloads to check it, the planner has no estimate, and the graph has no helper links. In the scale model from Challenge 3 with no region index at all, filtered recall@10 was about 0.01, and 51 to 127 of 200 filtered queries (depending on the run) came back with fewer than their 10 results. Index every field you filter on.

What Qdrant does with a filter

Qdrant does not fetch the top-k and then throw away non-matching points. A collection is stored as several segments (every collection on the server used for this lesson had six), and the planner picks a path for each segment separately:

query vector + filter
        │
        ▼
for each segment: estimate how many of its points match  (payload index)
        │
        ├── few: their vectors total less than full_scan_threshold (10,000 KB)
        │        └──► exact search over just the matching points
        │
        └── many ──► HNSW walk that checks the filter at every step,
                     using the extra per-value links
        ▼
merge the segments' results ──► top `limit` points that match

10,000 KB is about 6,667 vectors at 384 dimensions, or 1,667 at 1,536. Because the check is per segment, what counts is each segment's share of the matching points: with six segments and the matches spread evenly, a filtered set of up to roughly six times that size can still be searched exactly. In a measured test, a collection of 60,000 synthetic 384-d vectors held 100 tenants of 600 points each. A query filtered to one tenant with limit=10 returned 10 results on all 50 test queries, identical to exact search over that tenant. 600 × 1,536 bytes is 900 KB, far below the threshold, so the planner took the exact path.

There is a third, opt-in strategy for a walk through a graph that the filter leaves badly connected. ACORN (Qdrant 1.16+) also visits a neighbour's neighbours when the direct neighbours fail the filter. You turn it on per query with search_params=models.SearchParams(acorn=models.AcornSearchParams(enable=True)), and Qdrant then uses it only for filters estimated to pass under max_selectivity, 40% of the points by default. It improves recall at a price: the search docs put it at about 2–10× slower. Challenge 3 shows when you need it.

Compare a system that post-filters: it fetches the top 100 from the whole collection and then drops other tenants. With 1% of points matching, you expect 100 × 0.01 = 1 survivor for a request of 10. That starvation is why filtering must happen inside the search. The general theory, including pgvector's different behaviour, is in Metadata & Filtering; Qdrant's tenant-specific index settings (is_tenant) are in Advanced Qdrant.

Measure it: use client.count(..., count_filter=f) to check that a filter matches what you expect. Compare filtered results with the same query run with search_params=models.SearchParams(exact=True), and count filtered queries that return fewer than limit results when you know enough points match: short pages are a symptom you can alert on. Watch p95 latency for your most and least selective filters, because the planner takes different paths for them.

Move 4: One query, two retrievers

Symptom: exact codes, error numbers, names and rare terms lose to near-misses.

The one query method

Since Qdrant 1.10 the Query API can express every kind of search, and in qdrant-client 1.19 it is the only way to search: client.query_points. The old client.search, search_batch, recommend, discover and search_groups methods are gone from the client. Code from older tutorials fails with AttributeError: 'QdrantClient' object has no attribute 'search'. Their replacements are query_points, query_batch_points and query_points_groups.

Sparse vectors: a second retriever in the same collection

A sparse vector stores only its non-zero dimensions as index/value pairs. Qdrant scores sparse vectors with a dot product; they take no Distance setting. With modifier=models.Modifier.IDF (Qdrant 1.10+), Qdrant computes inverse document frequency from the collection itself and applies it at query time, so the statistics stay current as documents arrive. FastEmbed's Qdrant/bm25 model produces the other half of BM25: term-frequency weights for documents (with k = 1.2, b = 0.75 and an assumed average document length of 256 tokens unless you pass avg_len), and a weight of 1 per term for queries. The BM25 formula itself is taught in Sparse vs Dense Retrieval.

Our collection was created with only a dense vector. Since Qdrant 1.18 you can add a named vector to an existing collection and backfill it without re-uploading anything else:

from fastembed import SparseTextEmbedding

bm25 = SparseTextEmbedding("Qdrant/bm25")   # term weights; Qdrant adds IDF

client.create_vector_name(                  # Qdrant 1.18+: add a vector to an existing collection
    COLLECTION, "bm25",
    models.SparseVectorNameConfig(sparse=models.SparseVectorConfig(modifier=models.Modifier.IDF)),
)

records, _ = client.scroll(COLLECTION, limit=1000, with_payload=["text"])
sparse = bm25.embed([r.payload["text"] for r in records])
client.update_vectors(COLLECTION, points=[
    models.PointVectors(id=r.id, vector={"bm25": models.SparseVector(
        indices=s.indices.tolist(), values=s.values.tolist())})
    for r, s in zip(records, sparse)
])

q_sparse = next(bm25.query_embed("replacement filter for XK-4471"))
print(len(q_sparse.indices), q_sparse.values.tolist())   # 4 [1, 1, 1, 1]

On an older server, declare the sparse vector when you create the collection: sparse_vectors_config={"bm25": models.SparseVectorParams(modifier=models.Modifier.IDF)}. The query keeps four terms because the stopword "for" is dropped and XK-4471 splits into xk and 4471. Use query_embed for questions and embed for documents; they weight terms differently. Learned sparse models from FastEmbed (SPLADE-style, miniCOIL) go into the same kind of slot. Whether they need Modifier.IDF depends on the model, and FastEmbed's model list states it (requires_idf).

Fuse the two with prefetch

question ──┬──► MiniLM ──► dense vector ───► prefetch: top 20 by cosine ─┐
           │                                     ▲                       │
           │   query_filter (if any) ────────────┤                       ├──► RRF ──► top `limit`
           │                                     ▼                       │
           └──► BM25 ────► sparse vector ──► prefetch: top 20 by BM25 ───┘

def hybrid(question, flt=None, k_candidates=20, limit=3, show=True, fusion=None):
    q_dense = dense_model.encode(question).tolist()
    s = next(bm25.query_embed(question))
    q_sparse = models.SparseVector(indices=s.indices.tolist(), values=s.values.tolist())
    hits = client.query_points(
        COLLECTION,
        prefetch=[
            models.Prefetch(query=q_dense, using="dense", limit=k_candidates),
            models.Prefetch(query=q_sparse, using="bm25", limit=k_candidates),
        ],
        query=fusion or models.RrfQuery(rrf=models.Rrf()),  # RRF default: k = 2, zero-based ranks
        query_filter=flt,                          # applied to every prefetch
        limit=limit,
    ).points
    if show:
        for h in hits:
            p = h.payload
            print(f"{h.score:.4f}  {p['doc_id']}#{p['chunk_no']}  {p['text'][:48]}")
    return hits

q = "replacement filter for XK-4471"
s = next(bm25.query_embed(q))
bm = client.query_points(COLLECTION, using="bm25", limit=3,
    query=models.SparseVector(indices=s.indices.tolist(), values=s.values.tolist())).points
print("bm25 :", [(h.payload["doc_id"], round(h.score, 3)) for h in bm])
# bm25 : [('kb-102', 4.596), ('kb-101', 3.219), ('kb-106', 1.544)]

BM25 puts the right chunk first by a wide margin, because 4471 occurs in exactly one chunk and gets the highest IDF. Predict first: run hybrid(q). Which chunk ranks first after fusion?

hybrid(q)
# 0.8333  kb-101#0  Filter replacement for the XK-4417 air purifier:
# 0.7500  kb-102#0  The XK-4471 dehumidifier uses a washable mesh fi
# 0.5833  kb-106#0  Order genuine replacement filters from the parts

The course's RRF formula (see Hybrid Retrieval Systems) follows Cormack, Clarke & Büttcher (SIGIR 2009): ranks count from 1 and k = 60. Qdrant's default is steeper. A list's top result is worth 1/2 and its fifth is worth 1/6, so a single first place counts for a lot. models.RrfQuery(rrf=models.Rrf(k=60)) (Qdrant 1.16+) gives the flatter curve; with zero-based ranks, 1/(60 + r) equals the paper's formula with k = 59. Per-prefetch weights arrived in 1.17, and score-based DBSF fusion in 1.11 (see the hybrid queries docs).

Both moves together

Now combine the product filter from Move 3 with hybrid retrieval:

hybrid("replacement filter for XK-4471", flt=product_scope("XK-4471"))
# 0.8333  kb-106#0  Order genuine replacement filters from the parts
# 0.8333  kb-102#0  The XK-4471 dehumidifier uses a washable mesh fi
# 0.5000  kb-103#0  How to replace the filter in your air purifier:

A tie. Inside the filter, dense ranks kb-106 then kb-102, and BM25 ranks kb-102 then kb-106. Swapped ranks give identical RRF scores (1/2 + 1/3 each). Which of the two tied rows prints first is not something to rely on: across runs of this exact script, both orders appeared. The information that could break the tie, BM25's 4.596 against 1.544, was thrown away when the scores became ranks. DBSF keeps it. It rescales each list's scores (mean ± 3 standard deviations mapped onto 0 to 1) and adds them:

hybrid("replacement filter for XK-4471", flt=product_scope("XK-4471"), limit=2,
       fusion=models.FusionQuery(fusion=models.Fusion.DBSF))
# 1.3086  kb-102#0  The XK-4471 dehumidifier uses a washable mesh fi
# 1.1525  kb-106#0  Order genuine replacement filters from the parts

Do not switch fusion methods because of one query. You just saw how easily a six-chunk example can be made to say anything. Choose with a labeled query set, as in the Boss level below. If the order within a good candidate set matters, a reranker usually earns its latency: see Reranking Models.

Three practical details:

• Candidate pools. Each prefetch limit is that retriever's candidate pool. It must be at least limit + offset of the outer query, or pages come back short. Larger pools help only when the right chunk sits deep in one list, and they cost latency; measure recall at your final limit as you grow them. A reranker downstream typically wants 50–200 candidates.
• One document hogging the top-k. client.query_points_groups(..., group_by="doc_id", group_size=1) returns the best chunk per document. MMR diversification (Qdrant 1.15+) is query=models.NearestQuery(nearest=vec, mmr=models.Mmr(diversity=0.5, candidates_limit=100)). diversity weights variety: 0 ranks by relevance alone, and it is 1 − λ in the MMR formula of Context Augmentation.
• Filters and prefetch. A top-level query_filter is applied inside every prefetch, before fusion, as the diagram shows. You can also give a single Prefetch its own filter.

Move 5: Several vectors per point

Symptom: one embedding cannot serve two kinds of query, or you need to switch embedding models without taking search down.

Named vectors

Each named vector has its own size and distance, and a query picks one with using=. Typical splits:

• "title" and "body", embedded separately because short titles and long bodies behave differently;
• "text" and "image", each from its own model and each with the metric its model card names (CLIP-style image encoders, for example, are trained with cosine similarity on normalized vectors);
• "dense" and "bm25", as you just built.

A point may lack some of its named vectors. A query on using="x" simply never sees points without an x vector.

Upgrading the embedding model

Named vectors let you run old and new models side by side. Add the new vector, backfill it, compare on your labeled queries, switch using, and only then delete the old one:

new_model = SentenceTransformer("sentence-transformers/multi-qa-MiniLM-L6-cos-v1")  # also 384-d

client.create_vector_name(COLLECTION, "dense_v2", models.DenseVectorNameConfig(
    dense=models.DenseVectorConfig(size=384, distance=models.Distance.COSINE)))

records, _ = client.scroll(COLLECTION, limit=1000, with_payload=["text"])
vectors = new_model.encode([r.payload["text"] for r in records])
client.update_vectors(COLLECTION, points=[
    models.PointVectors(id=r.id, vector={"dense_v2": v.tolist()}) for r, v in zip(records, vectors)
])

q = "replacement filter for XK-4471"
for using, model, label in [("dense", dense_model, "old model, old vectors"),
                            ("dense_v2", new_model, "new model, new vectors"),
                            ("dense_v2", dense_model, "OLD model, new vectors")]:
    hits = client.query_points(COLLECTION, query=model.encode(q).tolist(), using=using, limit=3).points
    print(f"{label:23}", [(h.payload["doc_id"], round(h.score, 3)) for h in hits])
# old model, old vectors  [('kb-101', 0.686), ('kb-106', 0.552), ('kb-102', 0.455)]
# new model, new vectors  [('kb-101', 0.687), ('kb-106', 0.561), ('kb-103', 0.486)]
# OLD model, new vectors  [('kb-101', 0.518), ('kb-106', 0.473), ('kb-103', 0.378)]

Two lessons in three lines of output. First, the new model is not better on this query: kb-102 dropped out of its top 3. A model upgrade is an experiment to measure, not a free win. Second, the last line is the silent bug from Move 1. A query vector from the old model was compared with vectors from the new one. Both have 384 dimensions, so there was no error and the scores look plausible, but the geometry is meaningless. Keep the model name next to the vector name in your config, and have the query path assert that they match.

Cost: during the migration every point carries both vectors, so vector memory doubles until you run client.delete_vector_name(COLLECTION, "dense"). create_vector_name and delete_vector_name need Qdrant 1.18 or later. The alternative that works on any version is to build a new collection and switch a collection alias (client.update_collection_aliases) that your application queries. It costs a full second copy, payload included.

Multivectors: one matrix per point

Late-interaction models such as ColBERT keep one vector per token and score with MaxSim: for each query token, take its best match among the document's tokens, then add those up. Qdrant (1.10+) stores such a matrix under one name:

import numpy as np

client.create_collection("scratch_mv", vectors_config={
    "tokens": models.VectorParams(
        size=3, distance=models.Distance.DOT,
        multivector_config=models.MultiVectorConfig(comparator=models.MultiVectorComparator.MAX_SIM),
        hnsw_config=models.HnswConfigDiff(m=0),     # no graph: this vector only reranks candidates
    )})
doc_a = [[1, 0, 0], [0, 1, 0]]                    # 2 token vectors
doc_b = [[0.6, 0.8, 0], [0, 0, 1], [0.5, 0.5, 0.5]]  # 3 token vectors
client.upsert("scratch_mv", points=[
    models.PointStruct(id=1, vector={"tokens": doc_a}),
    models.PointStruct(id=2, vector={"tokens": doc_b}),
])
query = [[1, 0, 0], [0, 0, 1]]                    # 2 query token vectors
hits = client.query_points("scratch_mv", query=query, using="tokens").points
print([(h.id, round(h.score, 3)) for h in hits])  # [(2, 1.6), (1, 1.0)]

maxsim = lambda q, d: float((np.array(q) @ np.array(d).T).max(axis=1).sum())
print(maxsim(query, doc_a), maxsim(query, doc_b)) # 1.0 1.6
client.delete_collection("scratch_mv")

For doc_b, query token [1, 0, 0] matches [0.6, 0.8, 0] best (0.6) and [0, 0, 1] matches exactly (1.0), for a total of 1.6. Doc_a gets 1.0 + 0.

Cost: 200 tokens × 128 dimensions × 4 bytes is 102,400 bytes per chunk, 67 times one 384-d float vector. That is why the example sets m=0 (no HNSW graph) and why a multivector is usually a reranking stage over candidates that a cheaper vector found. Building that multi-stage pipeline is covered in Advanced Qdrant, and late interaction as a reranking method in Reranking Models.

Move 6: HNSW and quantization — knobs you can measure

Symptom: RAM over budget, recall below target, or "I raised hnsw_ef and nothing changed".

What you get by default

info = client.get_collection(COLLECTION)
print(info.status, info.points_count, info.indexed_vectors_count)
hnsw = info.config.hnsw_config
print(hnsw.m, hnsw.ef_construct, hnsw.full_scan_threshold, info.config.optimizer_config.indexing_threshold)
# green 6 0
# 16 100 10000 10000

hnsw_ef is the only one of these you can change per request. If you set it below limit, you still get limit results; they are just less likely to be the true nearest ones. Changing m or ef_construct (via update_collection) means the optimizer rebuilds the graph. And none of this matters for our six chunks: indexed_vectors_count is 0, so every query is already exact.

Measure before you tune

SearchParams(exact=True) gives brute-force ground truth. The share of the exact top-k that HNSW also returns is ANN recall@k:

def ann_recall(name, query_vectors, k=10, hnsw_ef=None, using=None):
    """Share of the exact top-k that the HNSW search also returned (ANN recall@k)."""
    total = 0.0
    for q in query_vectors:
        exact = client.query_points(name, query=q, using=using, limit=k, with_payload=False,
                                    search_params=models.SearchParams(exact=True)).points
        approx = client.query_points(name, query=q, using=using, limit=k, with_payload=False,
                                     search_params=models.SearchParams(hnsw_ef=hnsw_ef)).points
        total += len({p.id for p in exact} & {p.id for p in approx}) / k
    return total / len(query_vectors)

print(ann_recall(COLLECTION, [dense_model.encode("filter").tolist()], k=3, using="dense"))  # 1.0

A value of 1.0 on six unindexed chunks proves nothing, so here is a collection large enough to have a graph. The Quora duplicate-questions set gives pairs of questions that mean the same thing (here a 48,148-question sample, not the BEIR corpus used in ANN Algorithms). Index one phrasing of each pair and query with the other; a good system should return the partner. That gives you a second, different metric: IR recall@10 (the course's retrieval recall), whether the answer made the top 10.

import gzip, json, time
from huggingface_hub import hf_hub_download

path = hf_hub_download("sentence-transformers/embedding-training-data",
                       "quora_duplicates.jsonl.gz", repo_type="dataset")
seen, corpus, queries = set(), [], []
for line in gzip.open(path, "rt"):
    a, b = json.loads(line)          # two phrasings of the same question
    if a in seen or b in seen:
        continue
    seen.update((a, b))
    corpus.append(a)                 # index one phrasing...
    if len(queries) < 300:
        queries.append(b)            # ...and ask with the other: its answer is corpus[i]
corpus_vectors = dense_model.encode(corpus, batch_size=256)
query_vectors = dense_model.encode(queries).tolist()
print(len(corpus), "questions indexed,", len(queries), "queries")

def load(name, quantization=None):
    client.create_collection(name, quantization_config=quantization,
        vectors_config=models.VectorParams(size=384, distance=models.Distance.COSINE))
    client.upload_points(name, wait=True, batch_size=2000, points=[
        models.PointStruct(id=i, vector=v.tolist()) for i, v in enumerate(corpus_vectors)])
    time.sleep(2)                    # give the optimizer a moment to start
    while client.get_collection(name).status != models.CollectionStatus.GREEN:
        time.sleep(1)                # the optimizer is still building HNSW

def ir_recall(name, k=10, **params):  # did the paraphrase's partner make the top k?
    hits = [client.query_points(name, query=q, limit=k, with_payload=False,
                                search_params=models.SearchParams(**params)).points
            for q in query_vectors]
    return sum(i in {p.id for p in h} for i, h in enumerate(hits)) / len(hits)

load("quora")
info = client.get_collection("quora")
print(info.points_count, info.indexed_vectors_count)
for ef in [10, 32, None]:          # None = the default, ef_construct = 100
    print("hnsw_ef", ef,
          "| ANN recall@10", round(ann_recall("quora", query_vectors, hnsw_ef=ef), 3),
          "| IR recall@10", round(ir_recall("quora", hnsw_ef=ef), 3))

48148 questions indexed, 300 queries
48148 48148
hnsw_ef 10 | ANN recall@10 0.976 | IR recall@10 0.9
hnsw_ef 32 | ANN recall@10 0.997 | IR recall@10 0.9
hnsw_ef None | ANN recall@10 1.0 | IR recall@10 0.9

Read it slowly:

• ANN recall measures the index against brute force. Cutting hnsw_ef from the default of 100 to 10 lost about 2 points of it.
• IR recall measures the whole system against what the user needed. It did not move in this run, and in reruns it moved by at most one query of 300 (0.897 at hnsw_ef=10): the neighbours HNSW missed were almost never the answer.
• For 10% of the queries, MiniLM does not put the paraphrase in the top 10 even with exact search. That is an embedding and data problem. No HNSW setting fixes it.
• The exact numbers move a little between runs, because the graph and the segment layout differ each time. Across five runs, hnsw_ef=10 gave 0.976 to 0.986 and the default gave 0.999 to 1.000. In one run, 6,148 points sat in a segment below indexing_threshold and were searched exactly, which flatters ANN recall. Check indexed_vectors_count before you trust a benchmark, and treat the third decimal as noise.

(ANN recall@k and how ef works inside HNSW are covered in ANN Algorithms; IR recall and the other retrieval metrics in Retrieval Metrics.)

Quantization: smaller vectors, then rescore

Quantization keeps a compressed copy of every vector for the graph search and, optionally, rescores the best candidates with the original float32 vectors. A 384-d MiniLM vector takes 1,536 bytes as float32, 384 bytes as scalar int8 (4× smaller) and 48 bytes as 1-bit binary codes (32× smaller). Qdrant 1.19 also offers 1.5- and 2-bit binary encodings (since 1.15; the quantization guide quotes 24× for 1.5 bits, although 32 / 1.5 ≈ 21×), product quantization and TurboQuant (1.18+); Advanced Qdrant compares them. You switch quantization on with quantization_config when you create the collection, and choose rescoring per query with QuantizationSearchParams(rescore=…, oversampling=…).

How much accuracy a method costs depends on your model and data, so measure it. The script below loads the Quora vectors twice more, once with int8 and once with 1-bit binary codes, and compares each with exact search on the float32 collection.

Predict first: at the default oversampling of 1, a query with limit=10 takes 10 candidates by compressed score and rescores them with float32. Can rescoring then change which ten points come back, or only their order?

load("quora_int8", models.ScalarQuantization(scalar=models.ScalarQuantizationConfig(
    type=models.ScalarType.INT8, quantile=0.99)))
load("quora_binary", models.BinaryQuantization(binary=models.BinaryQuantizationConfig()))

def quant_recall(name, **qparams):
    total = 0.0
    for q in query_vectors:
        exact = client.query_points("quora", query=q, limit=10, with_payload=False,
                                    search_params=models.SearchParams(exact=True)).points
        got = client.query_points(name, query=q, limit=10, with_payload=False,
            search_params=models.SearchParams(
                quantization=models.QuantizationSearchParams(**qparams) if qparams else None)).points
        total += len({p.id for p in exact} & {p.id for p in got}) / 10
    return total / len(query_vectors)

for name in ["quora_int8", "quora_binary"]:
    print(name, "default:", round(quant_recall(name), 3),
          "| rescore=False:", round(quant_recall(name, rescore=False), 3),
          "| rescore=True:", round(quant_recall(name, rescore=True), 3),
          "| rescore + oversampling=2:", round(quant_recall(name, rescore=True, oversampling=2.0), 3))
for n in ["quora", "quora_int8", "quora_binary"]:
    client.delete_collection(n)

quora_int8 default: 0.989 | rescore=False: 0.989 | rescore=True: 0.999 | rescore + oversampling=2: 0.999
quora_binary default: 0.973 | rescore=False: 0.761 | rescore=True: 0.973 | rescore + oversampling=2: 0.992

The "default" column matches the docs: rescoring is on by default for binary (and the 1, 1.5 and 2-bit TurboQuant variants) but off for scalar. The binary default matches rescore=True, and the scalar default matches rescore=False. (A separate check of IR recall, as in the previous script, gave 0.90 for nearly every configuration. The exception was binary without rescoring, at 0.903 to 0.907: its different top 10 happened to catch one or two more paraphrase partners. At k = 10, these ANN differences moved the answer in or out for at most a couple of the 300 queries.)

Cost: rescoring reads up to oversampling × limit float32 originals per segment for every query, which is nearly free while they sit in RAM and can cost a random disk read each when they live on disk, so measure p95 under load with the originals where production will keep them. Qdrant 1.19 sets that place with one memory setting per structure: pinned (in RAM, never evicted), cached (on disk, pre-loaded into the page cache) or cold (on disk, read on first access). The defaults per structure, the layout for collections that outgrow RAM, node sizing and oversampling tuning are in Advanced Qdrant.

Boss level: the whole retrieval flow

Everything so far, in the order a query meets it:

question
   │  regex finds a product code? ──► product_scope filter (Move 3)
   ▼
dense + BM25 prefetch, 20 candidates each  (Move 4)
   │  filter applied inside both
   ▼
DBSF fusion ──► top k_context chunks, with payload text and ids
   ▼
prompt with numbered sources ──► generate() ──► answer with citations

import re

PRODUCT_CODE = re.compile(r"\bXK-\d{4}\b")

def retrieve(question, k_context=3):
    code = PRODUCT_CODE.search(question)
    flt = product_scope(code.group()) if code else None   # no code -> search everything
    return hybrid(question, flt=flt, limit=k_context, show=False,
                  fusion=models.FusionQuery(fusion=models.Fusion.DBSF))  # breaks Move 4's tie; re-check on labels

def build_prompt(question, hits):
    sources = "\n".join(
        f"[{h.payload['doc_id']}#{h.payload['chunk_no']}] {h.payload['text']}" for h in hits)
    return ("Answer only from the sources below and cite them like [kb-102#0]. "
            "If they do not contain the answer, say so.\n\n"
            f"Sources:\n{sources}\n\nQuestion: {question}")

def generate(prompt: str) -> str:
    return "(stub) call your LLM here"     # see the Classic RAG Pipeline lesson

question = "Do I need to buy a new filter for my XK-4471?"
prompt = build_prompt(question, retrieve(question))
print(prompt)
answer = generate(prompt)

Answer only from the sources below and cite them like [kb-102#0]. If they do not contain the answer, say so.

Sources:
[kb-102#0] The XK-4471 dehumidifier uses a washable mesh filter. Rinse it under warm water every two weeks.
[kb-103#0] How to replace the filter in your air purifier: switch it off, open the rear panel and slide in a new cartridge.
[kb-106#0] Order genuine replacement filters from the parts store to keep the warranty valid.

Question: Do I need to buy a new filter for my XK-4471?

The chunk that answers the question ("it's washable, rinse it") is first, the wrong product is absent, and every source carries an id the answer can cite. Prompt design, citation checking and the LLM call belong to Classic RAG Pipeline and Context Augmentation. This lesson's job ends at the sources block.

Did it actually help?

Anecdotes are not evidence. Write down queries with the chunk a correct answer must cite, and score every variant the same way. MRR (mean reciprocal rank) averages 1/rank of the right chunk: 1.0 if it is always first, 0.5 if always second.

labeled = [   # (question, the chunk a correct answer must cite)
    ("replacement filter for XK-4471", "kb-102#0"),
    ("XK-4471 shows E-217", "kb-102#1"),
    ("my XK-4471 says the tank is full", "kb-102#1"),
    ("are filters covered by the warranty?", "kb-105#0"),
    ("how do I take the cartridge out of the XK-4417", "kb-101#0"),
    ("where can I buy an original filter", "kb-106#0"),
]

def reciprocal_rank(hits, relevant):
    ids = [f"{h.payload['doc_id']}#{h.payload['chunk_no']}" for h in hits]
    return 1 / (ids.index(relevant) + 1) if relevant in ids else 0.0

def dense_only(question, limit=5):
    return client.query_points(COLLECTION, query=dense_model.encode(question).tolist(),
                               using="dense", limit=limit).points

def rrf_scoped(question):
    code = PRODUCT_CODE.search(question)
    return hybrid(question, flt=product_scope(code.group()) if code else None, limit=5, show=False)

systems = {
    "dense only": dense_only,
    "hybrid RRF": lambda q: hybrid(q, limit=5, show=False),
    "hybrid RRF + product scope": rrf_scoped,
    "hybrid DBSF + product scope": lambda q: retrieve(q, k_context=5),
}
for label, run in systems.items():
    rr = [reciprocal_rank(run(q), rel) for q, rel in labeled]
    print(f"{label:28} MRR={sum(rr) / len(rr):.3f}  per query {[round(x, 2) for x in rr]}")
# dense only                   MRR=0.889  per query [0.33, 1.0, 1.0, 1.0, 1.0, 1.0]
# hybrid RRF                   MRR=0.917  per query [0.5, 1.0, 1.0, 1.0, 1.0, 1.0]
# hybrid RRF + product scope   MRR=0.917  per query [0.5, 1.0, 1.0, 1.0, 1.0, 1.0]
# hybrid DBSF + product scope  MRR=1.000  per query [1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

Be honest about what this shows. Only the first query ever differs. The "RRF + product scope" row is the Move 4 tie: it printed 0.917 in this run and 1.000 in another, depending on which tied chunk came out first. A metric that changes between identical runs is telling you that a tie sits exactly at the cut. DBSF breaks that tie with the BM25 margin, which is why retrieve uses it here. That is a mechanism, not proof that DBSF is the better fusion for your data. Move 4's rule still applies: keep whichever method wins on a real labeled set. And six queries on six chunks is a smoke test that catches regressions, not an evaluation. On real data, sample 50–200 questions from your logs, label the chunks that answer them, report recall@k and MRR (see Retrieval Metrics), and re-run the same script after every change of model, fusion, filter or quantization.

Your turn: change one requirement.

1. Customers type xk4471 or XK4471. What breaks, and where do you fix it?
2. Dealers log in, and each may only see documents for the product lines they sell. What changes in the payload and the query?

Final round: no label on the problem

Real incidents don't say which move they want. Read the symptom, find the cause, then pick the move.

Challenge 1: the answer at the end

A support article became one long chunk. The fix for error E-217 sits in its last sentence. Queries about E-217 never retrieve it with dense search, but BM25 finds it at once. What is happening?

filler = " ".join(["The unit ships with a power cable, a quick start card and a drain hose."] * 20)
answer = "To clear error E-217, empty the water tank and hold reset for three seconds."
long_chunk = filler + " " + answer

q, v_long, v_filler, v_short = dense_model.encode(
    ["how to clear error E-217", long_chunk, filler, answer])
print(len(dense_model.tokenizer(long_chunk)["input_ids"]), dense_model.max_seq_length)  # 360 256
print(round(float(q @ v_long), 3), round(float(q @ v_short), 3))                       # 0.016 0.853
print((abs(v_long - v_filler) < 1e-5).all())                                           # True

Challenge 2: the tutorial from 2024

A teammate pastes this retrieval function from an older blog post. It fails on the first line that talks to Qdrant.

results = client.search(
    collection_name="docs",
    query_vector=query_vector,
    query_filter=Filter(must=[FieldCondition(key="published", range=Range(gte="2024-05-01"))]),
    limit=5,
)

Challenge 3: the filter that quietly stopped working

A 5-million-chunk collection has been live for months. Product asks for a new region filter, so you run create_payload_index(..., "region", "keyword") on the live collection. Nothing errors, and latency looks fine. A week later, users in mid-sized regions (a few percent of the data each) complain that obviously relevant documents never show up, and that some searches return six or seven results where they asked for ten. Small regions are fine.

Cheat sheet: symptom → move → cost

Before moving on, rebuild the opening failure from a blank file: create the collection, ingest with stable ids, run the dense query and see the wrong product win. Then fix it twice, once with a filter and once with hybrid retrieval, and explain why each fix works and what it would miss on its own. Finally, write six labeled queries for your data and print MRR for both. If you can do that, you can make Qdrant decisions with evidence instead of defaults.

Next: Advanced Qdrant for memory tiers, multitenancy, sharding, replication, snapshots and production tuning; Hybrid Retrieval Systems for fusion and weighting in depth; Metadata & Filtering for payload design and access control.