Query Decomposition

The comparison that heard only one side

An employee asks the People-team assistant: "Compare parental leave for the second parent in Berlin and Amsterdam." The assistant is a classic RAG pipeline over the company handbook: embed the question, take the top 3 chunks, answer from them. Here is a 17-chunk slice of that handbook and the retrieval step, using the small open model all-MiniLM-L6-v2. Each chunk carries two metadata fields, the office it applies to and the year it is about, which later sections use as filters.

import numpy as np
from sentence_transformers import SentenceTransformer

# chunk id: (text, office, year). office=None means the rule applies company-wide.
DOCS = {
    "leave-policy": ("Parental leave policy: company-paid parental leave applies to all employees "
                     "after three months of service, including adoptive and foster parents. Request "
                     "it in the HR portal at least eight weeks before it starts.", None, None),
    "leave-berlin": ("Parental leave, Berlin office: the birth parent gets 20 weeks at full pay and "
                     "the second parent 6 weeks at full pay, taken within 18 months of the birth.",
                     "Berlin", None),
    "leave-berlin-return": ("Returning from parental leave in Berlin: the second parent can work "
                            "part-time for up to 6 months after the leave ends.", "Berlin", None),
    "leave-berlin-faq": ("Berlin parental leave FAQ: can I split my parental leave into two blocks? "
                         "Yes, and you can combine it with the state parental allowance.", "Berlin", None),
    "leave-amsterdam": ("Amsterdam office: after a birth, the birth parent is off for 16 weeks and "
                        "the partner for 6 weeks; the company tops both up to full salary.",
                        "Amsterdam", None),
    "leave-lisbon": ("Parental leave, Lisbon office: the birth parent gets 20 weeks at full pay and "
                     "the second parent 28 working days at full pay.", "Lisbon", None),
    "vacation-berlin": ("Vacation, Berlin office: 30 days per year. Up to 5 unused days carry over "
                        "and expire on 31 March of the next year.", "Berlin", None),
    "vacation-amsterdam": ("Vacation, Amsterdam office: 25 days per year. Unused days carry over "
                           "until 1 July of the next year.", "Amsterdam", None),
    "transfer-vacation": ("Moving to another office: when you transfer between offices, all your "
                          "unused vacation days move with you in full. The new office's carry-over "
                          "rules apply from the next calendar year.", None, None),
    "remote-2025": ("Remote work allowance, effective 1 January 2025: 50 euros per month for "
                    "internet and electricity, paid with salary.", None, 2025),
    "remote-2026": ("Remote work allowance, effective 1 January 2026 (replaces the 2025 version): "
                    "75 euros per month, paid with salary; part-time staff get it pro rata.", None, 2026),
    "equipment": ("Home-office equipment budget: 1,200 euros every three years for a desk, chair "
                  "and monitor. Anything above the budget needs your department head's approval.",
                  None, None),
    "offsite-2025": ("Engineering offsite 2025 was held in Berlin, 8-10 October 2025.", None, 2025),
    "offsite-2026": ("Engineering offsite 2026: the engineering organisation meets in Lisbon, "
                     "12-14 October 2026. The travel desk books flights and hotels.", None, 2026),
    "perdiem-lisbon": ("Travel per diem, Lisbon: 55 euros per day for meals.", "Lisbon", None),
    "perdiem-berlin": ("Travel per diem, Berlin: 60 euros per day for meals.", "Berlin", None),
    "perdiem-amsterdam": ("Travel per diem, Amsterdam: 65 euros per day for meals.", "Amsterdam", None),
}
IDS = list(DOCS)
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
DOC_VECS = model.encode([DOCS[i][0] for i in IDS])     # unit-length rows (the model normalizes)

def search(query, k=3, office=None, year=None):
    """Top-k chunk ids by cosine. office/year are exact metadata filters (None = no filter)."""
    scores = DOC_VECS @ model.encode(query)
    hits = []
    for i in np.argsort(-scores):
        _, doc_office, doc_year = DOCS[IDS[i]]
        if office is not None and doc_office != office:
            continue
        if year is not None and doc_year != year:
            continue
        hits.append((IDS[i], round(float(scores[i]), 3)))
    return hits[:k]

question = "Compare parental leave for the second parent in Berlin and Amsterdam."
for rank, (doc_id, score) in enumerate(search(question, k=5), start=1):
    print(rank, doc_id, score)

Predict first: which three chunks reach the prompt? Where does leave-amsterdam, the only chunk that can answer the Amsterdam half, land?

1 leave-berlin-return 0.712
2 leave-berlin 0.67
3 leave-lisbon 0.635
4 leave-berlin-faq 0.617
5 leave-amsterdam 0.569

The prompt now holds Berlin's numbers and Lisbon's. A careful generator writes "The second parent gets 6 weeks in Berlin; the handbook excerpts do not cover Amsterdam." A less careful one reads Lisbon's "28 working days" as the other side of the comparison. Either way the user gets half an answer to a question the handbook answers in full.

Why the ranking went this way. One ranked list is serving two needs. The words the needs share, "parental leave for the second parent", dominate the query vector. Berlin's and Lisbon's chunks use exactly those words; Amsterdam's chunk says "the partner" and "after a birth". The two city names are 2 of the question's 11 words. So the need whose chunks match the shared wording takes every slot. Lexical retrieval does not rescue it: BM25 (rank_bm25, lowercased word tokens) also ranks leave-amsterdam 5th, and so does Reciprocal Rank Fusion of the two lists. Raising k to 5 would catch it in this toy. In a real handbook, where the Berlin headquarters has thirty parental-leave pages, no k you can afford would.

The fix is to stop making two needs compete for one list:

Query decomposition: rewriting one question into several sub-queries (sub-questions), each retrieved on its own, recording which sub-query needs another's answer, and merging the evidence so the answer covers every part.

It sits between two lessons you already know:

• Query Understanding & Intent rewrites one query for one need: follow-ups, typos, filters, HyDE.
• Agentic RAG Systems lets the model pick the next search after reading each result.
• Decomposition writes all the steps up front, then your code runs them, often in parallel.

In the vocabulary of Anthropic's Building effective agents (December 2024), a fixed split ("one search per office named") is sectioning, a parallelization workflow: "Breaking a task into independent subtasks run in parallel." A split written by a model for each question is an orchestrator-workers workflow adapted to retrieval, where "subtasks aren't pre-defined, but determined by the orchestrator based on the specific input." (In the post the workers are LLM calls and the orchestrator also combines their results; here most workers are plain searches run by your code.) Neither is an agent loop: once the plan exists, the model does not choose what to search next.

Here is the map. Each move starts from a symptom you can see in results, traces or metrics:

After the moves come the bill (what a plan costs in tokens and seconds) and how to measure a decomposer. This lesson assumes the agent loop and multi-hop retrieval from Agentic RAG Systems and single-query rewriting from Query Understanding & Intent.

Before any split: does this question need one?

Most questions do not. Sort real questions by shape and give each shape the cheapest design that works:

The rule behind the table has two conditions, and a split needs both. The answer combines parts that different documents answer on their own: you read that off the question. And one search leaves one of those parts out: you measure that on labeled questions of the same shape (the callout below). The first condition alone is not enough. The remote-allowance question has two parts from two documents, and one search already returns both, so a split would only add calls. Never split when one document answers the whole question, when the "parts" are constraints on a single need, or when the answer is a rule about how the parts relate.

The shapes are not this lesson's invention. The MultiHop-RAG benchmark (Tang & Yang, 2024) groups multi-document questions into inference, comparison, temporal and null queries, where null means the corpus holds no answer, and reports that existing RAG methods "perform unsatisfactorily" on them. The null type matters here: a part with no evidence must stay visible as "not covered" (Move 4).

Your turn. Which of these would you split, and how?

1. "Is the Amsterdam per diem higher than Lisbon's?"
2. "Do part-time staff in Lisbon get the remote work allowance?"
3. "Which office gives the most vacation days?"
4. "When is the 2026 offsite, and what per diem do I get there?"

💡 Measure before you build. Label 100–200 real questions by shape and compute the one-search pipeline's accuracy per shape. If comparisons are 3% of traffic and already right 90% of the time, a planner is a small win at a large cost. The same advice, for agent loops, is in Agentic RAG Systems.

Move 1: Give every part its own retrieval

Symptom: one side of a comparison, or one of several questions, never reaches the top k.

Write one sub-query per part and retrieve each on its own. Continuing from the block above:

sub_queries = {
    "Berlin": "parental leave for the second parent, Berlin office",
    "Amsterdam": "parental leave for the second parent, Amsterdam office",
}
for office, sub_query in sub_queries.items():
    print(f"{office:9} text only  :", search(sub_query, k=3))
    print(f"{office:9} + filter   :", search(sub_query, k=3, office=office))

Berlin    text only  : [('leave-berlin-return', 0.802), ('leave-berlin', 0.753), ('leave-berlin-faq', 0.731)]
Berlin    + filter   : [('leave-berlin-return', 0.802), ('leave-berlin', 0.753), ('leave-berlin-faq', 0.731)]
Amsterdam text only  : [('leave-lisbon', 0.669), ('leave-berlin-return', 0.644), ('leave-amsterdam', 0.618)]
Amsterdam + filter   : [('leave-amsterdam', 0.618), ('vacation-amsterdam', 0.472), ('perdiem-amsterdam', 0.305)]

The Amsterdam part now has its own list, and leave-amsterdam is in it. Look closer, though: with text only, Lisbon still ranks first for the Amsterdam sub-query, and a Berlin chunk second. The split gave Amsterdam three slots of its own. It did not make the sub-query any better at telling Amsterdam from Lisbon: "Amsterdam" is still one word in a query about "parental leave for the second parent". When a part is defined by a metadata value (an office, a year, a product line), a filter scopes it exactly and puts leave-amsterdam first. Text in the query only nudges.

Why it works

Each part gets its own k slots, so no part can take another's. The retrieval budget is allocated per need instead of being fought over. Each sub-query also drops the other part's words, so its vector points at one need.

What it costs, and the other fix

One retrieval per part. The retrievals are independent, so they run in parallel and cost the time of the slowest one. Fan-out tail latency and per-source timeouts are covered in Move 3 of Agentic RAG Systems. The real cost is whatever writes the sub-queries. A template for a recurring shape ("compare X across offices A and B" gives one step per office) costs no LLM call at all. A model-written split costs one call before any retrieval starts (the bill, below).

The honest alternative is a bigger candidate pool plus a reranker. Here k = 5 would catch Amsterdam. But a reranker reorders what the first stage returned; it cannot bring back a chunk that never made the pool (Reranking Models), and every extra chunk in the prompt is paid for on every query. Try both on your labeled comparison questions.

Measure it: part coverage on labeled comparison questions, with and without the split. For each question, take the share of its parts that have at least one relevant chunk in the final prompt, then average over the questions.

Move 2: Let one answer feed the next

Symptom: a part needs a value that only another document supplies, so you cannot write the second search until the first one returns.

"What per diem do I get at the 2026 engineering offsite?" No chunk mentions both the offsite and a per diem. Step 1 finds the offsite city; step 2 searches for that city's per diem. This is the bridge pattern from Agentic RAG Systems, planned before any retrieval instead of discovered in a loop.

OFFICES = ("Berlin", "Amsterdam", "Lisbon")

def extract_office(chunk_id):
    """Stands in for a small LLM call: 'Which office city does this text name? Name only.'"""
    named = [o for o in OFFICES if o in DOCS[chunk_id][0]]
    if len(named) != 1:
        raise ValueError(f"{chunk_id}: expected one city, found {named}")
    return named[0]

def per_diem_at_offsite(year=None):
    hop1 = search("Where is the 2026 engineering offsite held?", k=1, year=year)
    city = extract_office(hop1[0][0])                  # the bridge value
    hop2 = search(f"travel per diem in {city}", k=1)   # step 2 is written with it
    return hop1, city, hop2

print("no filter:", per_diem_at_offsite())
print("year=2026:", per_diem_at_offsite(year=2026))

Predict: the query text says "2026". Without the year filter, which city does the chain find?

no filter: ([('offsite-2025', 0.802)], 'Berlin', [('perdiem-berlin', 0.828)])
year=2026: ([('offsite-2026', 0.737)], 'Lisbon', [('perdiem-lisbon', 0.87)])

That is entity drift at the bridge: one wrong value turns every later step into a precise answer to the wrong question. Agentic RAG Systems showed that errors compound across hops (0.9 per hop gives about 0.73 over three hops). Here the fix is cheap and exact: a year in the query text is a hint, a year in the filter is a constraint.

Slots, not guesses

Write the dependent step with a slot: travel per diem in {s1}. The slot says two things your code can check: this step waits for s1, and the value comes from s1's evidence. Never let the planner fill it in. A language model knows a lot about engineering offsites in general and nothing about yours. Asked for a complete plan, it may write "per diem in Berlin" because Berlin appears in the question, or because it hosted last year. Move 5 catches that.

What fills a slot:

• Code, when the value has a known shape: a city from a fixed list, a ticket id matched by a regex, a key looked up in a catalog. Free and deterministic.
• A small extraction call otherwise: "From this text, which city hosts the offsite? Answer with the name only, or UNKNOWN." It costs a model round trip on the critical path.

Either way, refuse to guess. extract_office raises when a chunk names zero or two cities. Carry ids rather than names between steps, and when a step returns two candidates, follow both or ask the user. Both rules come from Agentic RAG Systems.

Least-to-most, and where planning stops

Least-to-most prompting (Zhou et al., ICLR 2023) is the classic form of this idea. The model first breaks a problem into simpler subproblems, then solves them in order, with the answers to earlier subproblems in the prompt for later ones. With code-davinci-002 and 14 examples, the authors report at least 99% accuracy on every split of the SCAN compositional-generalization benchmark, against 16% for chain-of-thought. Those are their benchmark numbers, not a forecast for your handbook.

In retrieval, the two styles differ in what flows between steps:

A plan is a bet that you know the shape of the answer before you retrieve anything. When you do not, for example "why did checkout fail last night?", where the next search depends on what the logs say, hand the question to the agent loop in Agentic RAG Systems.

Your turn. "How much parental leave does the second parent get at the office that hosted the 2025 offsite?" Write the steps. Which step can start at once? How many LLM calls sit on the critical path, counting the planner and the answer?

Measure it: bridge accuracy, meaning the share of labeled chains whose slot value is right (the city, the id), counted separately from final-answer accuracy. A wrong slot explains a wrong answer before you look at anything else.

Move 3: Write the plan down, then run it as a graph

Symptom: dependencies live in ad-hoc code, independent steps wait for each other, and nobody can tell from a log why step 3 ran after step 2.

A realistic question mixes the shapes:

"I work in Berlin and I'm moving to the office that hosts the 2026 engineering offsite. How much parental leave does the second parent get there compared with Berlin, and what happens to my unused vacation days?"

That is a bridge (which office?), a comparison (that office against Berlin), and a separate question about the move, which depends on the bridge too. Write it as data, a plan: a list of steps, each with an id, a query that may contain slots, the steps it depends on, and its filters.

s1  "Where is the engineering offsite held?"      year=2026 ──▶ extract city
 ├─▶ s2  "parental leave for the second parent, {s1} office"      office={s1}
 └─▶ s4  "unused vacation days when transferring from Berlin to {s1}"
s3  "parental leave for the second parent, Berlin office"   office=Berlin   (no inputs)

time  0.0 ── s1 retrieve ── 0.1 ── s1 extract (LLM) ── 0.9 ── s2, s4 retrieve ── 1.0 s
      0.0 ── s3 retrieve ── 0.1

The plan is a small directed acyclic graph (DAG): edges point from a step to the steps that need its answer. The executor below validates the plan before it costs anything, then starts every step as soon as its own inputs are ready. The timings use assumed latencies: 0.1 s per retrieval round trip and 0.8 s per extraction call (0.6 s to the first token plus 20 tokens at 100 tokens per second). asyncio.sleep stands in for both network calls.

import asyncio, re, time
from graphlib import TopologicalSorter

moving_question = ("I work in Berlin and I'm moving to the office that hosts the 2026 engineering "
            "offsite. How much parental leave does the second parent get there compared with "
            "Berlin, and what happens to my unused vacation days?")

plan = {  # what the planner returned (canned here, so the lesson runs offline)
    "split": True,
    "steps": [
        {"id": "s1", "query": "Where is the engineering offsite held?",
         "depends_on": [], "office": None, "year": 2026},
        {"id": "s2", "query": "parental leave for the second parent, {s1} office",
         "depends_on": ["s1"], "office": "{s1}", "year": None},
        {"id": "s3", "query": "parental leave for the second parent, Berlin office",
         "depends_on": [], "office": "Berlin", "year": None},
        {"id": "s4", "query": "unused vacation days when transferring from Berlin to {s1}",
         "depends_on": ["s1"], "office": None, "year": None},
    ],
}

MAX_STEPS = 6
SLOT = re.compile(r"\{(s\d+)\}")

def validate(plan):
    """Reject a malformed plan before it costs anything. Returns an execution order."""
    steps = plan["steps"]
    ids = [s["id"] for s in steps]
    if not 1 <= len(steps) <= MAX_STEPS:
        raise ValueError(f"{len(steps)} steps; allowed 1-{MAX_STEPS}")
    if len(set(ids)) != len(ids):
        raise ValueError("duplicate step ids")
    for s in steps:
        unknown = set(s["depends_on"]) - set(ids)
        if unknown:
            raise ValueError(f"{s['id']} depends on unknown steps {unknown}")
        used = set(SLOT.findall(s["query"] + (s["office"] or "")))
        if not used <= set(s["depends_on"]):
            raise ValueError(f"{s['id']} uses {used - set(s['depends_on'])} without waiting for it")
        if s["office"] and s["office"] not in OFFICES and not SLOT.fullmatch(s["office"]):
            raise ValueError(f"{s['id']}: unknown office {s['office']!r}")
    graph = {s["id"]: set(s["depends_on"]) for s in steps}
    return list(TopologicalSorter(graph).static_order())      # raises CycleError on a cycle

RETRIEVE_S, EXTRACT_S = 0.10, 0.80    # assumed latencies: one retrieval, one small LLM call

async def execute(plan, k=2):
    order = validate(plan)
    steps = {s["id"]: s for s in plan["steps"]}
    feeds = {d for s in plan["steps"] for d in s["depends_on"]}   # steps whose answer is needed
    tasks, t0 = {}, time.perf_counter()

    async def run(step):
        answers = {d: (await tasks[d])["answer"] for d in step["depends_on"]}
        fill = lambda text: SLOT.sub(lambda m: answers[m.group(1)], text)
        start = time.perf_counter() - t0
        await asyncio.sleep(RETRIEVE_S)                 # stands in for the retrieval round trip
        office = fill(step["office"]) if step["office"] else None
        hits = search(fill(step["query"]), k=k, office=office, year=step["year"])  # local, ~15 ms
        answer = None
        if step["id"] in feeds:                         # a later step needs a value from here
            await asyncio.sleep(EXTRACT_S)              # stands in for the extraction call
            answer = extract_office(hits[0][0])
        return {"query": fill(step["query"]), "hits": [d for d, _ in hits], "answer": answer,
                "start": start, "end": time.perf_counter() - t0}

    for sid in order:                   # create every task; each one waits only for its own inputs
        tasks[sid] = asyncio.create_task(run(steps[sid]))
    return {sid: await task for sid, task in tasks.items()}

t0 = time.perf_counter()
results = asyncio.run(execute(plan))
for sid, r in sorted(results.items()):
    print(f"{sid} {r['start']:.2f}-{r['end']:.2f} s  {r['query']!r} -> {r['hits']}"
          + (f" answer={r['answer']}" if r["answer"] else ""))
print(f"wall clock {time.perf_counter() - t0:.2f} s")

Predict: which steps start at time 0? When do s2 and s4 start, and how long does the whole plan take?

s1 0.00-0.92 s  'Where is the engineering offsite held?' -> ['offsite-2026', 'remote-2026'] answer=Lisbon
s2 0.92-1.04 s  'parental leave for the second parent, Lisbon office' -> ['leave-lisbon', 'perdiem-lisbon']
s3 0.00-0.13 s  'parental leave for the second parent, Berlin office' -> ['leave-berlin-return', 'leave-berlin']
s4 0.92-1.05 s  'unused vacation days when transferring from Berlin to Lisbon' -> ['transfer-vacation', 'vacation-berlin']
wall clock 1.06 s

Parallel execution saved 0.2 s. The one extraction call costs 0.8 s. That is the general shape: a plan's latency is set by the LLM calls on its longest path, not by how many retrievals it makes. RAG Architecture & Implementation makes the same point about agents: the time goes to model calls, not to searching.

Three details in the executor matter more than any framework:

• Each step waits only for its own inputs. Running the plan in rigid waves ("all level-1 steps, then all level-2 steps") is simpler but makes a quick step wait for a slow sibling it does not depend on. LLMCompiler (Kim et al., ICML 2024) builds exactly this: a planner emits a DAG of function calls, and a task-fetching unit dispatches each call once its dependencies are met. Against ReAct, the authors report up to 3.7× lower latency, up to 6.7× lower cost and up to about 9% higher accuracy on their benchmarks.
• Validation is not optional. Without validate, a plan in which s1 waits for s2 and s2 waits for s1 does not crash: both tasks wait forever. A step that uses {s1} without listing it in depends_on starts before s1 has an answer (the second plan below shows what happens then).
• The step cap lives in code. MAX_STEPS bounds the fan-out whatever the planner writes. The steps are flat here; if you let a step be decomposed again into sub-plans, cap the depth too.

Predict: what does validate say about these two plans?

cyclic = {"split": True, "steps": [
    {"id": "s1", "query": "travel per diem in {s2}", "depends_on": ["s2"], "office": None, "year": None},
    {"id": "s2", "query": "offsite held in {s1}", "depends_on": ["s1"], "office": None, "year": None}]}
eager = {"split": True, "steps": [
    {"id": "s1", "query": "Where is the engineering offsite held?", "depends_on": [], "office": None, "year": 2026},
    {"id": "s2", "query": "travel per diem in {s1}", "depends_on": [], "office": None, "year": None}]}
for p in (cyclic, eager):
    try:
        validate(p)
    except Exception as e:
        print(type(e).__name__, e)

CycleError ('nodes are in a cycle', ['s1', 's2', 's1'])
ValueError s2 uses {'s1'} without waiting for it

⚠️ A feedback loop in the world is not a cycle in the plan. "How does climate change affect farming, and how does farming affect the climate?" has two sub-questions that can both be written up front. They are independent, so they run in parallel. A plan cycle means each step needs the other's answer before its query can be written.

The real call

The canned plan above stands in for one model call. With Anthropic's Messages API and structured outputs, the response is constrained to your JSON Schema through output_config.format, with no beta header as of September 2026. The sketch needs pip install anthropic and an API key, and reuses validate from above. The model id is an example as of September 2026; pick a current model and measure it on your labeled questions.

import json
import anthropic

MODEL = "claude-sonnet-5"      # example id as of September 2026; pick a current model and measure

def nullable(schema):
    return {"anyOf": [schema, {"type": "null"}]}

PLAN_SCHEMA = {
    "type": "object",
    "properties": {
        "split": {"type": "boolean"},
        "steps": {"type": "array", "items": {
            "type": "object",
            "properties": {
                "id": {"type": "string"},
                "query": {"type": "string"},
                "depends_on": {"type": "array", "items": {"type": "string"}},
                "office": nullable({"type": "string"}),
                "year": nullable({"type": "integer"}),
            },
            "required": ["id", "query", "depends_on", "office", "year"],
            "additionalProperties": False,
        }},
    },
    "required": ["split", "steps"],
    "additionalProperties": False,
}

RULES = """You plan retrieval over a company HR handbook. Offices: Berlin, Amsterdam, Lisbon.
Return steps s1, s2, ... Each step is one search.
- Split only when the question has parts that can each be answered on their own: a comparison,
  separate questions, separate periods. If one policy could answer the whole question, set
  split to false and return one step with the question unchanged.
- If a step needs another step's answer, list that step in depends_on and write {s1} (etc.)
  where the answer goes. Never fill in such a value yourself.
- Keep every office, year, role and number from the question in the steps that need it.
  Add none.
- Put an office in "office" and a year in "year"; text in the query is not a filter."""

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY from the environment

def llm_plan(question):
    response = client.messages.create(
        model=MODEL,
        max_tokens=16000,
        system=RULES,
        messages=[{"role": "user", "content": question}],
        output_config={"format": {"type": "json_schema", "schema": PLAN_SCHEMA},
                       "effort": "low"},
    )
    if response.stop_reason != "end_turn":      # max_tokens or refusal: the JSON may be incomplete
        raise RuntimeError(f"no plan: {response.stop_reason}")
    plan = json.loads(next(b.text for b in response.content if b.type == "text"))
    validate(plan)                              # shape is guaranteed; sense is not
    return plan

What the schema can and cannot do:

• Every object needs additionalProperties: false, and the model can only return the fields you defined.
• The schema cannot express numeric ranges, and arrays accept minItems of only 0 or 1, so "at most 6 steps" is enforced by validate, not by the schema.
• Structured outputs guarantee the plan's shape, not its sense. A well-formed plan can still drop a year or invent an office. That is Move 5's job.
• The model may think before answering, so the text block is not necessarily the first content block; the code picks it by type. effort: "low" keeps planning short, and planning latency matters (see the bill).

The rules state only the first of the two split conditions, the shape. The planner cannot see whether one search would already return every part. Routing handles that condition before the planner is called: send it only the shapes that your labeled set shows being crowded out (the bill, below).

Other providers offer equivalent JSON-schema modes, and the pattern carries over. The closest research ancestor of this plan format is ReWOO (Xu et al., 2023). A planner writes every step up front with placeholders (#E1, #E2) for tool results, workers fill them, and a solver writes the answer. The authors report 5× token efficiency and a 4% accuracy gain on HotpotQA compared with a ReAct-style loop that reasons after every observation.

Measure it: the plan validity rate (how often validate rejects the planner's output), the distribution of steps per plan, and measured latency against the longest path you computed.

Move 4: Merge by part, then reconcile

Symptom: every part was retrieved, and the prompt, or the answer, still lost one.

Predict: take the two filtered lists from Move 1 (three chunks per office) and room for three chunks in the prompt. You pool the six chunks and keep the three most similar to the original question. Which part vanishes?

lists = {office: search(q, k=3, office=office) for office, q in sub_queries.items()}
by_question = dict(search(question, k=len(DOCS)))    # each chunk's score against the ORIGINAL question
pool = {doc for hits in lists.values() for doc, _ in hits}
resorted = sorted(pool, key=lambda d: -by_question[d])[:3]

def round_robin(lists, budget):
    """Rank 1 of every part first, then rank 2 of every part, ... skipping duplicates."""
    picked = []
    for rank in range(max(len(hits) for hits in lists.values())):
        for hits in lists.values():
            if rank < len(hits) and hits[rank][0] not in picked:
                picked.append(hits[rank][0])
                if len(picked) == budget:
                    return picked
    return picked

print("re-sorted  :", resorted)
print("round-robin:", round_robin(lists, 3))

re-sorted  : ['leave-berlin-return', 'leave-berlin', 'leave-berlin-faq']
round-robin: ['leave-berlin-return', 'leave-amsterdam', 'leave-berlin']

The same trap waits behind a reranker. A cross-encoder scoring the whole pool against the original question can crowd out the smaller side just as well (Reranking Models). The fix is the quota, not a different reranking query: rank within each part, and give every part at least one slot before the rest of the budget goes by score. Inside a part, scoring against the original question is a good default. It follows the rule from Agentic RAG Systems of judging evidence against the original question, and it still sees qualifiers such as "second parent" that a terse sub-query may have dropped.

A quota protects a part only if the part's candidates belong to it. Rank the Amsterdam sub-query's text-only list from Move 1 against the original question, and leave-berlin-return comes first, because the question names Berlin too. With the office filter, all three candidates are Amsterdam's, and leave-amsterdam comes first whichever query you rank by.

Agentic RAG Systems reranks its merged fan-out pool against the original question when every source serves the same need. When the lookups answer different parts of one question, as here, add the quota first.

Lay the prompt out by part, and put the original question on top. The sub-queries are how you searched, not what the user asked. Qualifiers that no step's filter carried, such as "second parent", reach the answer this way. Here is the layout for the Move 3 plan. The executor also returned a second hit for s1, s2 and s4 (remote-2026, perdiem-lisbon, vacation-berlin); they are left out here to keep it short:

Question: {the user's original question}
Answer every part. If a part has no evidence below, say the handbook does not cover it.
Cite chunk ids like [leave-lisbon].

Part s1: where the engineering offsite is held (2026)
[offsite-2026] Engineering offsite 2026: the engineering organisation meets in Lisbon, ...
Part s2: parental leave for the second parent, Lisbon office
[leave-lisbon] Parental leave, Lisbon office: ... the second parent 28 working days at full pay.
Part s3: parental leave for the second parent, Berlin office
[leave-berlin-return] Returning from parental leave in Berlin: ...
[leave-berlin] Parental leave, Berlin office: ... the second parent 6 weeks at full pay ...
Part s4: unused vacation days when transferring from Berlin to Lisbon
[transfer-vacation] Moving to another office: ... all your unused vacation days move with you in full ...

Reconcile before you compare

Sub-answers come from different documents, written by different people at different times.

• Units. Lisbon gives the second parent "28 working days", Berlin "6 weeks". With a five-day working week, 28 working days is 5.6 weeks, so Berlin gives slightly more. Convert before comparing, in code or in the answer instructions, and state the assumption.
• Versions. Both remote-allowance chunks match a question about the allowance, and the 2026 one says it replaces the 2025 version. Prefer the version in force by its effective date, and say which you used. Version conflicts in the prompt are handled in Context Augmentation; keeping superseded versions marked in the index is Data Freshness & Lifecycle.
• Missing parts. A part with no evidence stays in the answer as "not covered". Never fill it from another part's evidence: Lisbon's policy is not Amsterdam's.
• Disagreements. Two chunks for the same part say different things: show both with their ids instead of averaging them.

Pooled evidence or per-part answers? Some pipelines answer each part with its own small LLM call and then write the final answer from those sub-answers alone. The final prompt is short and the parts are easy to line up. But it costs a call per part, and the final writer never sees the chunks. Move 6 shows a rule that gets lost that way. Default to pooling the chunks into one answer call. Answer per part only when the evidence is too large for one prompt.

Your turn. Three offices, three chunks per office, and room for five chunks. Which ranks enter the prompt under round-robin? And which office gives the second parent more leave, Lisbon or Amsterdam?

Measure it: part coverage in the final prompt, not just in retrieval (a part can be retrieved and then lost at merge, which the failure map in RAG Architecture & Implementation calls ranked out), and the share of answers that address every part, judged against labels.

Move 5: Check the plan against the question

Symptom: the plan runs and the answer is fluent, but it answers a slightly different question: a year disappeared, or an office the user never named appeared.

A planner is a rewriter, so it inherits the two failures of any rewrite: dropping a constraint and inventing one. Query Understanding & Intent checks a single rewrite for both. Here you check the whole plan: every constraint in the question must appear in at least one step, and no step may name an entity the question did not, unless the entity arrives through a slot.

def constraints(text):
    """Offices and years named in a text. Extend with your own entity types."""
    found = {("office", o) for o in OFFICES if o in text}
    found |= {("year", int(y)) for y in re.findall(r"\b20\d\d\b", text)}
    return found

def check(question, plan):
    need = constraints(question)
    covered, invented, year_in_text_only = set(), set(), set()
    for s in plan["steps"]:
        in_text = constraints(SLOT.sub("", s["query"]))          # {s1} slots are filled later
        in_filter = set()
        if s["office"] and not SLOT.fullmatch(s["office"]):
            in_filter.add(("office", s["office"]))
        if s["year"] is not None:
            in_filter.add(("year", s["year"]))
        covered |= in_text | in_filter
        invented |= (in_text | in_filter) - need
        year_in_text_only |= {c for c in in_text - in_filter if c[0] == "year"}
    return {"dropped": need - covered, "invented": invented, "year only in text": year_in_text_only}

guessing_plan = {"split": True, "steps": [
    {"id": "s1", "query": "parental leave for the second parent, Amsterdam office",
     "depends_on": [], "office": "Amsterdam", "year": None},
    {"id": "s2", "query": "parental leave for the second parent, Berlin office",
     "depends_on": [], "office": "Berlin", "year": None},
    {"id": "s3", "query": "unused vacation days when transferring between offices",
     "depends_on": [], "office": None, "year": None},
]}
print("plan         :", check(moving_question, plan))
print("guessing_plan:", check(moving_question, guessing_plan))

plan         : {'dropped': set(), 'invented': set(), 'year only in text': set()}
guessing_plan: {'dropped': {('year', 2026)}, 'invented': {('office', 'Amsterdam')}, 'year only in text': set()}

guessing_plan is what a planner writes when it fills the bridge from its own imagination. It skipped the offsite lookup and assumed the new office is Amsterdam. Nothing would crash: retrieval would happily return Amsterdam's policy, and the answer would compare the wrong office with Berlin. The check flags both symptoms of the one mistake. The year 2026 is dropped, because the step that needed it is gone, and Amsterdam is invented. The third key enforces Move 2's finding: a year that appears only in query text is reported, because text does not filter.

When the check fails:

1. Re-plan once, with the complaint in the prompt: "Your plan dropped year 2026 and names Amsterdam, which the question does not mention."
2. Fall back to one search with the original question if the second plan also fails, and say the answer may be incomplete.
3. Log it. The failure rate per question shape is a planner metric.

What this check cannot see: qualifiers ("second parent", "part-time", "adoptive") and relations ("more than", "before", "moving from A to B"). Add your domain's role terms to the lexicon. Or add an LLM check ("List every condition in the question that no step covers"), which costs a call. Either way, keep the original question in the answer prompt (Move 4), so a qualifier that no step carried still reaches the answer.

Measure it: seed 50 plans with known defects (drop a year, swap an office, delete a step, fill a slot by hand) and count how many the check catches. Then run it on 50 good plans and count false alarms. A check that cries wolf gets switched off.

Move 6: Know when not to split

Symptom: a question the one-search pipeline answered correctly gets a worse answer after decomposition ships.

"Can adoptive parents in Berlin and Amsterdam take parental leave?" Two offices, so a planner trained on comparisons splits it, one step per office with an office filter. Continuing with execute from Move 3:

adoption = "Can adoptive parents in Berlin and Amsterdam take parental leave?"
split_plan = {"split": True, "steps": [
    {"id": "s1", "query": "parental leave for adoptive parents, Berlin office",
     "depends_on": [], "office": "Berlin", "year": None},
    {"id": "s2", "query": "parental leave for adoptive parents, Amsterdam office",
     "depends_on": [], "office": "Amsterdam", "year": None},
]}
print("one search :", [d for d, _ in search(adoption, k=3)])
split = asyncio.run(execute(split_plan, k=2))
print("split      :", {sid: r["hits"] for sid, r in split.items()})
with_original = {"split": True, "steps": split_plan["steps"] + [
    {"id": "s0", "query": adoption, "depends_on": [], "office": None, "year": None}]}
kept = asyncio.run(execute(with_original, k=2))
print("+ original :", {sid: r["hits"] for sid, r in kept.items()})

one search : ['leave-policy', 'leave-berlin', 'leave-berlin-return']
split      : {'s1': ['leave-berlin', 'leave-berlin-return'], 's2': ['leave-amsterdam', 'vacation-amsterdam']}
+ original : {'s1': ['leave-berlin', 'leave-berlin-return'], 's2': ['leave-amsterdam', 'vacation-amsterdam'], 's0': ['leave-policy', 'leave-berlin']}

One search puts the answer first: the company-wide policy says "including adoptive and foster parents". The split can never find it, because that chunk belongs to no office, and both filters exclude it by construction. The split answer will say "neither office's policy mentions adoption", which is fluent, cited and wrong. The plan assumed the answer lives per office. It lives company-wide.

Two cheap fixes:

• Keep the original question as a step. One extra retrieval, with no LLM call, brings the company-wide chunk back (s0). Start it while the planner is still writing, and it adds no latency either.
• Scope parts as "this office or company-wide" when your metadata allows an OR condition (Metadata & Filtering).

Your turn. "What happens to my unused vacation days if I move from Berlin to Amsterdam?" A planner splits it into a Berlin step and an Amsterdam step, each with an office filter, and asks a small model to answer each step before the final answer is written. What does the user get?

The other ways a split hurts:

• Over-decomposition. Fragments that are each answerable and jointly useless. "How do I export my data to CSV?" split into "What is CSV?" and "What is data export?" retrieves two glossary entries and loses the how-to. More steps than the question has needs is the signature.
• Cost blow-up. Fan-out multiplied by dependency depth, with every extra step paid in planner output tokens (the bill, next).

Signals that decomposition is hurting: parts answered "not covered" that the one-search pipeline answered; steps whose top chunks are identical; plans with more steps than the question names entities; and, on your labeled set, accuracy per shape falling for shapes that do not need splitting.

The bill: what a plan costs

Symptom: time to first token triples after decomposition ships, and cost per 1,000 queries climbs with it.

The same prices and speeds as in Agentic RAG Systems, with this lesson's token counts. Every number is an assumption for the exercise, not a price quote:

# Assumptions (round numbers for the exercise, not a price quote), as in Agentic RAG Systems:
PRICE_IN, PRICE_OUT = 3.00, 15.00   # USD per million input / output tokens
TTFT, SPEED = 0.6, 100              # s to first token; output tokens per s
RETRIEVE = 0.10                     # s per retrieval round; parallel retrievals overlap
CHUNK = 250                         # tokens per chunk in a prompt
ANSWER_IN, ANSWER_OUT = 400, 300    # answer prompt without chunks; the answer itself
PLAN_IN, PER_STEP = 700, 40         # planner prompt (rules + schema + question); JSON tokens per step
EXTRACT = (650, 20)                 # extraction call: 2 chunks in, one short value out
PART = (650, 60)                    # answering one part on its own: 2 chunks in, 60 tokens out

def secs(tokens_out):
    return TTFT + tokens_out / SPEED

def design(calls_before, path_before, answer_in):
    """calls_before: (in, out) of every LLM call before the answer; path_before: seconds on the
    longest path until the answer call starts; answer_in: tokens in the answer prompt."""
    calls = calls_before + [(answer_in, ANSWER_OUT)]
    tin, tout = sum(c[0] for c in calls), sum(c[1] for c in calls)
    usd = (tin * PRICE_IN + tout * PRICE_OUT) / 1e6 * 1000
    return len(calls), tin, tout, usd, path_before + TTFT, path_before + secs(ANSWER_OUT)

plan_call = lambda n: (PLAN_IN, PER_STEP * n)
rows = {
    "one search, 4 chunks": design([], RETRIEVE, ANSWER_IN + 4 * CHUNK),
    "2 parts x 2 chunks":   design([plan_call(2)], secs(80) + RETRIEVE, ANSWER_IN + 4 * CHUNK),
    "4-step plan, 1 bridge": design([plan_call(4), EXTRACT],
                                    secs(160) + RETRIEVE + secs(20) + RETRIEVE, ANSWER_IN + 8 * CHUNK),
    "same, parts answered":  design([plan_call(4), EXTRACT] + [PART] * 4,
                                    secs(160) + RETRIEVE + secs(20) + RETRIEVE + secs(60),
                                    ANSWER_IN + 4 * 60),
    "12 parts x 2 chunks":  design([plan_call(12)], secs(480) + RETRIEVE, ANSWER_IN + 24 * CHUNK),
}
print(f"{'design':23} calls  in_tok out_tok  USD/1k  first_s  total_s")
for name, (n, tin, tout, usd, first, total) in rows.items():
    print(f"{name:23} {n:5} {tin:7,} {tout:7} {usd:7.2f} {first:8.1f} {total:8.1f}")

design                  calls  in_tok out_tok  USD/1k  first_s  total_s
one search, 4 chunks        1   1,400     300    8.70      0.7      3.7
2 parts x 2 chunks          2   2,100     380   12.00      2.1      5.1
4-step plan, 1 bridge       3   3,750     480   18.45      3.8      6.8
same, parts answered        7   4,590     720   24.57      5.0      8.0
12 parts x 2 chunks         2   7,100     780   33.00      6.1      9.1

How to read it:

• Splitting in two adds one planner call. That is 38% more cost per 1,000 queries (12.00 against 8.70 USD), and the first answer token arrives at 2.1 s instead of 0.7 s. The planner's 80 output tokens alone take 0.8 s to generate.
• The Move 3 plan puts the planner, one extraction and two retrieval rounds on the critical path before the answer can start: 3.8 s to the first token.
• Answering each part separately adds four calls and 6.12 USD per 1,000 queries, and pushes the first token to 5.0 s.
• Twelve parts (say, four topics across three offices) make the plan itself 480 output tokens: 4.8 s of generation before a single search runs.

The rule of thumb: the time before the answer starts is the planning call (time to first token plus plan tokens divided by output speed), plus, for each level of the longest chain, one retrieval round and one extraction call where a later step needs a value. Parallel retrievals overlap; LLM calls on the chain do not.

The levers, and what each costs you:

• Route. Send only the shapes that need a split to the planner (see "Measure it" below). You pay with a router to maintain and measure. And watch the p95: if more than 5% of queries take the split path, the 95th-percentile query is a split query and carries the full planning time, however cheap the average looks (Query Understanding & Intent makes the same point about rewriting).
• Templates for recurring shapes. "Compare X across offices A and B" can produce its plan with no model call. You pay by writing and testing each template.
• Terse plans from a small, fast model. Short keys, no explanations, effort: "low". Measure the smaller planner's accuracy on your labels first.
• Overlap. Start the one-search retrieval with the original question while the planner writes. You need it anyway (Move 6).
• Deterministic slot filling for values with a known shape: no extraction call on the critical path.
• Cap steps in code and stream progress notes ("Looking up the 2026 offsite…") so the wait is visible.

Caching plans across users is risky. Two questions that differ only in a year or an office embed almost identically (cosine 0.953 between the 2026 and 2025 versions of the offsite question with this lesson's model), and a cached plan would carry the wrong constraint. If you cache, key on the extracted constraints, not on question similarity (Caching Strategies).

Your turn: with the same assumptions, what do three parts with two chunks each cost per 1,000 queries, and when does the first answer token arrive?

Measure it: is the decomposer earning its keep?

A decomposer is a bet that the split finds more of what the answer needs than one search does. Test that bet on labeled questions. Here are ten from this handbook, each with its gold parts: the chunks that must reach the prompt for a complete answer. The block compares one search against a zero-cost rule planner (one step per office or year named in the question), the same rule planner plus the original question, and an oracle router that splits only the questions labeled as needing a split by the two conditions from the start of the lesson. Every system gets the same 4-chunk prompt budget, merged with round_robin from Move 4.

LABELED = [  # question, gold parts (a part counts as covered if any of its chunks is in the context),
             # should split? (parts from different documents AND one search misses a part)
    ("Compare parental leave for the second parent in Berlin and Amsterdam.",
     [{"leave-berlin"}, {"leave-amsterdam"}], True),
    ("Does the second parent get more paid leave in Lisbon or in Amsterdam?",
     [{"leave-lisbon"}, {"leave-amsterdam"}], False),     # one search returns both parts
    ("How many vacation days do Berlin and Amsterdam give?",
     [{"vacation-berlin"}, {"vacation-amsterdam"}], False),
    ("What is the per diem in Berlin and in Amsterdam?",
     [{"perdiem-berlin"}, {"perdiem-amsterdam"}], False),
    ("How did the remote work allowance change between 2025 and 2026?",
     [{"remote-2025"}, {"remote-2026"}], False),
    ("Where is the 2026 engineering offsite?", [{"offsite-2026"}], False),
    ("What per diem do I get at the 2026 engineering offsite?",
     [{"offsite-2026"}, {"perdiem-lisbon"}], True),
    ("What happens to my unused vacation days if I move from Berlin to Amsterdam?",
     [{"transfer-vacation"}], False),
    ("Can adoptive parents in Berlin and Amsterdam take parental leave?", [{"leave-policy"}], False),
    ("What is the home-office equipment budget, and who approves purchases above it?",
     [{"equipment"}], False),
]

def rule_plan(q):
    """A zero-cost baseline planner: one step per office, or per year, named in the question."""
    offices = [o for o in OFFICES if o in q]
    years = sorted({int(y) for y in re.findall(r"\b20\d\d\b", q)})
    step = lambda i, office, year: {"id": f"s{i}", "query": q, "depends_on": [],
                                    "office": office, "year": year}
    if len(offices) >= 2:
        steps = [step(i, o, None) for i, o in enumerate(offices, 1)]
    elif len(years) >= 2:
        steps = [step(i, None, y) for i, y in enumerate(years, 1)]
    else:
        steps = [step(1, offices[0] if offices else None, years[0] if years else None)]
    return {"split": len(steps) > 1, "steps": steps}

def context(q, planner, keep_original, budget=4):
    plan = planner(q) if planner else None
    steps = plan["steps"] if plan else []
    if keep_original or not steps:
        steps = [{"query": q, "office": None, "year": None}] + steps
    lists = {i: search(s["query"], k=budget, office=s["office"], year=s["year"])
             for i, s in enumerate(steps)}
    return round_robin(lists, budget), len(steps)

SHOULD_SPLIT = {q: want for q, _, want in LABELED}
routed = lambda q: rule_plan(q) if SHOULD_SPLIT[q] else None     # an oracle router: an upper bound
systems = {"one search": (None, False), "rule split": (rule_plan, False),
           "rule split + original": (rule_plan, True), "split only when labeled": (routed, False)}
for name, (planner, keep) in systems.items():
    shares, incomplete, searches = [], [], 0
    for n, (q, parts, _) in enumerate(LABELED, start=1):
        ctx, used = context(q, planner, keep)
        hit = [bool(part & set(ctx)) for part in parts]
        shares.append(sum(hit) / len(parts))
        searches += used
        if not all(hit):
            incomplete.append(n)
    print(f"{name:23} part coverage {sum(shares) / len(LABELED):.2f}  "
          f"all parts {len(LABELED) - len(incomplete)}/{len(LABELED)}  "
          f"searches {searches / len(LABELED):.1f}  incomplete {incomplete}")

splits = [(rule_plan(q)["split"], should) for q, _, should in LABELED]
print("split when it should not:", sum(s and not want for s, want in splits),
      "| missed a needed split:", sum(want and not s for s, want in splits))

Predict first: does the rule planner beat one search on this set?

one search              part coverage 0.90  all parts 8/10  searches 1.0  incomplete [1, 7]
rule split              part coverage 0.75  all parts 7/10  searches 1.7  incomplete [7, 8, 9]
rule split + original   part coverage 0.85  all parts 8/10  searches 2.7  incomplete [7, 8]
split only when labeled part coverage 0.95  all parts 9/10  searches 1.1  incomplete [7]
split when it should not: 6 | missed a needed split: 1

Ten questions is a demonstration, not an evaluation: one question moves part coverage by 0.05 to 0.10. A real set needs 100–200 labeled questions, and confidence intervals, which Evaluation & Quality Metrics covers.

What to label per question: its shape, whether it should be split (both conditions), its gold parts with relevant chunk ids, the dependency edges between parts, and its constraints (offices, dates, roles). Then measure the planner and the executor separately:

Feeding the executor gold plans isolates retrieval from planning. If coverage is high with gold plans and low with the planner's plans, fix the planner. If it is low with both, fix retrieval or the merge. Compare systems on the same questions, with a paired test (Retrieval Metrics). Part coverage is the per-part version of the gold-in-prompt rate from Context Augmentation: does every part's gold chunk actually reach the prompt? Like that rate, it is computed per question and then averaged, so a question with two parts weighs no more than a question with one. The all-parts rate, the share of questions whose every part reaches the prompt, is the prompt-stage version of the all-found rate in Retrieval Metrics.

Final round: no label on the problem

Real traces do not say which move they need. Read the symptom, then pick.

Challenge 1: slower answers, fewer right ones

A support assistant sends every question through an LLM planner. On the labeled set, comparison questions (8% of traffic) go from 70% to 85% correct. Single-fact questions (80% of traffic) drop from 91% to 88%, and p50 latency goes from 1.9 s to 3.6 s. Keep it, drop it, or change it?

Challenge 2: the confident wrong city

A travel assistant answers "What is the hotel cap at this year's sales kickoff?" with last year's city and last year's cap. The trace: s1 "Where is the sales kickoff held?" returned last year's announcement first (0.81, against 0.78 for this year's). s2 searched for the hotel cap in that city.

Challenge 3: nine steps, three duplicates

"Compare parental leave, vacation days and the remote allowance across our three offices." The planner returns nine steps, one per topic per office. p95 is 8 s, and the three remote-allowance steps return identical chunks.

Cheat sheet: symptom → move → cost

Before moving on, take one question your pipeline gets wrong and write down its shape, whether it needs a split, the plan as JSON with dependencies and filters, the longest path in LLM calls, the constraints your check must find in the plan, and the labeled questions that would prove the split beats one search for that shape.

How would you measure it on your own data? Collect 100–200 real questions. Label each one's shape, gold parts with relevant chunk ids, dependencies and constraints. Run one search and the decomposer side by side and compare, per shape: part coverage in the final prompt, answer correctness, cost per 1,000 queries and p95 latency. Ship decomposition only for the shapes where it wins by more than it costs.

Next: Agentic RAG Systems for questions whose steps you cannot plan up front, Graph Rag for when the bridge relations themselves should be stored as data, Vectorless RAG for answering exhaustive questions from structured sources, and Evaluation & Quality Metrics to prove any of it helped.