Realistic Design Examples

A realistic design is a simple design that met its pressures one at a time

The interviewer says: "Design a notification service for our app." You have never designed one. That's fine. A candidate who memorized ten architectures has seen one, which is not the same as being able to defend it when the interviewer changes a requirement.

Start with what anyone would build first. Every team that needs to tell a user something calls the push or SMS provider directly, inside its own request handler. Marketing runs a nightly script that loops over users and does the same.

Now give it real load. Assume 50 million daily active users, 4 everyday notifications per user per day (order updates, new followers, login codes), and a marketing team that wants to reach 30 million users with one campaign. That is 200 million notifications a day: about 2,300 per second on average and 6,900 per second at a 3× peak. Two things break before anything else.

• Logins fail because an SMS vendor is slow. Say the auth service sends 500 login codes per second at peak and calls the vendor inline. A normal call takes 0.3 s, so about 150 requests are waiting on the vendor at any moment (Little's law: requests in flight = arrival rate × time each one spends inside). During a vendor incident every call hangs until its 10-second timeout: 500 × 10 = 5,000 requests stuck at once. The auth service runs out of threads, and even users who never asked for an SMS code can't log in.
• The campaign can't go faster than the provider allows. Firebase Cloud Messaging (FCM), which delivers Android pushes, documents a default quota of 600,000 messages per minute per project, which is 10,000 per second. If half of the 30 million recipients use Android, those 15 million pushes need 25 minutes of the entire budget. While the loop saturates that budget, order-status pushes start failing with 429 QUOTA_EXCEEDED.

The fix is not a bigger server. Each failure comes from a pressure in the requirements: a slow dependency on the request path, one budget shared by urgent and bulk traffic. Each pressure has a standard move, and every move has a price that you say out loud. That is the whole lesson in one sentence: a realistic design is the simple design plus one move per pressure, with the price of each move named.

This lesson walks through two complete designs that no other lesson in the course covers, a notification service and search autocomplete, then gives you three prompts with no label on them. It assumes the building blocks from Core Building Blocks, Databases & Storage and Messaging & Queues. The classic prompts each have their own lesson, linked at the end.

Before any move: pin down the contract and the numbers

Two designs can use the same boxes and still get opposite verdicts, because one of them answers a question nobody asked. So before choosing a move, find out what the system must promise. In the course's four phases these questions belong to Scope. Five of them change the design more than any others:

Turning answers into numbers (daily users → requests per second → storage) is the subject of Requirements Gathering. The shape of the conversation and the clock are in Interview Framework & Strategy and Design Process Steps. This lesson is about the part in the middle: turning pressures into decisions.

Every decision gets one sentence with four parts: the move, the requirement that forces it, the price, and what would make you change it.

Say it: "I'll put a queue between the auth service and the SMS vendor, because login must stay up when the vendor is slow. The price is that the auth service only learns the code was accepted, not delivered. I'd revisit it if the product needed a delivery confirmation before showing the next screen."

That sentence also sets up the follow-up you want. An interviewer who hears "the price is…" asks how you would manage the price, and that is a question you have prepared for.

Predict first: a shop sends a receipt email after each purchase, about 2,000 purchases a day. Which moves from the table does it need?

Worked design 1: a notification service

The worked designs follow the course's four phases (scope, sketch, deep dive, wrap-up) from Interview Framework & Strategy, so the headings tell you where each part would happen in the interview.

Scope: the prompt, and the questions that change it

"Design a notification service." Ask the five questions, and a good interviewer gives you something like this.

Functional requirements

• Internal services (auth, orders, social) send a notification to one user through an API.
• Channels: iOS push through Apple's APNs, Android push through FCM, SMS, email, and an in-app inbox.
• Marketing sends campaigns to segments of up to 30 million users.
• Users choose which types they get on which channel. Marketing opt-outs are honored, and non-urgent notifications wait until quiet hours end.
• The in-app inbox shows the last 90 days.

Non-functional requirements

• Login codes reach the SMS vendor within 5 s at p99. Order updates within 30 s. A campaign finishes within an hour.
• Once the API has accepted a notification, it is never lost. A duplicate is rare and harmless: annoying, never dangerous.
• A login code more than 5 minutes old is useless: drop it rather than deliver it late.

Out of scope: the template editor, campaign analytics and A/B tests. Say so out loud; it buys you time for the parts that matter.

Notice what the requirements already tell you. "Accepted, never lost" means durable storage before you reply. "Within 5 s" next to "30 million in an hour" means two very different kinds of traffic will share providers. Your design has to keep them apart.

Scope: the numbers

Assumptions, stated so the interviewer can correct them: 50 million daily active users, 4 everyday notifications per user per day (campaigns come on top), a 3× peak, half of them treated as Android pushes, 500-byte records kept for 90 days, three replicas.

DAY = 86_400
dau, per_user, peak = 50e6, 4, 3
per_day = dau * per_user
avg = per_day / DAY
print(f"notifications/day {per_day:,.0f}, avg {avg:,.0f}/s, peak {avg * peak:,.0f}/s")

fcm_budget = 600_000 / 60          # FCM default quota: 600,000 messages per minute
android_peak = avg * peak * 0.5    # half of all notifications: a generous stand-in for Android pushes
bulk_cap = 6_000                   # the share we let campaigns use
print(f"Android peak {android_peak:,.0f}/s, spare at peak {fcm_budget - android_peak - bulk_cap:,.0f}/s")

campaign_android = 30e6 * 0.5
print(f"campaign, Android part: {campaign_android / bulk_cap / 60:.0f} min")

store = per_day * 500 * 90         # every notification's 500-byte record, kept 90 days
print(f"notification store {store / 1e12:.0f} TB, {store * 3 / 1e12:.0f} TB with 3 replicas")

notifications/day 200,000,000, avg 2,315/s, peak 6,944/s
Android peak 3,472/s, spare at peak 528/s
campaign, Android part: 42 min
notification store 9 TB, 27 TB with 3 replicas

Each number makes a decision:

⚠️ These are Android numbers. iOS pushes go to APNs. Apple doesn't publish a sending quota like FCM's, but APNs does return 429 TooManyRequests when you send too many to one device. Your sender fleet sets the practical iOS rate, so give the iOS side the same pacing.

Sketch: the API and the data

POST /v1/notifications
Idempotency-Key: login-code:7d1e0c
Content-Type: application/json

{"user_id": "u_81f2", "type": "login_code", "params": {"code": "482913"}}

HTTP/1.1 202 Accepted
{"notification_id": "n_5c09", "status": "accepted"}

202 Accepted is honest: the request was accepted for processing, not finished. Other endpoints: GET /v1/users/:id/inbox?cursor=…, PUT /v1/users/:id/preferences, POST /v1/campaigns and POST /v1/campaigns/:id/cancel.

The type, not the caller, decides urgency and channels. If callers could choose, every team would mark everything urgent.

type           lane     channels            quiet hours?   expires after
login_code     urgent   sms                 ignored        5 min
order_status   normal   push, else inbox    ignored        2 h
new_follower   normal   push + inbox        respected      1 day
promo          bulk     push, email         respected      1 day (opted-in users only)

The tables, with the key first:

notifications   notification_id = hash(producer, idempotency key)    (90-day TTL)
                user, type, params, lane, status, created_at, expires_at
deliveries      (notification_id, target)          target = device, phone or address
                status, lease_until, attempts
inbox           user_id, then created_at newest first    (90-day TTL)
                notification_id, title, body, read
preferences     user_id -> per type and channel on/off, quiet hours, time zone
devices         user_id -> push tokens, platform, last seen

Preferences and devices are read for every notification, change rarely, and are small, so cache them in the router with a short TTL. The price: an opt-out can take up to one TTL to apply. With a 60-second TTL, a user who opts out may still get one more promo in the next minute. Say that, and invalidate the cache entry when the user saves the change.

Sketch: the architecture, with one write and one read traced

  auth / orders / social                 campaign service
          │ POST /v1/notifications               │ POST /v1/campaigns
          ▼                                      ▼
 ┌─────────────────────────┐          ┌─────────────────────────┐
 │ Notification API        │          │ Campaign planner        │
 │ dedup, store, enqueue   │          │ pages the segment,      │
 └──────┬───────────┬──────┘          │ paced, kill switch      │
        │           │                 └────────────┬────────────┘
     urgent       normal                          bulk
        ▼           ▼                              ▼
 ┌─────────────────────────────────────────────────────────────┐
 │ Router workers: preferences, devices, quiet hours, template │
 └──────┬──────────────┬──────────────┬──────────────┬─────────┘
        ▼              ▼              ▼              ▼
   APNs sender    FCM sender     SMS sender    email sender    one queue per
        │              │              │              │         channel and lane
        ▼              ▼              ▼              ▼
      APNs            FCM        SMS vendor    email vendor

  notification store: notifications, deliveries, inbox   ·   dead-letter queue

Walk one login code through it:

And one read, the user opening the inbox:

The read touches one partition and never waits on the send pipeline. That's why the inbox is its own table, keyed by user and written by the router, instead of a query over deliveries.

Two things the write trace exposes:

• A retry must not create a second code. If the auth service times out at step 1 and retries with the same key, the conditional insert fails. The API returns the existing notification_id and enqueues nothing new.
• Storing and enqueueing are two writes. If the API dies between them, the notification is stored but never queued. Two cheap fixes: when a retry finds its record still ACCEPTED, re-enqueue it; and run a sweeper that re-enqueues anything stuck in ACCEPTED for well over normal queue latency (say 30 s), recording a requeued_at so each record goes back at most once per interval. Without that limit, a router backlog would make the sweeper re-enqueue every waiting record on every pass, a retry storm of its own. Both fixes are safe for correctness because the router's delivery insert is conditional too. The general version of this problem, and the transactional outbox that solves it, is in Advanced Topics & Final Prep.

Which queue? Any durable one works. With Kafka, partition by user_id and use separate retry topics for delayed retries. With SQS or RabbitMQ, visibility timeouts or acknowledgements and dead-letter queues are built in. Name one, and give the reason.

Deep dive: the login code stuck behind 30 million coupons

This design has more risky parts than one interview can cover. The course framework allows one deep-dive topic in a 45-minute slot and two in 60, so pick the one the interviewer leans toward. This section and the four after it cover all of them, so you have each one ready.

Predict first: one queue feeds every sender, and together they drain it at 10,000 messages per second. At 9:00:00 marketing enqueues a 30-million-message campaign. At 9:00:05 a user asks for a login code. When is it sent?

A FIFO queue has one rule: your wait = the work ahead of you ÷ the drain rate. You can't make the drain rate infinite, so change what counts as "ahead of you". Give each priority its own lane: its own queues all the way down to each channel's sender, and its own share of each provider's budget. An urgent message then waits only behind other urgent messages.

For FCM with the numbers above:

• transactional pushes (order updates, new followers) take what they need, at most 3,472/s at peak;
• bulk is capped at 6,000/s;
• the remaining 528/s is headroom.

The price: with a fixed cap, a campaign takes 42 minutes instead of the 25 it would take with the whole budget, even at 3 a.m. when the reserve sits idle. A smarter scheduler lets bulk borrow unused reserve and hands it back when urgent traffic rises; that's a good extension to mention. One more trap: if the scheduler gives urgent traffic strict priority over a shared budget, a long urgent surge starves bulk completely. Give bulk a small guaranteed floor.

Say it: "Urgent and normal traffic get their own lanes, so a login code never queues behind a campaign, and each push provider's budget reserves a share for transactional pushes, so order updates are never throttled by one. The price is idle reserve and a slower campaign, 42 minutes instead of 25."

The follow-up you'll hear: "Why not just add a priority field to one queue?" Some brokers do have per-message priority (RabbitMQ has priority queues; Kafka and SQS don't). But a priority field reserves neither provider budget nor workers. Bulk messages that workers have already taken still use the budget, and a slow bulk channel still ties up the same workers. Lanes isolate queues, workers and budget; a priority field only reorders one queue.

Deep dive: exactly once is not on the menu

Duplicates come from three places, and each needs its own answer:

1. The producer retries after a timeout. The idempotency key and conditional insert at the API absorb it.
2. The queue redelivers. Most queues promise at-least-once delivery: a message whose consumer didn't acknowledge it comes back. The sender must notice that the delivery already happened.
3. The sender crashes after the provider accepted the push but before recording it. Nothing you store can tell that apart from "never sent". The job will run again.

The sender below handles the second case and shrinks the third to a small window. The table stands in for any store with conditional updates: SQL UPDATE … WHERE status = …, a DynamoDB condition expression, a Cassandra lightweight transaction.

class Deliveries:
    """Stand-in for a table with conditional updates (SQL UPDATE ... WHERE,
    a DynamoDB condition expression, a Cassandra lightweight transaction)."""
    def __init__(self):
        self.rows = {}

    def claim(self, key, worker, now, lease_s):
        row = self.rows.setdefault(key, {"status": "QUEUED"})
        if row["status"] == "SENT":
            return "done"                                 # finished earlier
        if row["status"] == "SENDING" and row["lease_until"] > now:
            return "busy"                                 # another worker holds the lease
        row.update(status="SENDING", owner=worker, lease_until=now + lease_s)
        return "claimed"

    def mark_sent(self, key, worker):
        row = self.rows[key]
        if row["owner"] == worker:                        # only the lease holder finishes
            row["status"] = "SENT"


def handle(job, store, provider_log, worker, now, crash_after_send=False):
    key = (job["notification_id"], job["device"])
    state = store.claim(key, worker, now, lease_s=10)     # lease > provider timeout (3 s)
    if state == "done":
        return f"{worker} t={now:>2}: already sent -> ack"
    if state == "busy":
        return f"{worker} t={now:>2}: leased elsewhere -> don't ack, retry later"
    provider_log.append((now, job["device"], job["notification_id"]))  # sent with a collapse id
    if crash_after_send:
        return f"{worker} t={now:>2}: sent, crashed before mark_sent (no ack)"
    store.mark_sent(key, worker)
    return f"{worker} t={now:>2}: sent -> ack"


store, log = Deliveries(), []
job = {"notification_id": "n-481", "device": "ios-7f2"}
print(handle(job, store, log, "A", now=0))                   # normal send
print(handle(job, store, log, "B", now=1))                   # the queue delivers it again

job2 = {"notification_id": "n-482", "device": "ios-7f2"}
print(handle(job2, store, log, "A", now=2, crash_after_send=True))
print(handle(job2, store, log, "B", now=5))                  # A's lease still runs
print(handle(job2, store, log, "B", now=12))                 # lease expired: B sends again
print("provider calls:", log)

A t= 0: sent -> ack
B t= 1: already sent -> ack
A t= 2: sent, crashed before mark_sent (no ack)
B t= 5: leased elsewhere -> don't ack, retry later
B t=12: sent -> ack
provider calls: [(0, 'ios-7f2', 'n-481'), (2, 'ios-7f2', 'n-482'), (12, 'ios-7f2', 'n-482')]

Read the last line: n-482 reached the provider twice. That is the window you cannot close. Three details decide whether the rest works:

• "Busy" must not acknowledge the message. If worker B acked at t=5, nobody would ever retry after A's crash. Leave the message for redelivery (a visibility timeout, or a delayed retry topic).
• The lease must outlast the provider timeout. With a 3 s timeout and a 10 s lease, nobody else may take over while a slow but healthy call is still running. With a 2 s lease, any slow call whose message the queue redelivers during the call is sent twice. So the queue's visibility timeout (or redelivery delay) must outlast the call too.
• The duplicate can still look like one notification. APNs merges requests that carry the same apns-collapse-id into a single notification on the device. FCM's Android tag makes a new notification replace a displayed one with the same tag. Use the notification_id for both.

Why it works: at-least-once delivery plus a claim you can check turns "the queue delivered it twice" into "the second copy did nothing". The only duplicates left come from a crash between the provider's "accepted" and your "sent", and the collapse id makes the device merge those.

The follow-up you'll hear: "Kafka has exactly-once semantics. Why do you need any of this?" Kafka's exactly-once covers reading from Kafka, processing, and writing the results back to Kafka in one transaction. For output to an external system, the Kafka documentation says you must store the consumer's position together with that output. You can't store your offset inside APNs. Exactly-once processing inside your system is achievable; an exactly-once side effect in someone else's system is not. You get at-least-once plus dedup, and you narrow the rest.

Deep dive: providers that say no

A provider's answer tells you what to do next. The wrong reflex is to retry everything.

Time matters too. Give every job the expiry from its type. A sender drops an expired job instead of sending it, and the provider gets the same expiry: apns-expiration for APNs and ttl for FCM. This matters because FCM's default ttl is four weeks. "Your driver is outside", sent to a phone that is offline, could otherwise arrive days later.

The retry storm. Suppose FCM has a 2-minute incident while you're sending 7,000 messages per second through it. That leaves 840,000 failed jobs. When FCM recovers, new traffic still needs 7,000/s of the 10,000/s budget, so the spare 3,000/s clears the backlog in 280 seconds, if the retries are spread out. If every failed job retries on a short fixed interval, say every 5 seconds, all 840,000 come back within 5 seconds of the recovery: 168,000 requests a second against a quota of 10,000. They drain the per-minute quota at once, so for the rest of that minute even new order updates and follower pushes get 429s, with a 60-second wait. A 2-minute blip becomes a much longer outage. Three moves prevent this:

• Exponential backoff with jitter. Each retry waits roughly twice as long as the last, plus a random amount, so failed jobs don't return in lockstep.
• A circuit breaker per provider. After a burst of failures, stop calling for a few seconds and let jobs wait in the queue instead of burning quota on calls that will fail. Then let a few trial calls through, and resume when they succeed.
• Drain urgent first. Pause the bulk lane until the urgent and normal backlogs are clear.

Also, don't start campaigns on the hour. Firebase reports that its traffic more than doubles in the first minutes of each hour, with smaller spikes at the quarter and half hour. It asks senders to avoid those marks and to ramp up gradually. That is the same herd problem, caused by many senders at once.

Deep dive: the campaign planner (fan-out)

"Send this to 30 million users" must not be one request that inserts 30 million rows. The planner pages through the segment 1,000 users at a time and puts one chunk job per page on the bulk lane; the router expands each chunk into deliveries. That's 30,000 chunk jobs. At 12,000 sends per second (6,000 Android plus the same rate for iOS), the planner releases about 12 chunks a second.

Pace it; don't dump it. The planner keeps only a few minutes of work queued; at 12,000 sends a second, five minutes is 3.6 million messages. If marketing spots a typo 3 minutes in, the kill switch stops the planner, and the router and senders skip whatever is still queued for that campaign. Had all 30 million been enqueued at the start, about 28 million deliveries would be sitting in the channel queues, every one of them waiting to be read and skipped, and any worker that forgot to check the flag would send the typo.

Quiet hours need a schedule, and it must be paced too. A user whose quiet hours end at 9:00 local time gets the promo after 9:00, not at 9:00 sharp. The router writes the delivery into a schedule table bucketed by minute, at a random minute in a window such as 9:05 to 9:45 that skips :15 and :30, and a scheduler hands each minute's bucket to the paced bulk lane when it comes due. Releasing a whole time zone at 9:00:00 would be exactly the top-of-the-hour dump the providers section warned about.

"A creator with 40 million followers just went live" is the same shape of problem. The heavy version of it, where followers' feeds must change too, is the fan-out problem in Design Social & Streaming Systems.

Deep dive: "Delivered" before "Out for delivery"

An order sends "Out for delivery" and then, 40 seconds later, "Delivered". The first send times out and waits in backoff for 60 seconds, so the second one lands first. Even if you send in order, Apple's documentation says that APNs, "as a best-effort service", may reorder notifications sent to the same device token.

Partitioning by user_id keeps your queue in order for one user, but retries and the provider can still swap two messages. A collapse id alone makes it worse: whatever arrives last replaces what's on screen, so the stale "Out for delivery" would replace "Delivered". Fix it in three places:

• Drop superseded jobs on the server. Keep the latest status version sent for each order. Before each send and each retry, the sender compares the job's version with it and discards anything older. That removes the retry case above, which is the common one.
• Collapse by order. Use the order id as the collapse id (APNs) or tag (FCM), so each newer send replaces the older notification instead of stacking under it. Duplicates of one send share that id too, so they still merge.
• Let the app show the latest state for the rare case where the provider reorders two sends. On Android, send data messages and have the app build the notification from the newest status it knows. On iOS, a Notification Service Extension can rewrite a notification's content before it's displayed, for example with the order's current status. Silently dropping one instead needs a special entitlement from Apple.

The price: a version per order to track on the server, code in the app, and a small window where the provider reorders two sends faster than the app can correct them. Say that it's rare and only confusing, never harmful.

Failure modes

Wrap-up

The last phase collects what you decided, names the top risks and says what changes at 10×. For this design it sounds like this:

Say it: "Producers get a 202 after a durable, deduplicated write. Lanes keep login codes from waiting behind campaigns, campaigns are paced, and each push provider's budget keeps a reserved share for transactional pushes. Claims and collapse ids narrow duplicates; nothing eliminates them. Top risks: the FCM quota, the duplicate window after a sender crash, and opt-outs that take up to a minute to apply. At 10×, peak becomes about 69,000 a second, and the Android half alone is 3.5 times the default FCM quota. So the first thing to break is the provider budget, not our servers: we'd need a quota increase before that launch. The API, the queues and the inbox store scale by adding nodes; the notification store grows to 90 TB, 270 TB with replicas."

Your turn: change one requirement

1. "Marketing wants the 30-million campaign done in 10 minutes." What rate does that need, and what do you say?

2. "At most 3 marketing pushes per user per day." Where does the check live, and what happens under concurrency?

3. "Users in the EU must have their data stored in the EU." What changes?

Worked design 2: search autocomplete

Scope: the prompt, and the questions that change it

"Design the suggestions that appear under a search box as the user types."

One requirement makes the design forgiving: suggestions are optional. If the service is down, the search box still works. That lets you fail open: hide suggestions and never block the search.

Scope: the numbers

Assumptions: 100 million daily active users, 5 searches each per day. The client waits for a short pause in typing (a debounce), which works out to about one request per two characters; with 12 characters typed per search, that's 6 requests per search. Peak is 3× average. For the index, keep 20 million popular queries averaging 25 characters, and store suggestions only for prefixes up to 20 characters.

DAY = 86_400
requests_per_day = 100e6 * 5 * 6
avg = requests_per_day / DAY
print(f"requests/day {requests_per_day:,.0f}, avg {avg:,.0f}/s, peak {avg * 3:,.0f}/s")

queries, avg_len, max_prefix = 20e6, 25, 20
prefixes_upper_bound = queries * max_prefix       # if no two queries shared a prefix
entry = 10 + 10 * 4 + 30                          # prefix bytes + 10 ids of 4 bytes + overhead
index = prefixes_upper_bound * entry + queries * avg_len
print(f"prefix entries <= {prefixes_upper_bound:,.0f}, index <= {index / 1e9:.1f} GB")

requests/day 3,000,000,000, avg 34,722/s, peak 104,167/s
prefix entries <= 400,000,000, index <= 32.5 GB

About 100,000 requests a second at peak, against an index of at most about 32 GB. Keep that pair in mind; it decides the architecture.

Sketch: precompute, don't search

The naive version runs this on every keystroke:

SELECT query FROM query_counts
WHERE query LIKE 'ip%'
ORDER BY count DESC
LIMIT 10;

An index on query can find the rows starting with "ip" quickly. But ORDER BY count has to look at every row in that range. For a one-letter prefix that is hundreds of thousands of rows or more, sorted from scratch, 100,000 times a second.

The answer for a prefix changes only when the counts change, and the counts change once a day. So compute each answer once a day instead of on every request. For every prefix of every popular query, keep the top k:

import heapq

def build_index(counts, k=3, max_prefix=20):
    """counts: {normalized query: popularity}. Returns {prefix: [top-k queries]}."""
    heaps = {}                                     # prefix -> min-heap of (count, query)
    for query, count in counts.items():
        for end in range(1, min(len(query), max_prefix) + 1):
            h = heaps.setdefault(query[:end], [])
            item = (count, query)
            if len(h) < k:
                heapq.heappush(h, item)
            elif item > h[0]:                      # beats the weakest of the current top k
                heapq.heapreplace(h, item)
    return {p: [q for c, q in sorted(h, reverse=True)] for p, h in heaps.items()}

counts = {
    "iphone 15": 9_500, "iphone 15 pro": 8_800, "ipad": 7_700,
    "instant pot": 5_300, "iphone 15 case": 4_200, "iphone charger": 3_900,
}
index = build_index(counts)
for prefix in ["i", "ip", "iphone 15", "ins"]:
    print(f"{prefix!r:12} -> {index[prefix]}")
print(len(index), "prefixes stored")

'i'          -> ['iphone 15', 'iphone 15 pro', 'ipad']
'ip'         -> ['iphone 15', 'iphone 15 pro', 'ipad']
'iphone 15'  -> ['iphone 15', 'iphone 15 pro', 'iphone 15 case']
'ins'        -> ['instant pot']
36 prefixes stored

Serving is now one hash lookup: prefix in, list out. The six queries have 65 characters between them but need only 36 prefix entries, because they share prefixes like "iphone". That's why 32 GB is an upper bound: real queries share even more.

A trie with the top k stored at each node is the same idea in a different shape: the tree shares the prefixes in memory, and each node holds its precomputed list. Both are fine to draw in an interview. What matters is that the top-k work happens once per prefix per day at build time, not 100,000 times a second at request time.

For input longer than 20 characters, filter the 20-character prefix's list. It may come back short, but users type about 12 characters before they pick a suggestion or search, so such long input is rare.

Sketch: replicate, don't shard

Sharding is for data that doesn't fit on one machine or writes that one machine can't take. This index is read-only between builds and at most about 32 GB, so it fits in the memory of an ordinary server. Put a full copy on every node and add nodes for throughput. At an assumed 20,000 requests per second per node, six nodes carry the whole peak. Run more than that, spread across regions, for failover.

Full copies buy you three things: any node can answer any prefix, no prefix can create a hot shard, and adding capacity is just starting another copy. The price: every node pays for the whole index in memory, and every build ships a whole new snapshot to every node.

Sketch: the serving path

 browser: debounce, per-prefix cache, drop stale answers
    │ GET /suggest?q=ipho
    ▼
 CDN edge ── hit ──► cached answer (TTL about 1 h; the index changes daily)
    │ miss
    ▼
 load balancer ──► suggest nodes (each holds the whole index in memory)
                         ▲
                         │ download, validate, swap atomically
                         │
 object storage: snapshot v42 ◄── index builder ◄── daily count job ◄── search logs

Every layer takes its share of the traffic:

• The browser debounces, remembers the answers it already has for this session (backspace is free), and cancels the request for "ip" when the user types "iph".
• The CDN can cache these responses because everyone gets the same answer for a prefix. The hottest keys, the short prefixes that every search passes through, are exactly the ones an edge cache serves best: there are only 26 one-letter and 676 two-letter lowercase prefixes. A one-hour Cache-Control: max-age costs little when the index only changes daily.
• The suggest nodes handle the misses with one in-memory lookup each.

Say it: "I'll precompute the top 10 for every prefix daily and keep the whole index in memory on every node, because the read path must serve about 100,000 requests a second under 50 ms and the index is at most 32 GB. I give up freshness: suggestions can be a day old. If trending matters, I'll add a small fast path."

Predict: the user types "ca", then "cat" 120 ms later. The request for "ca" misses the CDN and takes 180 ms. The request for "cat" hits the edge and takes 20 ms. What does the user see?

Deep dive: freshness and the build pipeline

search logs (queue) ──► count distinct users per normalized query per day
                    ──► 30-day window, recent days weighted more
                    ──► build snapshot v42 ──► validate ──► publish
nodes: load v42 in the background ──► swap the pointer ──► keep v41 for rollback

• Normalize before counting. Lowercase, trim, collapse spaces. Otherwise "iPhone 15" and "iphone 15" compete with each other.
• Count distinct users, not raw searches. A bot searching one phrase 50,000 times then counts once. Also cap how many different queries one account can push per day; networks of fake accounts need abuse detection on top.
• Validate before swapping. Check that the snapshot's size is within a few percent of yesterday's, that canary prefixes ("a", "ip") return results, and that no blocked term appears. Roll the new snapshot out to one node first. A bad build is the most likely way this system fails, because every node would load the same bad file.
• Removal can't wait a day. Keep a small blocklist on the nodes that filters answers at request time, and purge the CDN when an entry is added. The price is one set lookup per suggestion.
• Trending, if asked. A streaming job counts the last hour's queries into a small second index of the top k per prefix among trending queries only. Nodes merge it with the daily list at request time. The edge must keep up too: cut the CDN TTL for short prefixes to a few minutes, or purge them when the trending index changes; otherwise the CDN serves the pre-trending answer for up to another hour. The price: two lookups and a merge, one more pipeline to run, and a lower CDN hit rate.

Failure modes

Your turn

1. "Show my own recent searches first." What breaks, and what do you change?

2. "Support 20 languages." The index grows to about 650 GB. Now what?

3. "Traffic grows 10×." What changes?

Final round: no label on the prompt

Real prompts don't say which move they want. Find the contract, find the pressure, then choose the move and name its price.

Challenge 1: 10,000 consoles, 2 million shoppers

A retailer releases 10,000 discounted game consoles at 12:00, and about 2 million shoppers are waiting. Never sell more than 10,000. One console per customer. The site must stay up.

Before peeking:

1. Which pressure dominates, and which move answers it?
2. What is the write that sells one console, and where is its hot spot?
3. What protects the site from 2 million people pressing "Buy" at 12:00:00?

UPDATE stock
SET remaining = remaining - 1
WHERE sku = :sku AND bucket = :b AND remaining > 0;

Challenge 2: a polite web crawler

Crawl 1 billion pages a month. Never overload any website. Don't fetch the same URL twice. Store the pages for an indexing team.

Before peeking:

1. What is the average fetch rate, and what does "never overload" do to how you schedule it?
2. The crawler will discover about 5 billion distinct URLs. How do you answer "seen it?" cheaply?
3. Where do the pages go?

Challenge 3: 200,000 metrics per second

2 million devices each report a small measurement every 10 seconds. Dashboards show per-minute averages for the last 30 days. When a network outage ends, every device reconnects and uploads its backlog at once.

Cheat sheet: pressure → move → price

Before moving on, pick one of the two worked designs and explain it aloud from a blank page, phase by phase, as you would in an interview: the questions you'd ask and the numbers that decide something, the API and diagram with one write and one read traced, one deep dive, and a wrap-up that names the price of every move you made. Then ask a friend to change one requirement (10× traffic, a strict ordering rule, a new region) and re-decide out loud. When you practice the classic prompts, attempt each one as a timed mock, using the clock in Interview Framework & Strategy, before reading its lesson. Then compare your decisions with the lesson's, and write down every place where you chose differently and why.

Next: the classic warm-ups in Design URL Shortener & Rate Limiter, with the full URL Shortener design; then Design Social & Streaming Systems and its three designs: the Twitter/Instagram News Feed, YouTube / Netflix Streaming and a Chat Application. When you can run these moves on a fresh prompt, time yourself in Mock Interviews & Communication.