Chat Application (WhatsApp)

"Anything new for me?" times 100 million

Suppose your first design for a WhatsApp-style messenger is the one you would build over a weekend. The app POSTs each message to an API, a Postgres table stores it, and every open app asks GET /messages?after=… every 5 seconds.

Now give it real load. These are exercise assumptions, not WhatsApp's figures: 500 million daily active users, each sending 40 messages a day, an evening peak of 3× the daily average, and 100 million devices with the app open at that peak.

Predict first: which breaks first, the inserts or the polls?

Check your answer

The polls, by a factor of about 29. For every message sent at peak, the database answers roughly 29 "anything new?" queries, and nearly all of them return nothing. The design is also slow by construction: a message waits 2.5 s on average for the recipient's next poll. Shorten the interval to 2 s and the polls climb to 50 million a second. You cannot tune your way out of this. The server has to push.

The inserts are the second fire. 694,000 durable writes a second is far beyond one box, so the message data must be split across many machines. But the empty polls hit you first.

The fix is not one clever component. Each part of a real chat system exists because a specific pressure in the requirements forced it. Name the pressure and the move follows:

This lesson applies ideas taught elsewhere in the roadmap. Transport choices (WebSockets, SSE, long polling) are in Networking Basics. Delivery semantics and idempotent consumers in general are in Messaging & Queues. Sharding and hot keys are in Databases & Storage. Here you put them together for chat.

The questions that change the design

"Design WhatsApp" hides a fork that decides whether your storage stays bounded or grows by petabytes every year. Ask before you draw a single box (the general technique is in Requirements Gathering):

WhatsApp is the relay kind. Its privacy policy says delivered messages are deleted from its servers and undelivered ones are deleted after 30 days. Team chat products such as Slack or Discord are the archive kind: history lives on the server and new devices load it from there. This lesson designs the relay model and flags every place where the archive model differs.

Scope the rest out loud too. Voice and video calls, stories and payments are separate systems; say you will leave them out unless the interviewer wants them.

💡 Say it in the interview: "Before I draw anything: does the server keep chat history, or only deliver it? That decides whether my storage is a bounded window of undelivered messages or petabytes more every year."

The numbers we'll design for

Requirements, all stated as assumptions for this exercise:

• Features: 1:1 chats and groups of up to 1,000 members (WhatsApp's own cap is 1,024 at the time of writing). Sent, delivered and read ticks. Online and last seen. A typing indicator. A phone plus a few linked devices per user. Photos and files.
• Guarantees: end-to-end encryption. Once the sender sees one tick, the message is never lost. No device shows a message twice. Every member sees one order per conversation. Undelivered copies are deleted after 30 days.
• Speed: p99 under 500 ms from send to delivery when both devices are online in the same region.
• Traffic: 500M daily users, 40 sends each per day, 3× peak, 100M connected devices at peak. Users have 1.5 active devices on average (the phone, plus sometimes a laptop or tablet). 70% of messages are 1:1; 30% go to groups that have 25 other members on average. A stored message envelope is about 300 bytes (text; media goes to a blob store).

Look at the device rows. Deliveries, not sends, are the load: every send becomes about 13 device deliveries, and groups are 30% of sends but about 89% of deliveries. Keep that in mind for Moves 5 and 7.

Move 1: Keep a line open to every phone

Pressure: the server must reach a phone the moment a message exists, and polling is both slow and about 29× too expensive.

Each device opens one long-lived, encrypted, two-way connection and keeps it while the app is active. The server pushes messages down it; the device sends messages, acks and receipts up it. In a browser this is usually a WebSocket. Native apps often run their own protocol over TCP, encrypted with TLS or with an alternative such as the Noise Protocol Framework: WhatsApp's encryption whitepaper describes client–server traffic running inside Noise Pipes "for long running interactive connections". Either way, the server now holds a hundred million open connections, and that shapes everything else.

The gateway tier

A connection is state: a socket, a TLS session, buffers, and the fact "device X is here". So cut the system along that line.

• Gateways hold connections and do almost nothing else. They authenticate the device when it connects, pass its requests to the chat service, push deliveries down its socket, and answer its heartbeats. Keep them thin, so you can restart and scale them without touching business logic.
• Everything behind them is stateless (the chat service) or a store. Any chat-service worker can handle any request.

An L4 load balancer spreads new connections across gateways; least-connections is a sensible policy. It balances the connect, not the traffic afterwards: a socket opened at 9 a.m. stays on its gateway until it drops. Two consequences follow. A freshly added gateway fills up only as new connections arrive. And deploying a gateway means draining it: ask its clients to reconnect, spread over several minutes, instead of killing it.

How many gateways?

Memory is rarely the limit. At an assumed 20 KB per connection for TLS state and buffers, 250,000 connections use 5 GB. WhatsApp's engineering blog described pushing a single Erlang server past 2 million TCP connections back in 2012.

The number you plan around is the blast radius: every connection on a gateway that dies reconnects at once, somewhere else. So pick a comfortable cap, say 250,000 per gateway:

Why divide by 2/3 instead of adding a third? After a zone fails, the two surviving zones must carry all 100M connections by themselves. Adding a third would give 534 gateways, and the two surviving zones would hold only 356 of them, well short of the 400 needed.

💡 Say it in the interview: "I'll cap gateways at about 250k connections, not the most a box can hold, because that cap is my blast radius when one dies."

Heartbeats: is anyone still there?

A phone that drives into a tunnel does not send a goodbye. An idle TCP connection can look healthy for hours after the other end is gone. So one side sends a tiny heartbeat every so often. A native app sends its own. A browser cannot send WebSocket ping frames from JavaScript, so it sends a small app-level message instead, or the server sends ping frames, which browsers answer automatically. Heartbeats do two jobs: they detect dead connections, and they keep NAT and firewall mappings along the path from expiring. The interval is a trade-off between battery life and how fast you notice a dead phone.

Answer heartbeats inside the gateway, in memory. 100M devices beating every 30 s is 3.3 million heartbeats a second across the fleet, but only about 5,600 a second per gateway at a normal peak (600 gateways of about 167,000 connections each), and 8,300 a second for a gateway at the 250,000 cap. Do not turn each one into a write to a central store: that is a 3.3M writes/s workload that almost never says anything new. Presence (Move 6) needs only the moments when someone comes or goes.

When a gateway dies: the reconnect stampede

A gateway crashes. Up to 250,000 phones notice (a socket error, or missing heartbeat replies) and reconnect. If every app waits exactly one second and retries, the rest of the fleet gets 250,000 TLS handshakes, logins, registry writes and mailbox syncs in the same second. That spike can knock over a second gateway, whose phones then pile onto a third.

The fix is jitter: each client waits a random time, and backs off exponentially if the retry fails too.

import random
from collections import Counter

random.seed(7)
clients = 250_000                          # a gateway at its cap

def busiest_second(delays):
    return max(Counter(int(d) for d in delays).values())

same_delay = [1.0] * clients                           # everyone retries after 1 s
jittered = [random.uniform(0, 30) for _ in range(clients)]  # spread over 30 s
print(busiest_second(same_delay), busiest_second(jittered))

It prints 250000 8453. The busiest second drops from 250,000 connects to about 8,500, close to the even spread of 250,000 ÷ 30 ≈ 8,333 a second. The cost is that some users take up to 30 s longer to reconnect. Their phone still shows every message it already has and queues outgoing ones locally, so that is a price worth paying. Gateways can also protect themselves: accept new connections at a capped rate and tell the rest to retry later.

Your turn: a whole zone, a third of the fleet, fails at peak. How many phones reconnect, and over what window must they spread so the surviving gateways see at most 150,000 new connections a second?

Check your answer

100M ÷ 3 ≈ 33.3M reconnects. At 150,000 a second that takes 33.3M ÷ 150,000 ≈ 222 s, so the clients' jitter window must be about four minutes. Squeeze it into 60 s and the survivors face about 556,000 connects a second. Capacity still holds: the 400 surviving gateways at 250,000 each carry exactly 100M. That is why you planned for 600.

Move 2: Find the gateway that holds Bob

Pressure: Alice's phone is on gateway A; Bob's is on gateway 17. Gateway A cannot write to a socket it does not own. A socket cannot be stored in Redis or moved to another machine.

A registry, not a sticky session

Keep a connection registry: device → (gateway, connection id). A gateway writes the entry when a device connects and authenticates, and removes it when the connection closes. To deliver, the chat service looks up the recipient's devices and hands each delivery to the right gateway, either by a direct RPC to gateway 17 or by publishing on a channel that only gateway 17 listens to (one channel per gateway, not one per user).

Sticky sessions do not solve this. Stickiness decides where Bob's own connection lands; it cannot carry Alice's message from gateway A to gateway 17. And stickiness by IP address fails exactly when phones reconnect, because they reconnect after switching from Wi-Fi to cellular and now have a different IP.

The registry is small and hot. 100M entries at about 100 bytes each is 10 GB, which fits an in-memory store. It takes a write per connect and disconnect. Key it by user, with one field per device, so a single lookup returns all of a user's devices: one lookup per recipient, 5.7M a second at peak. Batch the lookups for group messages.

Store first, then route

Why nothing gets lost: the registry lookup and the push down the socket are a fast path. The truth is the recipient device's mailbox (Move 5). A delivery is written to that mailbox before any routing happens, and the only thing that removes it is an ack from that device. So a stale registry entry, a dead gateway or a dropped RPC can delay a message but never lose it.

That is also why "publish to a Redis Pub/Sub channel user:bob" is not a delivery design on its own. The Redis docs call Pub/Sub fire and forget: a subscriber that is disconnected misses what was published. It is a fine doorbell and a terrible only copy.

Predict: gateway 17 crashes hard, with no cleanup. For the next few seconds the registry still says Bob's phone is on gateway 17. What happens to a message sent to Bob in that window?

Check your answer

The chat service stores it in Bob's mailbox and tries gateway 17. The RPC fails or times out, so the message simply stays in the mailbox. Bob's phone notices the dead connection, reconnects to another gateway (with jitter), registers the new connection, and syncs its mailbox. He gets the message a little late, and his phone shows it once. Meanwhile, the registry learns that gateway 17 is gone (gateways heartbeat to a membership service) and purges its entries.

The registry itself can fail too. If a registry shard is lost, the gateways still hold their sockets and know which devices they hold, so each one re-registers its devices. Until they finish, deliveries to those devices fall back to push notifications and sync. Messages are delayed, not lost.

The reconnect race

Bob walks out of his house. His phone opens a 5G connection, which lands on gateway 4, before the old Wi-Fi socket on gateway 17 times out. A few seconds later gateway 17 declares the old socket dead and runs its cleanup: delete the registry entry for bob-phone. That deletes the new entry. Bob is connected, the registry says he is offline, and every message to him goes the slow way (push, then sync) until he reconnects again.

The fix is to give every connection an id and to delete an entry only if it is still yours:

registry = {}                      # device -> (gateway, connection_id)

def on_connect(device, gateway, conn_id):
    registry[device] = (gateway, conn_id)          # the newest connection wins

def on_close(device, gateway, conn_id):
    if registry.get(device) == (gateway, conn_id): # remove only *my* entry
        del registry[device]

on_connect("bob-phone", "gw-17", "c1")   # Bob on Wi-Fi
on_connect("bob-phone", "gw-04", "c2")   # he walks out: 5G socket opens first...
on_close("bob-phone", "gw-17", "c1")     # ...then the Wi-Fi socket times out
print(registry["bob-phone"])             # ('gw-04', 'c2'): still reachable

The same check works per field when entries are grouped by user. In a real registry the check and the delete must be one atomic step. In Redis, for example, that is a short Lua script, because Redis runs a script without interleaving other commands.

Move 3: One tick means "stored"

Pressure: mobile networks drop packets, acks and whole connections in the middle of a send. The sender will retry, and the server will redeliver.

Here is the architecture the rest of the lesson fills in:

 Alice's phone                                               Bob's phone
      |                                                           ^
 [L4 load balancer]                                     [L4 load balancer]
      |                                                           |
 [gateway A] --send--> [chat service] --deliver----------> [gateway 17]
                        (stateless)
                         |  |  |  |
                         |  |  |  +--> [push service] --> APNs / FCM
                         |  |  |        (only when a device has no connection)
                         |  |  +-----> [connection registry]
                         |  |           device -> (gateway, connection id)
                         |  +--------> [mailbox store]
                         |              one mailbox per device, inbox sequence
                         +-----------> [conversation shards]
                                        one owner per chat: dedup, number, store

Trace one message

Alice's phone sends "On my way" to Bob, whose phone is online. Their conversation's last message was number 1,206.

Each tick is a promise, and it goes out only after the thing it promises is true. One tick means "the server has it durably and will not lose it". Two ticks mean "one of Bob's devices has it". Blue means "Bob has seen it". Send the first tick before the commit, and a crash between those two steps shows Alice a tick for a message that no longer exists.

Retries are guaranteed; duplicates are your job

The ack in step 3 can be lost. Alice's phone cannot tell "the server never got it" from "the server stored it and the ack was lost", so it resends. Every hop in this system is at-least-once: retry until acked. The price is duplicates, and you remove them with idempotent receivers: the retry carries the same id, and the receiver recognizes it.

Where should the check happen? Where the message becomes durable, inside the same transaction that stores it. A cache check in front ("look the id up in Redis; if absent, insert; then write the id to Redis") has two holes. Two concurrent retries can both see "absent" and both insert. And a crash between the insert and the cache write lets the next retry insert again. A unique key on (sender_device, client_msg_id) in the store cannot be raced. A cache in front is fine as a fast path, as long as the unique key stays the rule.

Here is the conversation shard's accept step, runnable with Python's built-in SQLite:

import sqlite3

db = sqlite3.connect(":memory:", isolation_level=None)   # we issue BEGIN/COMMIT ourselves
db.executescript("""
CREATE TABLE conversation (conv_id TEXT PRIMARY KEY, last_seq INTEGER NOT NULL);
CREATE TABLE message (
    conv_id       TEXT,
    seq           INTEGER,
    sender_device TEXT,
    client_msg_id TEXT,
    body          BLOB,
    PRIMARY KEY (conv_id, seq),
    UNIQUE (sender_device, client_msg_id)    -- the dedup rule lives in the durable store
);
""")

def accept(conv_id, sender_device, client_msg_id, body):
    """Store the message once and return its sequence number.
    The server sends the first tick only after this returns."""
    db.execute("BEGIN IMMEDIATE")            # this conversation's single writer
    try:
        row = db.execute(
            "SELECT seq FROM message WHERE sender_device = ? AND client_msg_id = ?",
            (sender_device, client_msg_id)).fetchone()
        if row:                              # a retry: give the same answer again
            db.execute("COMMIT")
            return row[0]
        db.execute("INSERT OR IGNORE INTO conversation VALUES (?, 0)", (conv_id,))
        db.execute("UPDATE conversation SET last_seq = last_seq + 1 WHERE conv_id = ?",
                   (conv_id,))
        seq = db.execute("SELECT last_seq FROM conversation WHERE conv_id = ?",
                         (conv_id,)).fetchone()[0]
        db.execute("INSERT INTO message VALUES (?, ?, ?, ?, ?)",
                   (conv_id, seq, sender_device, client_msg_id, body))
        db.execute("COMMIT")                 # counter and message commit together
        return seq
    except Exception:
        db.execute("ROLLBACK")
        raise

print(accept("alice:bob", "alice-phone", "c-41", b"<ciphertext>"))  # 1
print(accept("alice:bob", "alice-phone", "c-41", b"<ciphertext>"))  # 1, the retry
print(accept("alice:bob", "bob-phone",   "c-07", b"<ciphertext>"))  # 2
print(db.execute("SELECT COUNT(*) FROM message").fetchone()[0])     # 2 rows

It prints 1, 1, 2 and 2. The retry got the same number back, so Alice sees one message with one tick and Bob gets one copy. Two details matter. The client id is generated on the phone before the first send and saved with the outgoing message, so a retry after an app restart reuses it. And the counter and the message commit together, which Move 4 relies on.

In a wide-column store such as Cassandra, the equivalent conditional insert is a lightweight transaction (IF NOT EXISTS). It checks only whether a row with that primary key exists, so the dedup check needs a row keyed by the client id, and it costs a Paxos round per write, which adds up at hundreds of thousands of sends a second. Many designs therefore give each conversation shard one writer process that does the check and the numbering, then writes to replicated storage.

Fan-out (step 4) runs after the commit, off the path of the first tick. A common way to drive it is a log such as Kafka, partitioned by conversation id so each conversation's messages stay in order. Get committed messages into the log through change data capture or an outbox table, so a crash between the commit and the publish cannot drop one (the outbox pattern is in Advanced Topics & Final Prep). Fan-out workers commit their log offsets only after the mailbox appends succeed. A worker that crashes re-reads from its last committed offset and appends again, which is harmless because appends are keyed by (device, conversation, number).

The device side: redeliver until acked

The server-to-device hop is at-least-once as well. If the ack does not arrive, the mailbox entry stays and goes out again on the next sync. So the phone deduplicates too: it ignores any message whose (conversation, number) it already has, and still acks it. The phone stores before it acks, so a crash between the two produces a redelivery that the phone quietly discards.

Failure drill

Say "exactly once" carefully. No protocol delivers a message exactly once over a network that can lose acks. You get at-least-once delivery plus idempotent processing, so each message takes effect once.

💡 Say it in the interview: "Every hop is at-least-once with retries, and every receiver deduplicates on an id the sender generated. The first tick means durably stored, so it goes out after the commit, never before."

Your turn: a teammate wants lower latency. The gateway will ack Alice as soon as it receives her message and forward it to the chat service in the background. What breaks, and what do you offer instead?

Check your answer

If the gateway crashes after acking and before forwarding, Alice sees a tick for a message the server never stored. Her phone has already cleared its outbox, so nothing will ever retry it. The message is gone, which breaks "never lost after one tick".

Offer this instead: keep the tick after the commit, and make the commit fast (a few milliseconds inside one region). What the user feels comes from the interface, not the ack. Alice's message appears in her chat instantly with a 🕓, and the tick replaces the clock a moment later.

Move 4: Everyone sees the same order

Pressure: two people type at the same moment, three people in a group answer each other, and messages travel through different gateways with different delays.

Why not timestamps?

Phone clocks are wrong. Users set them by hand and they drift, so sorting by the sender's clock can put a reply above the question it answers. Server clocks are better, but different machines still disagree by a few milliseconds, and two messages in the same millisecond need a tie-break anyway. Time zones are not the problem, since you store UTC; clock skew is. And a timestamp can never tell a client that something is missing.

One sequencer per conversation

Give every conversation a single owner, the shard chosen by hashing its conversation id, and let the owner number messages 1, 2, 3… as it stores them. That is exactly what accept() above does: the counter and the message commit in one transaction.

Why it works:

• One writer, so no duplicates and no going backwards. Only the owner hands out numbers, and the counter is stored with the messages. A replica that takes over after a failure has both the messages and the counter, so it continues from the right number. Compare a counter kept in a separate cache: Redis INCR is atomic on one node, but Redis replicates asynchronously, so after a failover the new primary can hand out numbers that were already used.
• Dense numbers make gaps visible. A client that has 1–41 and receives 43 knows 42 is missing.
• Replies land below questions. Bob can only reply to Alice's question after his phone received it, and it was numbered when it was stored, before delivery. His reply reaches the owner later and gets a higher number. Everyone renders by number, so nobody sees the answer above the question. Without one numbering, Carol could receive Bob's reply first (it took a faster path) and show it on top.

The cost: every write of a conversation goes through one owner. A 1:1 chat or a 1,000-member group is far below what one shard can handle. The owner's load is the conversation's write rate, so even a 200,000-member channel posting 5 messages a second is easy to number; what hurts there is the fan-out and the reads (Move 7). Order is per conversation only. No one needs a global order across all chats, and a single global counter would have to absorb all 694,000 sends a second.

The client holds the line

On the phone, render by number, skip what you already have, and hold anything that arrives early:

class ConversationView:
    """Client side: show each message once, in sequence order."""
    def __init__(self):
        self.shown_upto = 0      # every seq <= shown_upto is on screen
        self.early = {}          # seq -> text, arrived before the gap filled

    def receive(self, seq, text):
        if seq <= self.shown_upto or seq in self.early:
            return []            # a redelivery: already have it
        self.early[seq] = text
        shown = []
        while self.shown_upto + 1 in self.early:
            self.shown_upto += 1
            shown.append(self.early.pop(self.shown_upto))
        return shown

    def missing(self):
        """Sequence numbers to fetch if a gap does not fill quickly."""
        if not self.early:
            return []
        return [s for s in range(self.shown_upto + 1, max(self.early))
                if s not in self.early]

view = ConversationView()
for seq in [1, 2, 4, 3, 3, 5]:
    print(seq, view.receive(seq, f"m{seq}"), view.shown_upto, view.missing())

If a gap does not fill within a moment, the client asks the server for the missing numbers.

Your own messages need one extra rule. Alice's phone shows her message at the bottom right away, with a 🕓. When the ack comes back with a number, the message takes its place by that number. Usually that is where it already is. If Bob's message won the race and got 1,207 while hers got 1,208, his ends up just above hers, even if hers was on screen first. It is a small shuffle, and everyone ends up with the same order.

💡 Say it in the interview: "Order is per conversation. The conversation's single owner assigns numbers when it stores each message. Clients render by number and treat a gap as 'fetch the missing one', not 'reorder'."

The follow-up you will hear: "What if the owner dies?" A replica with the same committed messages and counter takes over. Sends that were never acked get retried by their phones and deduplicated by the unique key.

Move 5: The phone is off

Pressure: Bob's phone is in airplane mode for a 12-hour flight. Messages keep arriving for him.

A mailbox per device

Every device has a mailbox: the entries it has not acked yet, numbered by an inbox sequence that belongs to that device alone. The fan-out step writes an entry right after the message commits, and the entry is deleted when the device acks it. An entry points at the ciphertext the server stored: that device's own copy for a 1:1 message (Move 8), or one shared copy for a group message (Move 7).

What the shard keeps depends on the model. In the relay model it deletes a message body once every recipient device has acked it, or after 30 days. Two small things outlive the body: the conversation's counter, and the dedup record (the ids and the number; in accept() that is the message row with its body cleared). The dedup record must live as long as a phone may still retry that send. Here that is 30 days, the same horizon as undelivered messages; a phone that comes back later stops retrying and marks the message "not sent". In the archive model it keeps the message for good, and the mailbox is just an index of what each device still needs.

A mailbox is a queue: appended at the tail, read in order, deleted from the head. ⚠️ Queue-shaped tables are a known trap in log-structured stores such as Cassandra. Every delete leaves a tombstone that later reads must step over until compaction clears it. Choose a store that handles deletes well, or group entries into time buckets and drop whole buckets.

Push is a doorbell, not a mailbox

If a device has no connection, the push service sends a notification through APNs (iOS) or FCM (Android). Treat that notification as a hint that wakes the app so it connects and syncs, never as the delivery itself:

• Apple calls APNs "a best-effort service" that may reorder notifications, and while a device is offline it "stores only one notification per bundle ID". The payload limit is 4 KB.
• FCM keeps pending messages for up to 28 days, but a new message with the same collapse key replaces the old one.

So a push that says "new messages" is fine, and a push as the only copy is not. With end-to-end encryption the server cannot put readable text in the push at all (Move 8). And send one push per burst per conversation, not one per message.

Sync: two counters, two jobs

When Bob lands and his phone connects, it sends one number: the highest inbox sequence it has stored. The server streams the entries after it in pages, and the phone acks each page after storing it:

Bob's phone                        chat service                    mailbox store
    | CONNECT + login                  |                                  |
    | SYNC after inbox_seq 5120 ------>|-- read entries after 5120, 100 ->|
    |<-- page: 5121 .. 5220 -----------|<---------------------------------|
    | store locally, then ACK 5220 --->|-- delete entries up to 5220 ---->|
    |<-- page: 5221 .. 5320 -----------|                                  |
    |          ... until the mailbox is empty, then live pushes resume ...|

A page at a time with an ack after each gives you backpressure: a slow phone on a weak network pulls at its own pace, and a crash in the middle repeats only the page that was not acked.

Why a separate inbox sequence, when messages already have numbers? Because the conversation number from Move 4 is per conversation. Every chat has its own 1, 2, 3…, so a single "last number I saw" across all chats means nothing. Suppose Bob's family group is at 480 and his chat with his landlord is at 35. "Everything after 480" skips the landlord's messages 36 and 37 forever. So each counter gets one job:

• The conversation sequence decides the order in which a chat is displayed.
• The inbox sequence decides what this device still needs.

(Another design with the same effect: the phone sends a map of conversation → last number for every chat. It works, but the request grows with the number of chats, and the server still needs an index of which chats changed for Bob, which is what a mailbox is.)

Your turn: Bob's laptop has been off for 10 days, and 6,000 entries wait in its mailbox. Pages hold 100 entries, and each page costs one 250 ms round trip. How long does the sync take, and what would you change so Bob can read his newest chat quickly?

Check your answer

6,000 ÷ 100 = 60 round trips × 0.25 s = 15 s. Two fixes. First, pipeline: allow about four pages in flight before waiting for acks, which brings it to about 4 s. Second, send a summary first (which chats have new messages, and how many) so the chat list is correct immediately, then stream the bodies. You give up some simplicity: the phone must cope with pages arriving while it is still storing earlier ones.

How much does the server keep?

This is where the relay-or-archive question pays off. Use Little's law from Key Concepts & Terminology: items in the system = arrival rate × average time each one stays.

Relay model. Every device delivery passes through a mailbox. Assume 80% of deliveries are acked within about 1 s (the device is online) and 20% wait for the device to come back, 2 hours on average. A 1:1 entry carries that device's own ciphertext, about 300 bytes. A group entry is a 40-byte pointer to one shared ciphertext (Sender Keys, Move 7), which is kept until the slowest member device acks it: assume 6 hours on average. Dedup records are about 60 bytes and are kept 30 days.

Look where the bytes are: about 98% of the relay's storage is dedup records. Each one is tiny, but it stays 30 days while a mailbox entry stays minutes. That is Little's law again: what you keep longest dominates. A cheaper design borrows the idea behind Kafka's idempotent producer. Each phone numbers its own sends 1, 2, 3… and submits them strictly in order from its outbox, so the server keeps only the highest number it has accepted from each device. For 750M devices at about 40 bytes that is 30 GB, and the relay total drops to about 2.3 TB. The price: a phone may never skip ahead in its outbox, and a very late retry gets back only "already stored", not its conversation number.

Archive model. Store every message once and keep it.

One requirement moved storage from a bounded 110 TB (about 2.3 TB with per-device send counters) to 6.6 PB more every year. That is why you ask it first.

Some details for each model:

• Relay: delete each copy when its device acks, and delete what is still undelivered after 30 days. The sender's message then stays at one tick, which is the honest thing to show. Metadata such as who messaged whom and when may have its own retention rules (abuse investigations, legal requirements). That is a separate log with its own policy, not the message store.
• Archive: partition messages by (conversation, time bucket) and sort by sequence number within the partition. Load history with keyset pagination: "number below 1,207, newest 50", never an offset (Cassandra's query language has no OFFSET at all). Discord has described the same partitioning in Cassandra: partition key (channel_id, bucket), with buckets of about 10 days so partitions stay under roughly 100 MB, sorted by a time-sortable Snowflake message id rather than a per-channel counter. Partitioning in depth is in Databases & Storage.

User profiles, contacts and group membership are small, read-mostly tables. An ordinary relational database, sharded by user, is fine for them, and they are rarely where a chat interview is decided.

Move 6: Ticks, green dots and "typing…"

Receipts are messages too

"Delivered" and "read" travel from Bob back to Alice through the same machinery as messages: into Alice's mailbox, at-least-once, deduplicated. If Alice is offline, her ticks change when she next syncs.

Sent naively, receipts would outnumber messages: up to two per recipient (delivered and read are tracked per user, not per device), or 11.4 million a second at peak. Two tricks keep them cheap.

• A read receipt is a high-water mark. "Bob has read this chat up to 1,207" replaces fifty separate receipts when he opens a chat with 50 unread messages. Store one read cursor per (reader, conversation) and only ever move it forward, cursor = max(cursor, new), so a late or duplicated receipt cannot un-read anything.
• Delivered receipts are batched. A sync that brings 300 messages from 12 chats sends one delivered receipt per chat (the highest number), not 300.

In a group, keep a read cursor per member. Message n is "read by everyone" when the smallest cursor is at least n. The sender's phone shows that aggregate and fetches per-member details only when someone opens the message info. With several devices, count a message as delivered when the first of Bob's devices acks it, and keep one read cursor per user, so reading on the laptop clears the phone's unread badge too.

Presence is soft state

"Online" means "has an open connection right now", and the gateway knows that first: it saw the connect, it sees the close, it notices the heartbeats stop. So treat presence as soft state owned by the gateways:

• Connect: publish an "online" change.
• Clean close: publish "offline, last seen now", and write last seen to durable storage.
• Silence: after a timeout T with nothing received, treat the connection as closed.

Now the timeout arithmetic. With a heartbeat every h seconds, one lost heartbeat means the next one arrives 2h after the last good one. So T must be longer than 2h plus network jitter, or a single lost packet flips the user offline for everyone watching. Times below are measured from the last heartbeat that arrived:

The cost of a long T is that a phone that really died looks online for up to T. Also delay announcing "offline" by a few seconds, so a phone switching networks does not flash offline and online again for all its contacts.

Have the gateway write last seen when the connection ends, whether by close or timeout. Put it in a key-value store sharded by user, not in the relational users table: if each user closes the app 15 times a day, "offline" changes alone reach 500M × 15 ÷ 86,400 × 3 ≈ 260,000 writes a second at peak. Do not hang it on a cache key expiring. Redis, for instance, raises expiry events only when it actually deletes the key, which can be significantly later than the moment the TTL runs out; those events are off by default and fire-and-forget.

Presence is also where you choose availability. If the presence service is slow or cut off, show the last known state, or nothing, and never block the chat. The send path makes the opposite trade. The first tick waits for a replicated write, and a conversation owner that cannot reach its in-region replicas refuses the write, so the phone keeps the message in its outbox with a 🕓. That refusal is choosing consistency over availability, inside one region; across regions the Boss level below makes a different trade. One app, different answers per feature. (The precise CAP statement is in Key Concepts & Terminology.)

The presence fan-out trap

Predict first: at peak, which is bigger, message deliveries or presence notifications, if every online/offline change is pushed to all of the user's contacts who are online? Assume each user opens the app 15 times a day (30 changes), has 150 contacts, and 20% of them are online at peak.

Check your answer

Presence changes: 500M × 30 ÷ 86,400 ≈ 174,000 a second on average, ≈ 521,000 at peak. Each goes to 30 online contacts: ≈ 15.6 million pushes a second, 2.7 times the 5.7 million recipient deliveries of messages. Both numbers grow by the same devices-per-user factor, so the comparison holds: the green dot would cost more than the messages.

The move is subscribe on view. A client subscribes to the presence of the people on its screen right now (the open chat's header, perhaps a few recent chats) and unsubscribes when they leave the screen. The presence service pushes a change only to current subscribers, so most changes notify nobody. Many apps also let users hide their last seen or online status from some or all contacts; the server must apply that filter per viewer before publishing anything.

Typing: fire and forget

"Alice is typing…" is useful only while it is true. Send it only to devices that have that chat open. Never store it, never retry it, and let the client drop it after a few seconds without a refresh. This is the one message type where at-most-once is the right choice: a lost indicator costs nothing, and one replayed an hour later would be a bug.

Move 7: One message, a thousand phones

Pressure: a group message is one send and N − 1 deliveries. In our numbers, groups are 30% of sends and about 89% of device deliveries.

A pointer per member, or one log for all

With groups capped at 1,000, fan-out on write is the right default. A full 1,000-member group posting a message every second makes 999 member deliveries a second (about 1,500 device appends), which is easy, and online members get each message pushed immediately. For huge channels that post often, switch: store each message once, push it to online members through their gateways, and let offline members read the log from their cursor when they sync. Group the pushes by gateway, one RPC per gateway carrying the list of its members, not one per member. The two models live side by side: a member's mailbox can get a single "channel X has news up to 5,120" entry instead of every message. News feeds face the same write-or-read fan-out choice, with a celebrity twist; see Twitter/Instagram News Feed.

⚠️ The hot conversation. A huge channel's messages all live in one partition, and every member's reads and every fan-out worker hit it. Discord has described a single busy channel overloading its database node. Part of their fix sat in front of the database: services that coalesce identical requests and route by consistent hashing. For our design, the group cap keeps every conversation small enough for one owner. That cap is a design decision, not an accident.

Who was in the group when message 900 was sent?

Membership changes race with messages. If Dan is added while message 900 is being fanned out, does he get it? Make membership changes events in the same sequence: "899: Dan joined". Fan-out for message n uses the membership as of n, so the server and every client agree on who received what. Leaving works the same way. After "950: Erin left", message 951 is not fanned out to her, and with encryption the keys change as well (next section).

Encrypted groups: Sender Keys

With pairwise encryption, Alice's phone would encrypt each group message once per member device and upload hundreds of ciphertexts. WhatsApp's whitepaper describes a cheaper scheme called Sender Keys. Each member distributes a sender key to the group once, over the pairwise sessions. After that, "the sender transmits the single ciphertext message to the server, which does server-side fan-out to all group participants". So the server still fans out, but it fans out one ciphertext. The cost moves to membership changes: "whenever a group member leaves, all group participants clear their Sender Key and start over."

Your turn: product wants announcement channels of 50,000 members where only admins post, about 20 posts a day, and everyone must be notified. Which fan-out model, and what is the real pressure?

Check your answer

Write volume is not the problem: 20 × 49,999 ≈ 1M member deliveries a day (about 1.5M device appends), under 20 a second on average. Either model works, and fan-out on write keeps the reading side uniform. The real pressure is the burst: each post creates 50,000 deliveries at once. So pace it. Push to online members in per-gateway batches, and spread notifications to offline devices over a minute or so. The lesson: choose by messages per second × members, not by member count alone.

Move 8: What end-to-end encryption changes for the server

Pressure: only the devices of the sender and the recipients may read a message.

End-to-end encryption is a property: only the endpoints hold the keys. Schemes reach it in different ways; PGP-style email, for example, encrypts a per-message key with the recipient's public key. In the Signal Protocol, which WhatsApp uses, messages are not encrypted directly with Bob's public key. Public keys are used to agree on a session. Alice's phone fetches the public identity key, the signed prekey and, if any are left, one one-time prekey for each of Bob's devices from the server, and derives shared secrets from them. After that, every message is encrypted with a fresh symmetric message key. WhatsApp's whitepaper describes AES-256 in CBC mode with HMAC-SHA256, and a message key that "changes for each message transmitted". The server's part is small: store public keys, and hand out one-time prekeys, deleting each once it is used.

Every device is a recipient

WhatsApp's whitepaper describes client-fanout: the sending client "transmits a single message N number of times to N number of different devices", each copy encrypted with that device's pairwise session. The sender's own other devices get copies too, so they stay in sync. A 1:1 message to someone with a phone and two linked devices is therefore at least three ciphertexts. That is why the mailbox is per device, and why fan-out arithmetic multiplies by devices per user.

Media: encrypt first, then upload

A photo does not travel through the chat pipeline. According to the whitepaper, the sender's app generates a random key, encrypts the file, uploads the ciphertext to a blob store, and sends an ordinary encrypted message holding the key, a hash of the ciphertext and a pointer to the blob. Recipients download the blob, check the hash and decrypt. The blob store and the CDN in front of it only ever hold ciphertext. A CDN caches it at the edge like any other file, and without the key from the message it is useless. In this design the sender's app also makes the small preview thumbnail before encrypting, because the server cannot look at the photo. (Without end-to-end encryption you would protect media with access control instead, for example CDN-signed URLs or cookies. Media pipelines at scale are in YouTube / Netflix Streaming.)

A new laptop wants history

In the relay model the server has already deleted delivered messages, and any copy it still holds is encrypted for other devices. So history must come from a device that has it. The whitepaper describes the primary phone encrypting bundles of recent messages for a newly linked device, uploading them as encrypted blobs, and the new device downloading and decrypting them. An archive can be combined with end-to-end encryption too, as encrypted backups whose keys the server never holds.

💡 Say it in the interview: "With end-to-end encryption my server is a router and a mailbox for envelopes. It keeps the metadata it needs for routing, ordering and abuse limits. Search, previews, thumbnails and history for new devices move to the clients."

The follow-up you will hear: "How do you fight spam if you can't read messages?" With rate limits per account and per device (see Design URL Shortener & Rate Limiter), with metadata signals such as a new account messaging hundreds of strangers, and with user reports, where the reporter's device shares what it decrypted.

Boss level: a region goes dark

Your users are in Europe and Asia, and each region runs gateways, chat services and stores. Every device has a home region where its mailbox lives, and every conversation has a home region where its owner lives.

1. Alice (Europe) messages Bob (Asia). Where is the message numbered, where does it wait, and what does that cost?
2. The whole European region loses power. What happens to European users, and could anything be lost?

Check your answer

1. The conversation's home region numbers and stores it. If that is Europe, Alice's tick is fast. Fan-out then writes to Bob's mailbox in Asia, and an Asian gateway pushes it to him. That adds at least one long-distance one-way trip, and a full round trip or more if the fan-out waits for the remote mailbox's ack (Europe–Asia round trips are often 150–250 ms). It can still fit a 500 ms budget, but only just, so it is the first number to mention.

2. European phones reconnect, with jitter, to Asian gateways. Their mailboxes and the conversations homed in Europe were stored in Europe. Now the real trade-off appears:

• If Europe copied every write to another region synchronously, Asia takes over and nothing acked is lost. But every send paid a cross-region round trip before its first tick.
• If the copy was asynchronous, sends were fast, but messages acked in the last seconds before the outage may exist only in the dead region. For those, "never lost after one tick" was broken.

A common middle ground is synchronous copies across zones inside a region (which survives a zone failure) and asynchronous copies across regions (so losing a whole region can lose a few seconds of messages). Say that out loud as a known, rare exception.

One more trap: after an asynchronous failover, the new owner's counter may be behind numbers the old owner already handed out. Numbering 1,201 again would clash with a 1,201 that some phones already hold, and their dedup would silently drop the new message. So add an epoch that increments on every failover, number messages (epoch, n), and order by epoch first. A reissued n in a new epoch can never be mistaken for an old one.

Final round: no label on the prompt

Real prompts don't say which moves they need. Start from the requirements, find the pressures, then pick the moves.

Challenge 1: the support widget

Design chat for a customer-support widget. Website visitors (browser only, not logged in) talk 1:1 with support agents. 2 million visitors are connected at peak. Transcripts must be kept for 2 years and be searchable by the support team. No end-to-end encryption. Which of the eight moves change, and which disappear?

Check your answer

• Archive, not relay. Transcripts are retained and searched, so the server stores every message, and a search index is allowed because the server may read the text.
• Transport: a WebSocket from the browser. SSE for server-to-browser plus ordinary POSTs for sending also works if some corporate proxies break WebSockets (see Networking Basics).
• Ordering, dedup and the first tick stay exactly the same.
• Offline delivery mostly disappears. A visitor who closes the tab is gone, so the fallback is an email with the transcript, not a mailbox and push.
• Presence matters for agents, because routing a new chat to an available agent depends on it, and hardly at all for visitors.
• Groups and end-to-end encryption disappear.

Challenge 2: the game lobby

In-game party chat: parties of up to 8 players, 10 million players connected at peak, and messages vanish when the match ends. No history and no offline delivery. What is left of the design?

Check your answer

Much less than WhatsApp. Players are usually already connected to a game session, so the chat can ride that connection, and the session server that hosts the party can be its sequencer. A mailbox, push notifications and long-term storage all go away: an offline recipient simply misses the chat. Keep at-least-once delivery with dedup inside the session, since it is cheap for 8 people. If players can report abuse, keep a short moderation log, which is a retention requirement of its own. The lesson: removing a requirement removes components, and you should say so.

Challenge 3: "delete for everyone"

Add "delete for everyone" to your WhatsApp-style design.

Check your answer

A delete is a new message in the same conversation sequence: "1,250: delete 1,207". It flows through the same mailboxes, with the same guarantees and ticks. Devices that already showed 1,207 replace it with "This message was deleted". For devices that have not received it yet, do not simply remove 1,207 from their mailboxes: number 1,207 would then never arrive, and the Move 4 client would hold everything after it while asking for it forever. Replace the undelivered copy with a tombstone ("1,207: deleted"), so the number still arrives and the gap fills. Be honest about the limit: you cannot force a device to forget. Screenshots and modified clients exist, so the feature is a strong request, not a guarantee, and many apps also limit how long after sending it is allowed.

Cheat sheet: pressure → move → cost

Before moving on, explain the design aloud, as you would at a whiteboard, from a blank page. Start with the relay-or-archive question and the delivery arithmetic. Then trace one message from Alice's 🕓 to her blue ticks, naming what each tick promises. Say what happens when Bob's gateway dies mid-push, when the ack to Alice is lost, and when Bob's phone has been off for a week. Finish with one thing end-to-end encryption takes away from the server. If you can do that in about ten minutes as a rehearsal, you can fit the scope and the sketch into the first 20 minutes of a 45-minute slot and spend the deep dive (minutes 20–35) on delivery guarantees or group fan-out. The phases and the clock are in Interview Framework & Strategy.

Next: the shared toolkit behind feeds, streaming and chat is in Design Social & Streaming Systems. To rehearse this design against the clock, use Mock Interviews & Communication.