Chat Application (WhatsApp)
Design real-time messaging with WebSockets, message persistence, delivery receipts, and group chat.
Why Designing a Chat System Like WhatsApp Is a Career-Defining Interview Challenge
Think about the last time you sent a message and watched those two grey checkmarks turn blue. That tiny moment — confirmation that your words arrived and were read — represents an engineering miracle hiding in plain sight. Billions of people take it for granted every single day. If you want free flashcards to drill the core concepts from this lesson, they're embedded throughout — look for the interactive review blocks as you read. Now, back to those checkmarks. Behind them lies one of the most nuanced, multi-layered distributed systems problems in all of software engineering — and it's exactly the kind of problem that separates senior engineers from the rest in a system design interview.
Welcome to the deep end.
Why Chat Systems Are the Gold Standard of System Design Interviews
System design interviews are fundamentally tests of engineering judgment. Not syntax. Not algorithms. Judgment — the ability to look at an ambiguous, open-ended problem and make principled decisions under uncertainty. Chat application design is the gold standard of these tests precisely because it touches every dimension of distributed systems simultaneously.
When interviewers at companies like Meta, Google, and Amazon ask you to design a chat system, they're not just checking whether you know what WebSockets are. They're watching how you think. Do you immediately jump to code, or do you pause to ask clarifying questions? Do you understand the functional requirements (what the system does) vs the non-functional requirements (how well it does it)? Can you reason about trade-offs between consistency and availability? Do you know when a SQL database is the right choice, and when it absolutely isn't?
Chat applications force all of these questions to the surface simultaneously. That's why this problem appears in nearly every senior engineering interview loop at top-tier technology companies. According to reported interview experiences on platforms like Glassdoor and Blind, variations of "design WhatsApp" or "design a messaging system" appear in roughly 30–40% of senior software engineering system design rounds at FAANG-tier companies.
🤔 Did you know? WhatsApp was acquired by Facebook in 2014 for $19 billion — at the time, the company had only 55 engineers serving hundreds of millions of users. That ratio is possible only when your system design is extraordinarily efficient.
The Scale Context: Numbers That Should Make Your Eyes Water
Before we talk about design decisions, we need to internalize the scale we're dealing with. This isn't a toy problem.
WhatsApp currently serves over 2 billion users across 180+ countries. At peak, the platform handles more than 100 billion messages per day. Let's put that in perspective:
100,000,000,000 messages / day
÷ 86,400 seconds/day
= ~1,157,407 messages per second (average)
And that's the average. New Year's Eve, major sporting events, and breaking news create spikes that dwarf the average. Your system must handle peaks that may be 5–10x the baseline load without degrading the user experience.
Now consider the additional data flows beyond raw messages:
📊 WhatsApp Scale Breakdown (Approximate):
Messages: 100B/day
Media files: ~5B/day (images, videos, documents)
Active connections: ~500M concurrent users at peak
Presence updates: ~10B/day (online/offline status changes)
Push notifications: ~50B/day
This scale context matters in an interview for one critical reason: design decisions that seem reasonable at small scale become catastrophically wrong at massive scale. A naive relational database schema that works beautifully for 10,000 users will collapse under 2 billion. A polling mechanism that feels "good enough" for a prototype creates unsustainable server load at global scale. Every architectural choice must be evaluated through the lens of: what happens when this runs at 100x the load I'm imagining right now?
🎯 Key Principle: In system design interviews, always anchor your decisions to scale. When you propose a component or pattern, explicitly state what scale it can handle and what the failure mode looks like beyond that boundary.
The Four Design Goals That Define a Chat System
Before diving into components, let's establish the north stars — the design goals that every technical decision must serve. Understanding these deeply is what allows you to make defensible choices and articulate them confidently.
Low Latency
Latency in a messaging context means the time between when Alice sends a message and when Bob sees it. Users have an almost zero-tolerance threshold for perceptible lag in real-time communication. Research from Google's human factors team suggests that latency above 150ms starts to feel "sluggish" in conversational interfaces. Above 400ms, users begin to doubt whether their message was sent at all.
This forces architectural decisions away from request-response patterns (where every message requires a full round trip) and toward persistent, bidirectional connections that keep the communication channel warm.
High Availability
Availability is about keeping the system operational even when components fail — and in distributed systems, components will fail. The goal is typically expressed as "nines": 99.9% availability means ~8.7 hours of downtime per year. 99.99% means ~52 minutes. WhatsApp targets something in the 99.99%+ range, which means individual server failures must be invisible to users.
This shapes decisions about replication, redundancy, geographic distribution, and failure detection.
Message Ordering
Message ordering is deceptively hard in distributed systems. In a single-server world, messages arrive in the order they're processed. But in a distributed system with multiple servers, network partitions, and concurrent writes, maintaining a coherent "happened-before" relationship between messages requires explicit engineering effort.
Imagine Alice sends "Are you coming tonight?" and Bob replies "Yes!" — but due to a network quirk, Bob's server processes his reply before Alice's question arrives. Without proper ordering guarantees, the conversation thread could render in reverse, making communication incoherent.
Delivery Guarantees
Delivery guarantees define the contract your system makes with users. There are three levels:
| Level | Meaning | Risk |
|---|---|---|
| At-most-once | Message delivered 0 or 1 times | Messages can be lost |
| At-least-once | Message delivered 1 or more times | Duplicates possible |
| Exactly-once | Message delivered exactly 1 time | Hardest to implement |
WhatsApp uses the at-least-once model at the transport layer and then deduplicates at the client level. This is a pragmatic choice: preventing duplicates is far easier than recovering from lost messages, and the user experience of occasionally seeing a duplicate (which the client silently discards) is better than messages that vanish.
The Core Components We Will Design Together
A production-grade chat system is not a single service — it's an ecosystem of specialized components, each optimized for a specific job. Here's the map of what we'll build across this lesson:
┌─────────────────────────────────────────────────────────────┐
│ CHAT SYSTEM OVERVIEW │
├─────────────────────────────────────────────────────────────┤
│ │
│ [Client A] ──WebSocket──► [Chat Server / Gateway] │
│ │ │
│ ┌───────────────┼───────────────┐ │
│ ▼ ▼ ▼ │
│ [Messaging [Presence [Notif- │
│ Service] System] ication │
│ │ │ Pipeline] │
│ ▼ ▼ │ │
│ [Storage [Cache ▼ │
│ Layer] Layer] [Push Service] │
│ (Messages, (APNs/FCM) │
│ Media, Users) │
│ │
│ [Client B] ◄──WebSocket── [Chat Server / Gateway] │
│ │
└─────────────────────────────────────────────────────────────┘
🔧 The Messaging Service is the beating heart — it receives messages from senders, routes them to the correct recipients, and ensures delivery guarantees are met. This involves protocol selection (WebSocket vs. long polling vs. SSE), connection management at massive scale, and message queue architecture.
📡 The Presence System tracks who is online, offline, or "last seen." This sounds simple until you realize that tracking 500 million concurrent connections in real time and broadcasting status changes to potentially thousands of contacts per user is a fan-out problem of enormous complexity.
💾 The Storage Layer must handle two radically different access patterns: hot data (recent messages that users are actively scrolling through) and cold data (message history from two years ago that needs to be retrievable but isn't accessed frequently). These patterns call for different database technologies.
🔔 The Notification Pipeline handles the case where the recipient is not connected. When Bob's phone is off, Alice's message must be queued and reliably delivered when Bob comes back online — or pushed as a notification through Apple's APNs or Google's FCM.
How to Scope This Problem in a Real Interview
Here's where most candidates stumble before they've even started designing. They hear "design WhatsApp" and immediately launch into drawing boxes and arrows. This is a critical mistake.
A senior engineer's first move is to scope the problem by separating functional requirements from non-functional requirements and explicitly confirming both with the interviewer.
Functional Requirements (What the System Does)
These are the features your system supports. In an interview, you should propose a reasonable scope and confirm it:
## Think of functional requirements as your feature checklist
## In an interview, verbalize these and get confirmation
functional_requirements = [
"One-on-one messaging between users",
"Group messaging (up to N users — clarify N with interviewer)",
"Message delivery status (sent, delivered, read)",
"Online/offline presence indicator",
"Media sharing (images, video, documents)", # Clarify: is this in scope?
"Message history / persistence",
"Push notifications for offline users",
]
## Features you might explicitly DEFER to manage scope:
out_of_scope = [
"Voice and video calls",
"End-to-end encryption key management",
"Message reactions",
"Disappearing messages",
]
The act of explicitly stating what's out of scope is as impressive to interviewers as what you include. It signals that you understand system design is about making deliberate trade-offs, not solving everything at once.
Non-Functional Requirements (How Well the System Does It)
These are the quality attributes and constraints:
## Non-functional requirements define the performance envelope
non_functional_requirements = {
"scale": "2 billion users, 100B messages/day",
"latency": "Message delivery P99 < 500ms under normal load",
"availability": "99.99% uptime (< 52 min downtime/year)",
"consistency": "Eventual consistency acceptable for message history; "
"strong ordering required within a single conversation",
"durability": "No message loss after acknowledgment",
"storage": "Messages retained for X days (clarify with interviewer)",
}
💡 Pro Tip: In a real interview, explicitly saying "I want to confirm my non-functional requirements before I start designing" signals seniority. Interviewers at Google and Meta have explicitly stated in engineering blogs that they want candidates to drive this conversation, not wait to be guided.
The Mental Model: Why This Problem Is Uniquely Hard
💡 Mental Model: Think of designing a chat system like designing a postal service — but where every letter must be delivered in under a second, the letter carrier must know in real time which recipients are home, letters must arrive in the order they were sent even if they travel through different sorting centers, and the system can never go down.
This mental model surfaces the three core tensions that make chat systems uniquely difficult:
1. Push vs. Pull: Should the server push messages to clients as they arrive, or should clients periodically ask "do I have new messages?" Pull (polling) is simple to implement but creates enormous unnecessary load at scale. Push requires persistent connections, which introduces connection management complexity.
2. Consistency vs. Availability: The CAP theorem tells us that in the presence of a network partition, we must choose between consistency and availability. For a messaging system, we generally favor availability (users can still send and receive messages, even if ordering is temporarily imperfect) over strict consistency (blocking all operations until the system is fully synchronized).
3. Fan-out on Write vs. Fan-out on Read: When Alice sends a message to a group of 500 people, do you immediately copy that message to all 500 recipient inboxes (fan-out on write) or store it once and compute each user's inbox on demand (fan-out on read)? Each approach has different latency, storage, and complexity trade-offs — and the right answer depends on read/write ratios and group size distribution.
⚠️ Common Mistake — Mistake 1: Treating a chat system as a simple CRUD application. Candidates who propose a straightforward REST API with a relational database and polling will immediately signal to the interviewer that they haven't thought about real-time delivery, connection management, or scale. Chat systems require fundamentally different architectural patterns from typical web applications.
❌ Wrong thinking: "I'll have clients poll the server every second for new messages — simple and stateless!"
✅ Correct thinking: "Polling every second for 500M concurrent users means 500M requests/second just for checking — that's unsustainable. I need persistent bidirectional connections."
🧠 Mnemonic: Remember the four chat design pillars with LOAD — Latency (keep it low), Ordering (maintain it), Availability (protect it), Delivery (guarantee it). Every design decision you make in a chat system interview should be justified by how it serves one or more of these four properties.
Connecting It All: The Journey of a Single Message
Before we dive into each component in detail in subsequent sections, let's trace the complete journey of a single message from Alice to Bob. This end-to-end view is a powerful tool for both understanding the system and structuring your interview answer.
Alice types "Hey Bob!" and hits Send
1. CLIENT → SERVER (Transport)
Alice's device sends message via WebSocket connection
to a Chat Gateway server.
2. SERVER → QUEUE (Reliability)
Gateway writes message to a distributed message queue
(e.g., Kafka). Message gets a unique ID and timestamp.
Server ACKs receipt to Alice → first checkmark ✓
3. QUEUE → DELIVERY SERVICE (Routing)
Delivery service reads from queue, looks up Bob's
current connection server (via a routing table in cache).
4. DELIVERY → BOB (Transport)
If Bob is ONLINE: message pushed directly via
Bob's WebSocket connection → second checkmark ✓✓
If Bob is OFFLINE: message stored in Bob's
message queue + push notification sent via FCM/APNs
5. BOB READS MESSAGE (Receipt)
Bob's client sends read receipt back to server.
Server updates delivery status → blue checkmarks ✓✓
6. STORAGE (Persistence)
Message written to durable storage (e.g., Cassandra)
for message history retrieval.
Total time for steps 1-4 (online path): target < 100ms P50
This end-to-end trace immediately reveals all the interesting design questions: How do you maintain millions of persistent WebSocket connections? How do you route a message to the correct server where Bob is connected? How do you handle the transition from online to offline atomically? What database can handle write-heavy message storage at this scale? Each of these questions is a section in this lesson.
📋 Quick Reference Card: Chat System Design Scope Summary
| 🎯 Dimension | 📊 Target Value | 🔧 Key Decision Driven By This |
|---|---|---|
| 🌐 Users | 2B total, 500M concurrent | Connection management strategy |
| 📨 Throughput | 100B messages/day | Queue and storage architecture |
| ⚡ Latency | P99 < 500ms | Protocol selection (WebSocket) |
| 🔁 Availability | 99.99% | Replication and failover design |
| 💾 Durability | No loss after ACK | Write-ahead logging, queues |
| 📋 Ordering | Per-conversation | Sequence IDs, single-writer patterns |
The rest of this lesson will equip you to make principled, defensible decisions for every cell in that table. We'll go component by component, decision by decision, building up a complete system design that you can walk through confidently in any interview — whether you're talking to a junior engineer or a principal architect at Meta.
Let's build something real.
Core Architecture: Real-Time Messaging, Protocols, and Connection Management
Before a single message can travel from Alice's phone to Bob's, someone has to make a fundamental engineering decision: how do those two devices stay in touch with the server? This choice ripples through your entire architecture, affecting latency, battery life, server resource consumption, and scalability. Getting it right is the first real test an interviewer is watching for.
The Four Approaches to Real-Time Communication
Let's walk through the evolution of real-time communication on the web, because understanding why each approach falls short will make your final choice feel inevitable rather than arbitrary.
Short Polling is the naive starting point. The client sends an HTTP request every N seconds asking "any new messages for me?" The server responds immediately—with data if available, or with an empty response if not. It's simple to implement, but it's deeply wasteful. With 1 million connected users polling every 3 seconds, you're generating over 300,000 HTTP requests per second, the vast majority returning nothing. Each request also carries the full overhead of HTTP headers, TCP handshake amortization, and server thread allocation.
Client Server
|---GET /messages?---->|
|<----[] (empty)-------|
| (wait 3 seconds) |
|---GET /messages?---->|
|<----[] (empty)-------|
|---GET /messages?---->|
|<----[{msg: "Hi!"}]--|
Long Polling is a clever improvement. The client sends a request, but the server holds the connection open until a message arrives or a timeout occurs (typically 30–60 seconds). When a message arrives, the server responds immediately and the client immediately opens a new long-polling request. This reduces empty responses dramatically, but it still has serious drawbacks: each pending request ties up a server thread (in thread-per-request models), latency spikes when the timeout hits, and the pattern is fundamentally half-duplex—the client can't easily send and receive simultaneously without managing two separate connections.
Server-Sent Events (SSE) is a one-way streaming protocol built into the browser's EventSource API. The server pushes a stream of events to the client over a single long-lived HTTP connection. This is genuinely excellent for notification feeds, stock tickers, or news updates—anywhere the server broadcasts and the client only listens. But for chat, you need bidirectional communication. With SSE, you'd still need a separate HTTP channel for the client to send messages, which adds complexity and asymmetry.
WebSockets solves all of this. A WebSocket connection starts as a standard HTTP request and then upgrades to a persistent, full-duplex TCP connection. Once established, either party—client or server—can send a message at any moment with minimal framing overhead (as little as 2 bytes per frame, compared to hundreds of bytes of HTTP headers). The connection stays open for the lifetime of the session.
🎯 Key Principle: WebSockets are the industry standard for chat applications because chat is inherently bidirectional. You want the server to push messages instantly and you want the client to send messages on the same channel—no polling, no separate upload endpoint.
📋 Quick Reference Card: Real-Time Protocol Comparison
| 🔧 Mechanism | ↔️ Direction | ⚡ Latency | 📦 Overhead | ✅ Best For | |
|---|---|---|---|---|---|
| 🔄 Short Polling | Repeated HTTP requests | Client→Server | High | Very High | Simple status checks |
| ⏳ Long Polling | Held HTTP request | Client→Server | Medium | High | Lightweight notifications |
| 📡 SSE | Streamed HTTP response | Server→Client only | Low | Medium | News feeds, dashboards |
| 🔌 WebSockets | Persistent TCP tunnel | Full-duplex | Very Low | Very Low | Chat, gaming, collaboration |
How a WebSocket Connection Actually Works
Understanding the lifecycle helps you answer follow-up questions about reconnection, heartbeats, and load balancing. Here's what happens step by step:
- Handshake: The client sends an HTTP GET with an
Upgrade: websocketheader. The server responds with101 Switching Protocols. - Open: The TCP connection is now a WebSocket channel. Both sides can send frames immediately.
- Message Exchange: Messages flow in both directions as frames. A frame can carry text (UTF-8) or binary data.
- Ping/Pong: Either side can send a
pingframe; the other must respond withpong. This acts as a heartbeat to detect dead connections and keep NAT/firewall mappings alive. - Close: Either side sends a
closeframe with an optional status code, and the other acknowledges it.
Here's a practical WebSocket server in Node.js using the ws library that demonstrates connection tracking and message routing between two users:
// chat-server.js — Basic WebSocket chat server with user routing
const WebSocket = require('ws');
const server = new WebSocket.Server({ port: 8080 });
// connectionMap holds userId -> WebSocket connection
// This is the core of "stateful" chat servers
const connectionMap = new Map();
server.on('connection', (socket, request) => {
// In production, extract userId from a JWT in the query string or cookie
const userId = request.url.split('userId=')[1];
if (!userId) {
socket.close(4001, 'Authentication required');
return;
}
// Register this connection
connectionMap.set(userId, socket);
console.log(`User ${userId} connected. Active connections: ${connectionMap.size}`);
// Handle incoming messages
socket.on('message', (rawData) => {
const message = JSON.parse(rawData);
// Expected shape: { toUserId: string, text: string, messageId: string }
const { toUserId, text, messageId } = message;
const recipientSocket = connectionMap.get(toUserId);
if (recipientSocket && recipientSocket.readyState === WebSocket.OPEN) {
// Recipient is connected to THIS server — deliver directly
recipientSocket.send(JSON.stringify({
fromUserId: userId,
text,
messageId,
timestamp: Date.now()
}));
// Acknowledge delivery to sender
socket.send(JSON.stringify({ type: 'ACK', messageId }));
} else {
// Recipient is offline or on a DIFFERENT server
// In a real system, publish to Kafka/Redis here (covered below)
socket.send(JSON.stringify({ type: 'PENDING', messageId }));
}
});
// Handle disconnection
socket.on('close', () => {
connectionMap.delete(userId);
console.log(`User ${userId} disconnected.`);
});
// Handle errors without crashing the server
socket.on('error', (err) => {
console.error(`Socket error for user ${userId}:`, err.message);
connectionMap.delete(userId);
});
});
This example reveals something critical that many candidates miss: the connectionMap. The server is literally holding a mapping of user IDs to open socket objects in memory. This is not a stateless REST endpoint—it cannot be, by definition.
Stateful Chat Servers vs. Stateless HTTP Services
This distinction is one of the most important architectural concepts in this entire problem—and one of the most commonly fumbled in interviews.
❌ Wrong thinking: "I'll just put the chat service behind a load balancer like my REST APIs and scale horizontally."
✅ Correct thinking: "Chat servers are inherently stateful. Each server owns the connections it has open, and routing must account for that."
In a standard stateless microservice (say, a user profile service), any request from Alice can go to any server instance because the server holds no client-specific state in memory. The load balancer can round-robin freely.
A Chat Service (sometimes called a Connection Manager or Presence Server) is different. It holds thousands or millions of open WebSocket connections, each registered to a specific user. If Alice's connection is on Server A and Bob's is on Server B, and Alice sends a message to Bob, Server A cannot deliver it directly—it doesn't have Bob's socket.
Alice Bob
| |
| |
[Chat Server A] [Chat Server B]
Alice's socket Bob's socket
is HERE is HERE
When Alice sends to Bob:
Server A knows Alice but NOT Bob's socket location.
⚠️ Common Mistake: Designing the chat service as stateless and trying to store WebSocket handles in Redis. WebSocket connections are OS-level file descriptors. They cannot be serialized and shared across processes. The connection object must live in the same process that accepted it.
💡 Mental Model: Think of each Chat Server as a telephone switchboard operator who knows only the extensions plugged into their board. To connect to an extension on another board, the operators need a way to talk to each other.
Horizontal Scaling: The Cross-Server Message Problem
As your system grows, you'll run dozens or hundreds of chat server instances. The question becomes: when Alice (on Server A) messages Bob (on Server B), how does Server A figure out where Bob is, and how does the message get there?
The solution is a message broker acting as the inter-server communication backbone. The two most common choices are Apache Kafka and Redis Pub/Sub.
Redis Pub/Sub is simpler and lower-latency but doesn't persist messages. Each chat server subscribes to a channel named after each user it's currently hosting. When Server A needs to deliver a message to Bob, it publishes to Bob's channel. Server B (which hosts Bob's socket) is subscribed to that channel and receives the message instantly, then pushes it down Bob's WebSocket.
Alice sends "Hi Bob!"
|
v
[Chat Server A]
Looks up: "Is Bob on me?" → No
Publishes to Redis channel: user:bob
|
v
[Redis Pub/Sub]
Channel: user:bob
|
v
[Chat Server B] ← subscribed to user:bob channel
Looks up Bob's local socket
Pushes "Hi Bob!" down WebSocket
|
v
Bob
Kafka is preferred for durability and replay. Each chat server publishes messages to a Kafka topic (often partitioned by conversation ID to preserve ordering). A consumer group of chat servers—or a dedicated delivery service—reads from Kafka and routes to the right socket. Kafka retains messages for a configurable period, enabling retry and offline delivery guarantees.
💡 Real-World Example: WhatsApp uses a variant of this pattern with Erlang/OTP as their connection manager (chosen specifically for its world-class support for millions of concurrent lightweight processes). Each Erlang process manages one user's connection and can message other processes directly—the language's actor model makes the routing problem almost native.
🤔 Did you know? A single modern chat server running in Node.js or Go can handle 50,000–100,000 concurrent WebSocket connections on a single machine, primarily because these runtimes use event-loop or goroutine models that don't require a thread per connection.
Fan-Out Patterns for Group Messaging
One-to-one messaging is challenging enough, but group chats introduce a new scaling problem called the fan-out problem. When Alice sends a message to a group with 500 members, the system needs to deliver that message to all 500 recipients. How you handle this architecturally has enormous performance consequences.
There are two primary patterns:
Write-on-Send (Push Fan-Out / Fan-Out at Write Time)
When Alice sends a message, the system immediately writes a copy of the message (or a reference to it) into each recipient's message queue or inbox. Delivery is eager.
## Pseudocode: Write-on-Send (Fan-Out at Write Time)
def send_group_message(sender_id, group_id, message_content):
message = {
'id': generate_uuid(),
'sender': sender_id,
'content': message_content,
'timestamp': now()
}
# Persist the canonical message once
message_store.save(message)
# Fan out to every group member's inbox
members = group_service.get_members(group_id) # Returns [user_id, ...]
for member_id in members:
if member_id == sender_id:
continue
# Write a pointer (message_id) into each user's inbox
inbox_service.push(user_id=member_id, message_id=message['id'])
# Optionally, deliver in real time via pub/sub
pubsub.publish(channel=f'user:{member_id}', data=message)
Pros: Reading is fast—each user's inbox is pre-populated. Real-time delivery is straightforward.
Cons: For groups with thousands of members (or celebrity users with millions of followers, like Instagram's feed problem), writing millions of inbox entries per message creates enormous write amplification. A group with 10,000 members sending 100 messages per minute generates 1 million inbox writes per minute from that group alone.
Write-on-Receive (Pull Fan-Out / Fan-Out at Read Time)
Instead of writing to every member's inbox at send time, the system stores the message once and lets each recipient fetch it when they connect or request it. The group's timeline is shared, not duplicated.
## Pseudocode: Write-on-Receive (Fan-Out at Read Time)
def send_group_message(sender_id, group_id, message_content):
message = {
'id': generate_uuid(),
'group_id': group_id,
'sender': sender_id,
'content': message_content,
'timestamp': now()
}
# Write ONCE to the group's shared message log
group_message_log.append(group_id=group_id, message=message)
# Notify members in real time (lightweight signal only)
group_members = group_service.get_members(group_id)
for member_id in group_members:
pubsub.publish(
channel=f'user:{member_id}',
data={'type': 'NEW_GROUP_MSG', 'group_id': group_id, 'message_id': message['id']}
)
def get_group_messages(user_id, group_id, last_seen_message_id):
# Each user reads from the shared log, filtering from their last seen position
return group_message_log.fetch_after(
group_id=group_id,
after_id=last_seen_message_id
)
Pros: Write cost is constant regardless of group size. Storage is far more efficient—one copy of the message, not N copies.
Cons: Read time is higher since the client must query the shared log. Real-time notification still requires fan-out (but of a lightweight signal, not the full message payload).
🎯 Key Principle: WhatsApp and most production chat systems use a hybrid approach. For small groups (under ~100–500 members), write-on-send is fine. For large groups, write-on-receive from a shared log is used. The pub/sub notification is always a lightweight signal regardless of group size.
SMALL GROUP (<500 members)
Write-on-Send
┌─────────────────────────────┐
Alice ──► Message ─┤ Copy → inbox[B] │
│ Copy → inbox[C] │
│ Copy → inbox[D] ... (500x) │
└─────────────────────────────┘
Fast reads, expensive writes
LARGE GROUP (>500 members)
Write-on-Receive
┌─────────────────────────────┐
Alice ──► Message ─┤ Write ONCE → group log │
│ Signal ──► pub/sub ──► all │
└─────────────────────────────┘
Members pull from shared log on demand
Cheap writes, slightly slower reads
⚠️ Common Mistake: Candidates often design write-on-send for all groups without thinking about scale. An interviewer will immediately probe: "What happens when a group has 100,000 members?" Having the hybrid answer ready demonstrates senior-level thinking.
🧠 Mnemonic: Think of Write-on-Send as a photocopy machine—you make a copy for everyone before the meeting. Think of Write-on-Receive as a bulletin board—you post once and people walk up to read it when they arrive. Small meetings: photocopy. Town halls: bulletin board.
Putting It Together: The Architecture Sketch
At this point in an interview, you should be able to draw this high-level flow confidently:
Client (Alice) Client (Bob)
| |
| WebSocket | WebSocket
| |
[Chat Server A]──────────────────────[Chat Server B]
| \ / |
| [Redis Pub/Sub / Kafka] |
| | |
| [Message Queue / Log] |
| | |
| [Message Storage Service] |
| | |
| [Database] |
| |
└──────────[Load Balancer / API Gateway]────┘
|
[Auth Service] [User Service]
[Presence Service] [Media Service]
The WebSocket connections land on stateful Chat Servers. Those servers coordinate via a message broker. Messages are persisted asynchronously. Stateless services (auth, user lookup, media upload) handle everything else via normal HTTP.
💡 Pro Tip: In your interview, explicitly call out the stateful vs. stateless split early. Say something like: "The Chat Service is intentionally stateful because it must maintain WebSocket connections. Every other service in this system is stateless and scales horizontally without coordination. I'm designing this boundary deliberately." This signals architectural maturity.
With this foundation in place—protocol selection, connection lifecycle, stateful server design, cross-server routing, and fan-out strategies—you have the core real-time communication architecture sorted. The next critical question is where all these messages actually live, which is what we'll tackle in the storage design section.
Storage Design: Choosing the Right Database for Messages, Users, and Media
Every message you send on WhatsApp travels through a sophisticated data layer before it reaches your friend's phone. Designing that layer is one of the most nuanced parts of a chat system interview because there is no single database that handles everything well. The data access patterns for messages, user profiles, contact graphs, and media files are so different that each demands its own purpose-built solution. Understanding why you choose each technology — not just what to choose — is what separates a junior answer from a senior one.
Why Relational Databases Fall Short for Messages
The instinct for many candidates is to reach for a familiar relational database like PostgreSQL or MySQL. It works for user accounts, so why not messages? The answer lies in write volume and access pattern mismatch.
Consider WhatsApp's scale: over 100 billion messages are sent daily. Even at a fraction of that scale, a relational database faces two structural problems. First, relational databases use row-oriented storage, meaning every write acquires locks and updates B-tree indexes in ways that create contention at high write throughput. Second, the access pattern for chat messages is almost never "give me all messages where sender = X" — it is overwhelmingly "give me the last 50 messages in conversation Y, ordered by time." Relational indexes can handle this, but they do not scale horizontally in the way that a chat system requires.
🎯 Key Principle: Choose your database based on your dominant read and write access patterns, not familiarity or general capability.
When a relational database receives millions of concurrent inserts across thousands of chat conversations, it becomes a bottleneck. You can shard a relational database manually, but now you have lost joins, foreign key constraints, and transactions across shards — the very features that made relational databases appealing in the first place.
The Case for Wide-Column Stores
Wide-column stores like Apache Cassandra and HBase were designed for exactly this workload. They are optimized for high write throughput, horizontal scalability, and efficient range scans over sorted keys. In Cassandra, data is distributed across nodes using consistent hashing, meaning you can add nodes without resharding existing data. Writes are appended to an in-memory structure called a memtable and then flushed to immutable SSTables on disk — there are no locks, no B-tree rebalancing, and no contention between concurrent writers.
Relational DB Write Path:
INSERT → Lock row → Update B-tree index → Write to page → Unlock
(serialized at the page/row level)
Cassandra Write Path:
INSERT → Write to commit log → Write to memtable → Return OK
(non-blocking; flush to SSTable happens asynchronously)
The trade-off is that Cassandra sacrifices ad-hoc query flexibility. You cannot run arbitrary SQL joins. Instead, your schema must be designed around your queries — you model your tables to serve specific read patterns. This is a fundamentally different design philosophy, and it is one you must articulate clearly in an interview.
⚠️ Common Mistake — Mistake 1: Designing a Cassandra schema like a relational schema with normalized tables. Cassandra requires denormalization by design. You model one table per query pattern, not one table per entity. ⚠️
Designing the Message Schema in Cassandra
The central query for a chat inbox is: "Retrieve the N most recent messages in conversation C, ordered by time." Everything about your Cassandra schema should serve this query.
Cassandra organizes rows using a partition key and a clustering key. The partition key determines which node stores the data, while the clustering key determines the sort order of rows within that partition. For messages, we want all messages in a single conversation to live in the same partition (so they can be retrieved in a single disk read) and to be sorted by time descending.
Here is a practical Cassandra schema for messages:
-- Cassandra CQL Schema for Chat Messages
CREATE TABLE messages (
conversation_id UUID, -- Partition key: all msgs in one convo live together
message_id TIMEUUID, -- Clustering key: time-based UUID for natural ordering
sender_id UUID, -- Who sent the message
message_type TEXT, -- 'text', 'image', 'video', 'voice'
content TEXT, -- Message body (or media URL for non-text types)
status TEXT, -- 'sent', 'delivered', 'read'
created_at TIMESTAMP, -- Human-readable timestamp (redundant but convenient)
PRIMARY KEY ((conversation_id), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC) -- Latest messages first
AND default_time_to_live = 7776000; -- 90-day TTL for auto-expiry
A few design decisions here deserve explanation. The TIMEUUID type (also called UUID version 1) encodes a timestamp directly into the UUID, which gives us two benefits: rows are naturally sorted by creation time, and there are no UUID collisions even when two messages arrive at the same millisecond from different servers. This is far safer than using a plain timestamp as a clustering key, which would overwrite rows if two messages shared the same timestamp.
The default_time_to_live setting is important for storage management. Cassandra supports per-row TTLs natively, which allows messages to expire automatically without running expensive DELETE operations. This maps cleanly to WhatsApp's approach of not storing messages on the server once they are delivered.
💡 Real-World Example: WhatsApp's actual infrastructure has historically used a customized version of Ejabberd with a backing store that evolved toward this kind of time-series partitioning. Facebook Messenger uses a similar pattern with their HBase-based message store, where the row key encodes the thread ID and timestamp.
For paginating through a conversation's history, the client sends a cursor (the message_id of the oldest message currently loaded) and the query fetches the next page:
-- Fetch next page of messages before a given cursor
SELECT message_id, sender_id, message_type, content, status, created_at
FROM messages
WHERE conversation_id = :conv_id
AND message_id < :cursor_message_id -- Keyset pagination, not OFFSET
LIMIT 50;
This is keyset pagination (also called cursor-based pagination), and it is dramatically more efficient than OFFSET-based pagination. With OFFSET 1000, the database must scan and discard 1,000 rows before returning results. With keyset pagination, Cassandra jumps directly to the cursor position using the clustering key index — O(log n) instead of O(n).
⚠️ Common Mistake — Mistake 2: Using OFFSET-based pagination for message history. At page 100 with 50 messages per page, you are scanning 5,000 rows to return 50. This degrades linearly with scroll depth. Always use cursor-based pagination for time-ordered data. ⚠️
User Profiles and Contact Graphs
User account data — names, phone numbers, profile pictures, account settings — has a very different access pattern from messages. This data is read far more often than it is written (users rarely change their name), involves small record sizes, and benefits from strong consistency guarantees for things like authentication. These characteristics make a relational database like PostgreSQL a natural fit.
-- PostgreSQL schema for user accounts
CREATE TABLE users (
user_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
phone_number VARCHAR(20) UNIQUE NOT NULL, -- Primary identifier in WhatsApp
display_name VARCHAR(100),
profile_pic_url TEXT, -- Points to object storage
public_key TEXT, -- For E2E encryption key exchange
last_seen TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_users_phone ON users(phone_number);
The phone_number field is central to WhatsApp's identity model — users are identified by phone number, not email or username. This table is small enough (even at billions of users) to be effectively cached in Redis, reducing load on the relational database dramatically.
Contact Lists: Relational vs. Graph
The contact/friendship model raises an interesting architectural question. Contact relationships in WhatsApp are directed and asymmetric: if I have your number saved, you appear in my contacts, but you might not have my number saved. This is unlike a bidirectional friendship graph.
For WhatsApp's specific use case, a simple relational join table works well:
CREATE TABLE contacts (
owner_user_id UUID REFERENCES users(user_id),
contact_user_id UUID REFERENCES users(user_id),
nickname VARCHAR(100), -- Optional local alias
added_at TIMESTAMPTZ DEFAULT NOW(),
PRIMARY KEY (owner_user_id, contact_user_id)
);
CREATE INDEX idx_contacts_owner ON contacts(owner_user_id);
Querying "all contacts for user X" is a single indexed lookup on owner_user_id. For WhatsApp's feature set — showing your contact list, seeing which contacts are on WhatsApp — this is sufficient.
🤔 Did you know? Graph databases like Neo4j shine when you need to traverse multi-hop relationships — "friends of friends who also follow this channel." WhatsApp's contact model is effectively one-hop, so the overhead of a graph database is not justified. LinkedIn or Facebook's "people you may know" feature, however, benefits enormously from graph traversal because it involves 2nd and 3rd-degree connections.
❌ Wrong thinking: "I should use Neo4j for all social relationships because users have contacts." ✅ Correct thinking: "I should use a graph database only when my queries require multi-hop traversal. For simple contact lists, a relational join table with proper indexes is simpler and faster."
Blob Storage for Images, Video, and Voice Messages
Media files represent a categorically different storage problem. A text message might be 100 bytes. A video message might be 50 megabytes. Storing binary blobs inside Cassandra or PostgreSQL would be catastrophically wasteful — you would blow past your database's memory budgets, slow down compaction, and make backups enormous.
The standard architecture for media in chat systems is a decoupled blob storage strategy:
┌─────────────┐ 1. Upload media ┌──────────────────┐
│ Client │ ─────────────────────► │ Media Upload │
│ │ │ Service │
└─────────────┘ └────────┬─────────┘
│ 2. Store blob
▼
┌─────────────────┐
│ Object Storage │
│ (e.g., S3) │
└────────┬────────┘
│ 3. Return URL
▼
┌─────────────┐ 4. Send message ┌──────────────────┐
│ Client │ (with media_url only) │ Message Store │
│ │ ─────────────────────► │ (Cassandra) │
└─────────────┘ └──────────────────┘
Later, recipient fetches media:
┌─────────────┐ 5. GET /media/{id} ┌──────────────────┐
│ Recipient │ ─────────────────────► │ CDN Edge Node │
│ │ ◄───────────────────── │ (CloudFront etc)│
└─────────────┘ 6. Serve cached blob └──────────────────┘
Object storage services like Amazon S3, Google Cloud Storage, or Azure Blob Storage are designed for exactly this workload. They provide effectively unlimited capacity, 11-nines of durability through replication, and per-object lifecycle policies. The key architectural pattern is that your message record stores only a URL, not the binary data.
In the Cassandra schema above, the content field for an image message would contain something like https://cdn.whatsapp.net/media/abc123/image.jpg rather than raw bytes. This keeps your message store lean and fast while offloading the heavy lifting to purpose-built infrastructure.
CDN Delivery for Global Performance
A Content Delivery Network (CDN) sits in front of your object storage and caches media files at edge nodes geographically close to users. When a user in Tokyo receives a video message from someone in São Paulo, the video is fetched from S3 once, cached at the Tokyo CDN edge, and served locally to all Tokyo-region users who request the same media ID. This reduces latency from hundreds of milliseconds to tens of milliseconds and dramatically reduces egress costs from your origin storage.
💡 Pro Tip: In an interview, mention that media URLs should be pre-signed or time-limited. Rather than making media files publicly accessible forever, generate short-lived signed URLs (e.g., valid for 24 hours). This limits unauthorized redistribution and is consistent with WhatsApp's privacy model. Most object storage services (S3, GCS) support this natively.
🧠 Mnemonic: "Store the URL, serve the CDN" — your database holds the address, not the content. Like a library card catalog that holds call numbers, not the books themselves.
Data Retention, Archival, and Encryption Implications
WhatsApp's design philosophy has a fundamental impact on server-side storage that is worth addressing explicitly in an interview: end-to-end encryption (E2EE) means the server stores ciphertext it cannot read.
In WhatsApp's E2EE model using the Signal Protocol, each client generates a public/private keypair. When Alice sends a message to Bob, her app encrypts it with Bob's public key. The server relays ciphertext — it never holds the plaintext. This has direct consequences for your storage design:
- No server-side search: You cannot index message content for search because the server cannot read it. Full-text search must happen on-device.
- No server-side backup by default: WhatsApp's servers are a relay, not a mailbox. Once a message is delivered, the server deletes it. The
default_time_to_livein your Cassandra schema is not just storage optimization — it is an architectural commitment. - Key storage is critical: The
public_keyfield in the users table becomes security-critical infrastructure. WhatsApp uses a dedicated key server to distribute and verify public keys.
Message Lifecycle on WhatsApp Servers:
Sender Client ──encrypt──► Server ──store temporarily──► Recipient Client
│ │
(ciphertext only) ──decrypt──►
│ (plaintext,
TTL expires or on device only)
delivered → DELETE
Data retention policies should address three tiers:
| Tier | Data Type | Retention | Storage |
|---|---|---|---|
| 🔥 Hot | Undelivered messages | Until delivered or 30 days | Cassandra |
| 🌡️ Warm | Recent media (last 90 days) | Fixed TTL | S3 Standard |
| ❄️ Cold | Compliance/audit logs (metadata only) | 1-7 years | S3 Glacier / Coldline |
Note that the cold tier stores metadata — timestamps, sender/receiver IDs, message sizes — not message content. This satisfies legal data retention requirements in many jurisdictions without compromising E2EE privacy guarantees.
⚠️ Common Mistake — Mistake 3: Conflating message content retention with metadata retention in your design. Many candidates say "we delete everything when delivered" without acknowledging that metadata logs (for fraud detection, rate limiting, and legal compliance) have different retention requirements and live in separate systems. ⚠️
Archival strategy for media uses object storage lifecycle rules. In AWS S3, you can configure a bucket to automatically transition objects between storage classes:
// S3 Lifecycle Policy for media objects
{
"Rules": [
{
"Status": "Enabled",
"Filter": { "Prefix": "media/" },
"Transitions": [
{
"Days": 30,
"StorageClass": "S3_INTELLIGENT_TIERING"
},
{
"Days": 90,
"StorageClass": "GLACIER_INSTANT_RETRIEVAL"
}
],
"Expiration": {
"Days": 365 // Hard delete after 1 year if not accessed
}
}
]
}
This policy automatically moves infrequently accessed media to cheaper storage tiers without any application-level code changes. Cold storage costs roughly 10x less than hot storage on major cloud providers, making lifecycle policies one of the highest-ROI infrastructure decisions for a media-heavy application.
Putting It All Together: The Storage Architecture
┌─────────────────────────────────────────────────────┐
│ CHAT STORAGE LAYER │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌───────────┐ │
│ │ Cassandra │ │ PostgreSQL │ │ Redis │ │
│ │ │ │ │ │ │ │
│ │ • Messages │ │ • Users │ │ • Session │ │
│ │ • Time TTL │ │ • Contacts │ │ • Presence│ │
│ │ • Sharded │ │ • Auth │ │ • Pub/Sub │ │
│ │ by convID │ │ tokens │ │ │ │
│ └─────────────┘ └─────────────┘ └───────────┘ │
│ │
│ ┌──────────────────────────────────────────────┐ │
│ │ Object Storage (S3) + CDN │ │
│ │ • Images, Video, Voice, Documents │ │
│ │ • Lifecycle: Hot → Warm → Cold → Delete │ │
│ │ • Pre-signed URLs, served via CDN edge │ │
│ └──────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
Each component owns a clear domain. Cassandra handles the high-volume, append-heavy message stream. PostgreSQL manages strongly-consistent user and relationship data. Redis acts as the operational cache and pub/sub bus for presence and session state. Object storage with CDN handles all binary media. No single database is asked to do everything.
📋 Quick Reference Card: Storage Technology Selection
| 🔧 Technology | 🎯 Use Case | 📚 Why It Fits |
|---|---|---|
| 🔥 Cassandra | Message history | High write throughput, time-ordered clustering keys, TTL support |
| 🗄️ PostgreSQL | Users, contacts | Strong consistency, relational joins, low write volume |
| ⚡ Redis | Sessions, presence | Sub-millisecond reads, pub/sub, automatic expiry |
| ☁️ S3 + CDN | Media files | Unlimited capacity, lifecycle policies, geographic distribution |
| 🔗 Graph DB | (Only if needed) | Multi-hop social traversal — not required for basic contact lists |
The discipline of this section — matching storage technology to access pattern rather than defaulting to a single database — is exactly the kind of architectural thinking that impresses interviewers. It signals that you understand tradeoffs, not just tools.
Practical Deep Dive: Message Delivery Guarantees, Ordering, and the Presence System
At this point in the design, you have a protocol chosen, connections managed, and a storage layer decided. Now comes the part where most candidates either pull ahead of the pack or fall behind: the behavioral correctness of the system. How does a message actually get from Alice to Bob without being lost, duplicated, or shown in the wrong order? How does Bob's phone know Alice is typing right now? These aren't cosmetic features — they are the core reliability contracts your users depend on every day. Let's build each one from first principles.
At-Least-Once Delivery and Idempotency: Eliminating the Phantom Message
The fundamental challenge in distributed messaging is that networks lie. A packet sent is not a packet received, and a packet received is not a packet acknowledged. This forces you to choose one of three delivery semantics:
- At-most-once: Fire and forget. Messages may be lost but never duplicated.
- At-least-once: Retry until acknowledged. Messages may be duplicated but never lost.
- Exactly-once: The holy grail. Messages are never lost and never duplicated.
True exactly-once delivery is theoretically impossible across a network without coordination tricks, but you can emulate it by combining at-least-once delivery on the transport layer with idempotent processing on the receiving end.
Here's how it works in practice. When Alice's client sends a message, it generates a client-generated idempotency key — a unique ID (typically a UUID or a snowflake ID) that the client owns. The server stores this key alongside the message. If the network drops the acknowledgment and Alice's client retries, the server detects the duplicate key and returns the already-stored message ID without creating a second record.
Sequence Diagram: At-Least-Once Delivery with Acknowledgment
Alice Client Chat Server Bob Client
| | |
|-- SEND msg ------->| |
| {id: "uuid-1", | |
| text: "Hey", | |
| idempotency_key | |
| : "ik-abc123"} | |
| |-- persist to DB ------>|
| | (check ik-abc123 |
| | not duplicate) |
| |-- PUSH to Bob -------->|
|<-- ACK {msg_id} ---| |
| | [Bob receives]
| [ACK lost!] | |
| | |
|-- SEND msg ------->| (retry after 3s) |
| (same ik-abc123) | |
| |-- detect duplicate --->|
| | (ik-abc123 exists) |
|<-- ACK {msg_id} ---| (return same msg_id) |
| (same as before) | |
On the server side, the deduplication check is a fast key-value lookup. You store the idempotency key in Redis with a TTL of 24–48 hours — long enough to catch any reasonable retry window but short enough to avoid unbounded growth.
import uuid
import redis
import time
r = redis.Redis(host='localhost', port=6379, db=0)
def handle_send_message(payload: dict, db_session) -> dict:
"""
Handles an incoming send-message request from a client.
Returns the canonical message record (new or existing duplicate).
"""
idempotency_key = payload.get("idempotency_key")
sender_id = payload["sender_id"]
conversation_id = payload["conversation_id"]
text = payload["text"]
# Step 1: Check Redis for a duplicate in the retry window
dedup_redis_key = f"idem:{sender_id}:{idempotency_key}"
existing_msg_id = r.get(dedup_redis_key)
if existing_msg_id:
# Duplicate detected — return the already-stored message
# This makes the operation idempotent from the client's perspective
return {"status": "duplicate", "message_id": existing_msg_id.decode()}
# Step 2: Persist the new message to the database
message_id = str(uuid.uuid4())
db_session.execute(
"""
INSERT INTO messages (id, conversation_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, NOW())
""",
(message_id, conversation_id, sender_id, text)
)
db_session.commit()
# Step 3: Store the idempotency key in Redis with a 48-hour TTL
r.setex(dedup_redis_key, 48 * 3600, message_id)
return {"status": "created", "message_id": message_id}
This code does three things in sequence: it checks Redis first (fast path, sub-millisecond), then writes to the database if it's a new message, then caches the mapping so retries are handled without a database hit. The 48-hour TTL means a client retrying after a multi-hour connectivity loss is still protected.
⚠️ Common Mistake: Storing idempotency keys only in the database makes deduplication a slow, locking operation. Always use a fast cache layer like Redis as your first line of defense.
Message Ordering: Logical Clocks and Per-Conversation Sequence IDs
Once you've guaranteed delivery, you face the next challenge: messages must appear in the right order. This sounds trivial until you have multiple servers, clients sending simultaneously, and network latency making wall-clock timestamps unreliable.
Using raw wall-clock time (created_at TIMESTAMP) for ordering is dangerous. Two messages sent one millisecond apart from clients in different time zones or with unsynchronized clocks will sort incorrectly. The solution is a per-conversation sequence ID — a monotonically increasing integer scoped to each conversation.
🎯 Key Principle: Never trust client-provided timestamps for message ordering. Use a server-assigned sequence number that is authoritative and monotonic.
Here's how the sequencing works: each conversation has a counter stored in Redis. When a message arrives, the server atomically increments this counter and assigns the resulting value as the message's seq_id. Because Redis operations are single-threaded, two messages cannot receive the same sequence number.
import redis
r = redis.Redis(host='localhost', port=6379, db=0)
def assign_sequence_id(conversation_id: str) -> int:
"""
Atomically increments and returns the next sequence number
for a given conversation. Uses Redis INCR which is atomic.
"""
key = f"seq:{conversation_id}"
# INCR is atomic in Redis — safe under concurrent access
seq_id = r.incr(key)
return seq_id
def handle_out_of_order_on_client(message_buffer: list, new_message: dict) -> list:
"""
Client-side function to insert a newly received message into the
correct position in a sorted buffer, handling out-of-order delivery.
Returns the sorted buffer.
"""
# Add the new message to the buffer
message_buffer.append(new_message)
# Sort by server-assigned sequence ID — the source of truth
message_buffer.sort(key=lambda m: m["seq_id"])
# Detect gaps: if seq_ids are not contiguous, we have missing messages
gaps = []
for i in range(1, len(message_buffer)):
expected = message_buffer[i-1]["seq_id"] + 1
actual = message_buffer[i]["seq_id"]
if actual != expected:
gaps.append((expected, actual - 1)) # range of missing seq_ids
if gaps:
# Signal to the client to request missing messages from server
print(f"Gap detected, requesting seq_ids: {gaps}")
# In a real client, this triggers a fetch for missing messages
return message_buffer
The client-side buffer approach handles an important edge case: messages delivered out of order due to network conditions. The client holds messages in a sorted buffer keyed by seq_id and only renders them once any gaps are resolved. If a gap persists beyond a short timeout (say 500ms), the client proactively requests the missing messages from the server.
💡 Mental Model: Think of seq_id like page numbers in a book. Even if pages arrive in the wrong order during shipping, you always know exactly where each page belongs and can tell immediately if a page is missing.
Designing the Three Delivery Receipts
The iconic WhatsApp receipt system — one gray tick, two gray ticks, two blue ticks — is elegant in its simplicity but surprisingly nuanced to implement. Each state transition represents a distinct event with its own database write pattern.
Delivery Receipt State Machine
[Message Created]
|
v
✓ SENT
(one gray tick)
Server has stored the message
|
v
✓✓ DELIVERED
(two gray ticks)
Bob's device has received the message
|
v
✓✓ READ
(two blue ticks)
Bob has opened the conversation
Sent is the easiest: it happens when the server writes the message to the database and sends an ACK back to Alice. No additional tables needed — the message row existing is the receipt.
Delivered requires Bob's client to send an acknowledgment back to the server when it receives the push notification and loads the message. The server then updates the message row:
-- Schema addition: delivery tracking columns on the messages table
ALTER TABLE messages ADD COLUMN delivered_at TIMESTAMP NULL;
ALTER TABLE messages ADD COLUMN read_at TIMESTAMP NULL;
-- Triggered when Bob's client sends a DELIVERED ack
UPDATE messages
SET delivered_at = NOW()
WHERE id = 'msg-uuid-here'
AND delivered_at IS NULL; -- Idempotent: ignore if already set
-- The server then pushes a receipt event back to Alice's client
-- {"type": "receipt", "msg_id": "msg-uuid-here", "status": "delivered"}
Read is triggered when Bob opens the conversation and the messages become visible. Rather than updating each message individually — which would generate O(n) writes every time someone opens a chat — you use a high-water mark approach: store the latest seq_id that Bob has read per conversation, then mark all earlier messages as read in a single operation.
-- High-water mark table: one row per user per conversation
CREATE TABLE read_receipts (
user_id UUID NOT NULL,
conversation_id UUID NOT NULL,
last_read_seq_id BIGINT NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, conversation_id)
);
-- When Bob opens a conversation, upsert his high-water mark
INSERT INTO read_receipts (user_id, conversation_id, last_read_seq_id, updated_at)
VALUES ('bob-uuid', 'conv-uuid', 482, NOW())
ON CONFLICT (user_id, conversation_id)
DO UPDATE SET
last_read_seq_id = GREATEST(read_receipts.last_read_seq_id, EXCLUDED.last_read_seq_id),
updated_at = NOW();
The GREATEST() function ensures the high-water mark only ever moves forward — you can't accidentally un-read a message by receiving an out-of-order receipt event.
Building the Presence System
The green dot next to a contact's name is one of the most psychologically loaded features in a chat app — and one of the most dangerous at scale if implemented naively. The challenge is that presence is inherently ephemeral and high-frequency: millions of users connecting, disconnecting, and keeping connections alive every second.
🎯 Key Principle: Presence data is the opposite of message data. Messages are permanent and write-rare. Presence is transient and write-constant. They need fundamentally different storage strategies.
The architecture uses three components working together:
Heartbeat signals: Each connected client sends a lightweight heartbeat ping to its WebSocket server every 30 seconds. The server-side handler is a single Redis write.
Redis TTL keys: When a heartbeat arrives, the server sets or refreshes a key in Redis: presence:{user_id} with a value of "online" and a TTL of 60 seconds (twice the heartbeat interval, giving a buffer for one missed beat). If a user closes the app without sending a disconnect event, this key will simply expire after 60 seconds and the user will appear offline automatically.
Last-seen timestamps: When a user goes offline (either by explicit disconnect or TTL expiry), the server writes their last_seen timestamp to a durable database (PostgreSQL or Cassandra). This is a single row update per user and is write-infrequent because it only triggers on state transitions, not on every heartbeat.
Presence System Architecture
Bob's Phone WebSocket Server Redis PostgreSQL
| | | |
[heartbeat every 30s] | | |
|--- PING ------------------>| | |
| |-- SET presence:bob | |
| | "online" EX 60 --->| |
| | | |
[app closed / no heartbeat] | | |
| | [key expires after 60s] |
| |<-- keyspace notif ---| |
| | (presence:bob expired) |
| |-- UPDATE users SET --|-----> last_seen=NOW()|
| | last_seen=NOW() | |
⚠️ Common Mistake: Setting a heartbeat TTL equal to the heartbeat interval (30s = 30s TTL) means a single delayed packet causes a false offline flicker visible to all of a user's contacts. Always use TTL = 2× heartbeat interval as a minimum buffer.
For subscribing to presence updates, contacts need to know when Bob comes online or goes offline. Rather than polling Redis, you use Redis keyspace notifications (or a Pub/Sub channel) to push presence change events to the WebSocket servers managing Bob's contacts, which then push updates to those clients' screens.
💡 Real-World Example: WhatsApp deliberately delayed presence updates for privacy — they noticed that extremely precise last-seen timestamps enabled stalkerish behavior. The system architecture supports high-precision timestamps, but the product decision was to bucket them into "today", "yesterday", "last week". This is a great point to raise in an interview to show product thinking.
🤔 Did you know? At WhatsApp's scale (~2 billion users), even storing a single presence key per user in Redis requires careful sharding. A single Redis node maxes out around 100–200GB. Presence keys are distributed across a Redis cluster with consistent hashing on user_id.
Offline Message Queuing: The Reconnection Flush
When Bob is offline, messages addressed to him cannot be pushed over a WebSocket — there is no open connection. The system needs to buffer those messages and deliver them when Bob reconnects.
The flow has two phases:
Buffering Phase (Bob is offline): When the server tries to deliver a message to Bob and finds no active WebSocket connection, it stores the message in an offline message queue. The simplest implementation is a sorted set in Redis keyed by user ID, with the seq_id as the score. Alternatively, for durability, messages are already in the primary database — the server just needs to track the "last delivered" cursor per user.
Flush Phase (Bob reconnects): When Bob's client opens a WebSocket connection, the first thing it sends is a sync request containing its last known seq_id. The server queries the messages table for all messages in Bob's conversations with a seq_id greater than Bob's cursor, and streams them down the socket in batches.
Offline Queue Flush Sequence
Bob's Client WebSocket Server Message DB
| | |
[reconnects] | |
|--- CONNECT ------------->| |
|--- SYNC_REQUEST -------->| |
| {last_seq_id: 477} | |
| |-- SELECT messages ----->|
| | WHERE conv_id IN ... |
| | AND seq_id > 477 |
| | ORDER BY seq_id |
| | LIMIT 100 |
| |<-- rows 478..577 ------||
|<-- MESSAGE_BATCH --------| |
| (seq_ids 478..577) | |
|--- ACK {577} ----------->| [update delivered_at] |
| | |
|<-- MESSAGE_BATCH --------| [if more messages...] |
| (seq_ids 578..604) | |
The batching is crucial — don't send thousands of messages in a single frame. Use a page size of 50–100 messages and wait for an ACK before sending the next batch. This provides natural back-pressure if Bob's phone is on a slow connection.
💡 Pro Tip: The cursor-based reconnection approach means you don't actually need a separate offline queue table in most cases. The primary messages table is the queue — you just need to efficiently query "messages after cursor X for user Y." Make sure your index covers (conversation_id, seq_id) to make this query fast.
🧠 Mnemonic: Think of the reconnection flush as "catch me up" — the client tells the server its last known position, and the server plays back everything it missed, like rewinding a podcast to where you left off.
Putting It All Together: The Reliability Stack
📋 Quick Reference Card: Delivery Reliability Components
| 🔧 Component | 🎯 Problem Solved | 📦 Implementation |
|---|---|---|
| 🔑 Idempotency Keys | Duplicate messages on retry | UUID per send + Redis dedup cache |
| 🔢 Sequence IDs | Out-of-order delivery | Redis INCR per conversation |
| ✓ Sent Receipt | Confirm server persistence | ACK on DB write |
| ✓✓ Delivered Receipt | Confirm device receipt | Client ACK → DB update |
| 👁 Read Receipt | Confirm message seen | High-water mark upsert |
| 💚 Presence | Online/offline status | Redis TTL key + heartbeat |
| 📬 Offline Queue | Buffered delivery | Cursor-based reconnection flush |
The elegance of this design is that each layer solves exactly one problem without coupling to the others. Idempotency keys live at the send layer. Sequence IDs live at the storage layer. Receipts live at the delivery-confirmation layer. Presence lives in its own isolated Redis namespace. They compose together cleanly because each has a single, well-defined responsibility.
In your system design interview, walking through this layered approach — rather than hand-waving at "we'll use WebSockets and a database" — is exactly what distinguishes a senior-level answer. You're not just describing what to build; you're explaining why each mechanism exists, what failure mode it prevents, and what trade-offs it introduces.
Common Pitfalls and What Interviewers Are Really Evaluating
Reaching the final stretch of a chat system design interview, many candidates feel a false sense of security. They've sketched a WebSocket server, mentioned Cassandra, drawn a few arrows, and called it done. But experienced interviewers — engineers who have built systems at WhatsApp, Telegram, or Slack scale — are watching for something more nuanced: your ability to anticipate failure modes before they bite you. This section dissects the most damaging mistakes candidates make and, more importantly, reveals the mental model that separates a "hire" from a "strong hire" response.
Pitfall 1: The Single-Database Monolith Trap
The most statistically common mistake in chat system design interviews is treating message storage as a straightforward database problem. Candidates draw a box labeled "Messages DB," maybe mention PostgreSQL or MySQL, and move on. At 1,000 users, this is fine. At 500 million users exchanging 100 billion messages per day — WhatsApp's real-world numbers — this design collapses catastrophically.
Hot spots are the core problem. In a single unpartitioned database, all writes for a given chat thread land on the same node. A viral group chat with 50,000 members generates a continuous thundering herd of writes to one partition. Read latency spikes, write throughput saturates, and the single node becomes a single point of failure (SPOF).
Without partitioning (dangerous):
User A ──┐
User B ──┤──► [Single Messages Table] ◄── 💀 Hot spot
User C ──┘ (all writes here)
...50k users
With partitioning by chat_id (correct):
User A ──┐
User B ──┤──► Partition Router
User C ──┘ │
├──► [Shard 0: chat_id hash 0-25%]
├──► [Shard 1: chat_id hash 25-50%]
├──► [Shard 2: chat_id hash 50-75%]
└──► [Shard 3: chat_id hash 75-100%]
The correct approach, as covered in the storage section of this lesson, is to partition by chat_id using consistent hashing. But the interview pitfall isn't just not knowing this — it's not proactively raising it. Interviewers want to hear you say: "I'd partition the messages table by chat_id to distribute write load and avoid hot spots. One tradeoff is that cross-chat queries become expensive, but those are rare in a messaging workload."
⚠️ Common Mistake: Partition Key Selection Error
A subtler version of this pitfall is choosing the wrong partition key. Some candidates suggest partitioning by user_id (the sender). This sounds logical but creates severe hot spots for popular users — imagine partitioning by sender in a broadcast channel where one account sends millions of messages. Always partition messages by chat_id, not sender_id.
Here's a simplified schema that illustrates the correct partitioning strategy in Cassandra-style CQL:
-- Cassandra schema with correct partition key
CREATE TABLE messages (
chat_id UUID, -- partition key: distributes load by conversation
message_id TIMEUUID, -- clustering key: orders messages within a partition
sender_id UUID,
content TEXT,
created_at TIMESTAMP,
status TEXT, -- 'sent', 'delivered', 'read'
PRIMARY KEY (chat_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);
-- Queries by chat_id are single-partition (fast)
-- message_id as TIMEUUID gives natural time ordering
-- Avoids cross-partition scatter-gather for common access patterns
The chat_id as partition key ensures all messages for a conversation live together (efficient retrieval) while distributing different conversations across nodes (no hot spots). The TIMEUUID clustering key gives you time-ordered retrieval within a partition without a separate sort step.
💡 Pro Tip: Mention time-to-live (TTL) policies. Production chat systems don't keep every message forever on hot storage. WhatsApp stores messages only until delivery; after that, the device holds the history. Discussing TTL shows operational maturity that impresses senior interviewers.
Pitfall 2: The Stateless Server Illusion
WebSockets are stateful connections. When a client connects to Chat Server 3 and maintains an open socket there, any message destined for that client must reach Chat Server 3 specifically — not Chat Server 1 or Chat Server 7. Candidates who design a fleet of chat servers and assume they can route traffic freely are committing what interviewers call the stateless server illusion.
The problem manifests clearly in a diagram:
❌ Wrong thinking — treating chat servers as stateless:
Alice ──WebSocket──► [Chat Server 1] Bob ──WebSocket──► [Chat Server 3]
│
"Send to Bob" ──► Load Balancer ──► [Chat Server 1 or 2 or 3?]
↑
Bob's connection is on Server 3!
Message gets LOST if sent to Server 1.
✅ Correct thinking — connection registry with routing:
Alice ──WebSocket──► [Chat Server 1] Bob ──WebSocket──► [Chat Server 3]
│
"Send to Bob"
│
┌─────────▼──────────┐
│ Connection Registry│ (Redis: user_id → server_id)
│ Bob → Server 3 │
└─────────┬──────────┘
│
Route to [Chat Server 3] ──► Bob's open socket
The correct solution requires a connection registry — a fast lookup store (Redis is the canonical choice) that maps each connected user_id to the specific server instance holding their WebSocket connection. When Server 1 needs to deliver a message to Bob, it queries Redis, discovers Bob is on Server 3, and either calls Server 3 directly (via an internal gRPC call) or publishes to a message broker (Kafka, Redis Pub/Sub) that Server 3 subscribes to.
## Simplified connection registry logic using Redis
import redis
import json
r = redis.Redis(host='redis-cluster', port=6379)
def register_connection(user_id: str, server_id: str, ttl_seconds: int = 300):
"""Called when a user opens a WebSocket connection."""
key = f"conn:{user_id}"
r.setex(key, ttl_seconds, server_id) # TTL auto-clears stale entries on disconnect
def get_user_server(user_id: str) -> str | None:
"""Returns the server_id hosting this user's connection, or None if offline."""
key = f"conn:{user_id}"
server_id = r.get(key)
return server_id.decode() if server_id else None
def route_message(sender_id: str, recipient_id: str, message: dict):
"""Routes a message to the correct chat server."""
target_server = get_user_server(recipient_id)
if target_server is None:
# User is offline — queue for push notification
queue_push_notification(recipient_id, message)
return
if target_server == MY_SERVER_ID:
# Recipient is on THIS server — deliver directly to their socket
deliver_to_local_socket(recipient_id, message)
else:
# Recipient is on a different server — route via message broker
publish_to_server_channel(target_server, recipient_id, message)
def queue_push_notification(user_id: str, message: dict):
"""Enqueue push notification for offline users (APNs/FCM)."""
r.rpush(f"push_queue:{user_id}", json.dumps(message))
This code illustrates three critical behaviors: registration with TTL (so crashed servers don't leave ghost entries), local vs. remote routing, and the fallback to push notifications for offline users — which brings us directly to the next pitfall.
🎯 Key Principle: Sticky sessions (handled at the load balancer layer via IP hashing or cookie-based affinity) ensure that reconnecting clients return to the same server, reducing registry churn. Mention this when discussing the connection layer.
Pitfall 3: Forgetting That Users Go Offline
This is a surprisingly common omission. Candidates design an elegant real-time system and then implicitly assume all users are always connected. The interviewer asks: "What happens when Bob's phone has no signal?" — and the candidate freezes.
The answer requires integrating Apple Push Notification Service (APNs) for iOS and Firebase Cloud Messaging (FCM) for Android. These are the only authorized channels to wake a mobile device from sleep and deliver a notification. Your system must:
- Detect offline state — the connection registry returns
Nonefor the user - Store the device token — obtained during app registration and stored per user
- Trigger a push notification — via APNs/FCM API, including message preview
- Handle delivery on reconnect — the client syncs missed messages from the server on next connection
Offline message delivery flow:
Alice sends message to Bob (Bob is offline)
│
▼
Chat Server checks connection registry
│
[No active connection for Bob]
│
├──► Store message in Cassandra (messages table)
│
└──► Push Notification Service
│
├──► APNs ──► Bob's iPhone 🍎
└──► FCM ──► Bob's Android 🤖
│
[Bob sees notification, opens app]
│
Bob's app reconnects WebSocket
│
App fetches unread messages from server
(using last_seen_message_id cursor)
⚠️ Common Mistake: Assuming Push = Delivery Push notifications are best-effort. APNs and FCM make no guarantees of delivery. A well-designed system uses push notifications to trigger the client to fetch messages, not as the delivery mechanism itself. The source of truth is always the server-side message store.
💡 Real-World Example: WhatsApp's end-to-end encryption creates an interesting constraint here — the server cannot read message content, so push notifications contain only metadata ("You have a new message from Alice") and the encrypted payload. The app decrypts on-device. Mentioning this in an interview demonstrates real-world awareness.
Pitfall 4: Conflating Group Chat With One-on-One Chat
One-on-one chat and group chat look similar on the surface — both involve sending a message from a sender to recipient(s). But at scale, they diverge dramatically, and failing to distinguish them is a red flag for senior interviewers.
The core problem with group chat is fan-out explosion. In a one-on-one chat, one message creates one delivery event. In a group with N members, one message creates N-1 delivery events. For WhatsApp groups capped at 1,024 members, a single message triggers up to 1,023 simultaneous delivery operations. Now multiply by millions of groups sending messages concurrently.
Fan-out comparison:
One-on-One: Group Chat (N=1024):
Alice ──► [Server] ──► Bob Alice ──► [Server] ──► Member 1
──► Member 2
1 write, 1 delivery 1 write, ──► Member 3
──► ...
──► Member 1023
= 1023 delivery operations!
The two primary strategies for handling fan-out are:
Fan-out on write (push model): When Alice sends a message, immediately push a copy or notification to every member's inbox/queue. Recipients read from their personal queue. Low read latency, high write amplification.
Fan-out on read (pull model): Store the message once in the group's shared log. Each member fetches from the shared log, tracking their own read cursor. Low write amplification, higher read complexity.
Production systems typically use a hybrid approach: fan-out on write for small groups (fast, simple), fan-out on read for very large groups or broadcast channels (write amplification becomes prohibitive). Telegram uses a channel model (pull); WhatsApp uses push delivery but limits group size precisely to control fan-out costs.
🎯 Key Principle: Candidates who proactively say "I want to distinguish group chat from one-on-one because the fan-out characteristics are fundamentally different" immediately signal architectural maturity.
What Interviewers Are Really Evaluating
Beyond specific technical knowledge, experienced interviewers are watching for five meta-skills that distinguish senior-level thinking.
1. Tradeoff Articulation Over Single Answers
There is no universally correct answer to "which database should you use for messages?" The correct interview behavior is to name two or three options, state what each optimizes for, and then commit to one with a clear reason. Cassandra gives you write throughput and horizontal scaling at the cost of eventual consistency. PostgreSQL gives you strong consistency and flexible queries at the cost of scaling complexity. A candidate who says "just use Cassandra" without explaining why sounds memorized. A candidate who says "Cassandra is strong here because our workload is write-heavy and eventual consistency is acceptable for message history — here's the tradeoff" sounds like an engineer.
2. Proactive Scoping and Constraint Setting
The best candidates don't wait to be asked clarifying questions — they proactively scope the problem. "Before I start, I want to establish: are we designing for 1:1 chat only, or group chat too? What's the maximum group size? Do we need message history, or just real-time delivery? Do we need end-to-end encryption?" This behavior signals that you understand real engineering requires understanding requirements, and it lets you tailor your design rather than boil the ocean.
💡 Pro Tip: Spend 3-5 minutes on requirements clarification before drawing a single box. Interviewers consistently rate this as one of the highest-signal behaviors in a design interview.
3. CAP Theorem Awareness for Chat Systems
The CAP theorem states that a distributed system can guarantee at most two of: Consistency, Availability, and Partition Tolerance. For a chat system, partition tolerance is non-negotiable (network partitions happen). The real question is whether you sacrifice consistency or availability.
CAP Theorem applied to chat:
Consistency
△
│
CP ─────────┤─────────── CA
(ZooKeeper) │ (RDBMS)
│
───────────────┴───────────── Availability
AP systems
(Cassandra, DynamoDB)
│
Partition Tolerance
For message delivery, you want AP (availability + partition tolerance): it's better to deliver a message slightly out of order than to block all chat because one region is partitioned. For account authentication, you likely want CP: a user should never be able to log in with invalid credentials, even during a partial outage.
Demonstrating that different subsystems of the same application may have different CAP positioning is a strong signal of distributed systems maturity.
🤔 Did you know? WhatsApp operates on a variant of the XMPP protocol built on Erlang (now Elixir), specifically because Erlang's actor model and lightweight processes handle millions of concurrent connections with extreme efficiency. The original WhatsApp server team famously ran 2 million concurrent connections on a single server node during optimization testing.
4. Knowing When to Simplify
Counterintuitively, interviewers penalize over-engineering as much as under-engineering. A candidate who immediately jumps to "we need Kafka, Kubernetes, a service mesh, distributed tracing, and a custom consensus protocol" without any justification is signaling that they apply complexity as a reflex rather than as a solution to specific problems. Good engineers ask: "What is the simplest design that handles the stated requirements? What problem forces us to add each layer of complexity?"
Complexity justification ladder:
Simple chat (1k users):
[WebSocket Server] + [PostgreSQL] — done. No Kafka needed.
│
│ What breaks at 10M users?
▼
Add: Connection registry (Redis) + horizontal server scaling
│
│ What breaks at 100M users?
▼
Add: Message queue (Kafka) + partitioned Cassandra + CDN for media
│
│ What breaks at 500M users?
▼
Add: Geographic distribution + multi-region replication + tiered storage
Walking through this "what breaks next" ladder shows interviewers that you add complexity purposefully, not decoratively.
5. Operational and Failure Awareness
Strong candidates think about what happens when things go wrong, not just when they go right. What happens when the Redis connection registry crashes? (Chat servers can't route — messages queue in Kafka; registry rebuilds from reconnecting clients.) What happens when a push notification service is down? (Messages are durable in Cassandra; clients sync on next connection.) What happens when a message is delivered but the client crashes before acknowledging? (At-least-once delivery with idempotency keys prevents true loss but may create duplicates — client deduplicates by message_id.)
📋 Quick Reference Card: Pitfall vs. Recovery
| 🚫 Pitfall | ✅ Recovery Signal |
|---|---|
| 🔧 Single unpartitioned messages DB | Partition by chat_id; mention consistent hashing |
| 🔧 Stateless WebSocket servers | Connection registry in Redis; broker-based routing |
| 🔧 No offline handling | APNs/FCM integration; pull-on-reconnect pattern |
| 🔧 Group chat = 1:1 chat | Fan-out problem; hybrid push/pull by group size |
| 🔧 No CAP awareness | Differentiate: AP for delivery, CP for auth |
| 🔧 Over-engineering upfront | Complexity ladder: justify each layer with a breaking point |
🧠 Mnemonic: Remember the five meta-skills interviewers prize with STACKS:
- Scope before designing
- Tradeoffs over single answers
- Awareness of CAP theorem
- Complexity justified, not decorative
- Know failure modes (what breaks when)
- Separate subsystems by their consistency needs
The candidates who walk out of chat system design interviews with "strong hire" decisions aren't the ones who memorized the most components. They're the ones who demonstrated that they think like engineers who have been paged at 3am because their design decision from six months ago just became a production incident. Anticipating failure before it happens — and designing around it while clearly articulating why — is the skill that chat system interviews are ultimately measuring.
Key Takeaways and Interview Cheat Sheet for Chat Application Design
You started this lesson knowing that designing a chat system sounds deceptively simple. You finish it knowing exactly why it is not. A chat application like WhatsApp sits at the intersection of nearly every hard problem in distributed systems: persistent connections at scale, message ordering without a global clock, storage engines that can absorb billions of writes, and consistency guarantees that users experience emotionally — a missed message or a phantom "delivered" tick destroys trust. This final section locks everything into memory, arms you with a structured interview narrative, and gives you the precise vocabulary and numbers that separate a senior-level answer from a junior-level one.
Quick Reference: Component Decisions at a Glance
When an interviewer asks "what would you use for X?" you want an immediate, confident answer with a one-sentence justification. The table below is your rapid-fire reference.
📋 Quick Reference Card: Chat System Component Decisions
| 🔧 Component | ✅ Recommended Choice | 💡 One-Line Justification | ⚠️ What You Give Up |
|---|---|---|---|
| 🔌 Client Transport | WebSocket | Full-duplex, persistent, low-latency push without polling overhead | Stateful servers, harder horizontal scaling |
| 💬 Message Store | Apache Cassandra | Partition key on (chat_id) gives linear write scalability and fast range reads | No cross-partition joins, eventual consistency by default |
| 👤 User / Metadata Store | PostgreSQL (or MySQL) | User profiles, contacts, and group membership need ACID transactions | Vertical scaling ceiling, sharding complexity at extreme scale |
| 🖼️ Media Storage | Object Store (S3-compatible) | Cheap, durable, CDN-friendly; messages reference a URL, not blob bytes | Extra network hop; URL expiry management required |
| 📨 Message Broker | Apache Kafka | Durable, ordered, replay-capable log; decouples send from delivery | Operational complexity; at-least-once requires idempotent consumers |
| 🟢 Presence Store | Redis (with TTL) | Sub-millisecond reads, automatic expiry on heartbeat timeout | Memory-only by default; Redis Sentinel or Cluster needed for HA |
| ⚡ Push Notifications | APNs / FCM via async worker | Platform-native reach when WebSocket is closed; fire-and-forget | Delivery not guaranteed; cannot replace in-app socket for reliability |
| 🔍 Message Search | Elasticsearch | Full-text inverted index; Cassandra cannot do arbitrary text search | Sync lag; eventual consistency with primary store |
💡 Pro Tip: Interviewers are not looking for the "correct" row in a lookup table. They want you to say the name and immediately explain the tradeoff you accepted. Silence after naming a technology reads as memorization, not understanding.
The Recommended Interview Narrative Flow
Strong candidates do not free-associate. They walk through a mental checklist in a predictable order that signals senior-level thinking. Here is the five-phase framework you should internalize before your interview.
┌─────────────────────────────────────────────────────────────────┐
│ CHAT SYSTEM INTERVIEW NARRATIVE FLOW │
│ │
│ Phase 1: Requirements Clarification (~3 min) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ • 1:1 or group chat? Max group size? │ │
│ │ • Online indicators, read receipts, typing indicators? │ │
│ │ • Media: images only, or video too? │ │
│ │ • Target scale: DAU, message volume? │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ↓ │
│ Phase 2: Capacity Estimation (~4 min) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ • Messages/sec → storage/day → storage/year │ │
│ │ • Concurrent WebSocket connections → server count │ │
│ │ • Bandwidth for media │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ↓ │
│ Phase 3: High-Level Design (~5 min) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ • Draw: Client → LB → Chat Server → Kafka → DB │ │
│ │ • Introduce Service Registry for WebSocket routing │ │
│ │ • Mention CDN + Object Store for media │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ↓ │
│ Phase 4: Deep Dive — Messaging + Storage (~10 min) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ • Message ID strategy (Snowflake / ULID) │ │
│ │ • Cassandra schema: partition + clustering keys │ │
│ │ • At-least-once + idempotency key = effectively-once │ │
│ │ • Fan-out on write vs. read for group messages │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ↓ │
│ Phase 5: Edge Cases + Extensions (~5 min) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ • Offline delivery / message queue per device │ │
│ │ • Multi-device sync │ │
│ │ • E2E encryption key distribution │ │
│ │ • Presence at scale, spam detection │ │
│ └──────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
🎯 Key Principle: Spend the first 30 seconds of Phase 1 explicitly saying "I want to clarify requirements before designing anything." This signals maturity. Most candidates jump straight to drawing boxes.
Three Numbers to Memorize for Instant Credibility
Numbers anchor abstract design discussions to reality. Memorize these three anchors and practice deriving them out loud.
Number 1 — Message Throughput Target
WhatsApp processes roughly 100 billion messages per day. That is approximately 1.16 million messages per second at peak. For a 500M DAU assumption in your interview, assume each user sends 40 messages/day: 500M × 40 / 86,400 ≈ 230,000 messages/second. Round to ~200K messages/sec for a clean working number.
Number 2 — Storage Estimation Math
A single text message record (message_id, sender_id, chat_id, content, timestamp, status) costs roughly 1 KB. At 100 billion messages/day: 100B × 1KB = 100TB/day. Over a year that is ~36 petabytes for text alone, before replication (typically 3×). This is why Cassandra, not PostgreSQL, is the answer — and this math is what proves you understand why.
Number 3 — Acceptable Latency Threshold
Users perceive a chat message as "instant" if it arrives within 200 milliseconds end-to-end on a good connection. The 99th percentile target at WhatsApp scale is often cited at under 500ms. This threshold is why WebSocket beats long-polling (which adds a full HTTP round-trip per cycle) and why presence heartbeats use a 5–30 second interval rather than per-second pings.
🧠 Mnemonic: "1-100-200" — 1 million messages/sec, 100TB/day storage, 200ms latency target. Say these three numbers early in your estimation phase and the interviewer knows you have done your homework.
The Five Core Tradeoffs, Crystallized
Every strong system design answer is ultimately a series of justified tradeoffs. Here are the five you must be able to articulate without hesitation.
Tradeoff 1 — WebSocket vs. Polling
WebSocket keeps a persistent TCP connection open. The server can push at any time with no request overhead. The cost is statefulness: the load balancer must use sticky sessions or a routing layer (e.g., a Redis-backed service registry) to direct replies to the correct chat server holding the client's socket.
Long-polling is stateless and simpler to scale horizontally but adds one full HTTP round-trip latency per message cycle and wastes server threads holding open connections waiting for events.
✅ Correct thinking: WebSocket for real-time chat; HTTP for non-latency-sensitive REST APIs (user registration, profile updates).
❌ Wrong thinking: "WebSocket everywhere" — file upload and user profile updates do not need a persistent socket; standard HTTPS is more efficient there.
Tradeoff 2 — Cassandra vs. Relational
Cassandra's partition key on chat_id means all messages for a conversation live on the same node, enabling O(1) lookups and sequential disk reads sorted by timestamp. Write throughput scales linearly as you add nodes.
A relational database like PostgreSQL gives you joins, transactions, and a rich query model — but a single table with 100TB of messages will destroy even the best PostgreSQL cluster without heroic sharding work.
✅ Correct thinking: Cassandra for the message log, PostgreSQL for user/group metadata that needs transactional integrity.
Tradeoff 3 — Fan-out on Write vs. Fan-out on Read
For 1:1 chats, there is no fan-out problem — write one copy, the other party reads it. For group chats, you have a choice. Fan-out on write pre-copies the message into each member's inbox at send time: reads are O(1) but writes scale with group size. Fan-out on read stores one copy and every member queries it: writes are O(1) but read amplification grows with group size and active reader count.
💡 Real-World Example: WhatsApp uses fan-out on write for small groups and a hybrid approach for very large groups (broadcast lists). Instagram uses fan-out on write for normal users but fan-out on read for celebrity accounts with millions of followers — the same architectural tension.
Tradeoff 4 — At-Least-Once vs. Exactly-Once
Kafka guarantees at-least-once delivery. A consumer crash after processing but before committing the offset re-delivers the message. True exactly-once across distributed systems requires two-phase commit or Kafka's idempotent producer + transactional API, which adds latency and complexity.
The practical answer for chat: use at-least-once delivery with idempotent consumers. Each message carries a client_message_id (UUID generated on the client). The consumer checks this ID before writing; duplicates are discarded. This achieves effectively-once semantics without distributed transaction overhead.
Tradeoff 5 — Consistency vs. Availability (CAP)
Presence data (online/offline) is a perfect example of choosing availability over consistency. If the Redis node holding your presence data is partitioned, it is better to show a slightly stale "online" indicator than to make the chat application unavailable. The user forgives a 30-second staleness window. They do not forgive an app that fails to load.
Message delivery, however, leans toward consistency: showing a "delivered" tick on the sender's screen when the message was actually lost is a trust-breaking bug.
🎯 Key Principle: Not every component in the same system needs to make the same CAP tradeoff. Presence favors AP; message acknowledgment favors CP. Saying this out loud in an interview demonstrates that you understand CAP as a per-component decision, not a global system property.
Practical Code: Patterns Worth Knowing Cold
You will not write code on a whiteboard during a system design interview, but knowing what the implementation looks like proves your design is grounded in reality.
Idempotent Message Insert in Cassandra
## Python pseudocode — Cassandra idempotent write with LWT (Lightweight Transaction)
## The IF NOT EXISTS clause prevents duplicate inserts when Kafka re-delivers a message.
from cassandra.cluster import Cluster
from uuid import UUID
cluster = Cluster(['cassandra-node-1', 'cassandra-node-2'])
session = cluster.connect('chat_keyspace')
## Prepared statement: only insert if this client_message_id has never been written.
## Cassandra's IF NOT EXISTS uses Paxos — slightly slower, but guarantees idempotency.
insert_stmt = session.prepare("""
INSERT INTO messages (
chat_id,
message_id,
client_message_id,
sender_id,
content,
created_at
) VALUES (?, ?, ?, ?, ?, ?)
IF NOT EXISTS
USING TTL 31536000 -- 1-year TTL; messages auto-expire
""")
def deliver_message(chat_id: str, message_id: UUID,
client_msg_id: str, sender_id: str,
content: str, created_at: int) -> bool:
"""
Returns True if the message was newly written,
False if it was a duplicate (safe to ACK upstream).
"""
result = session.execute(insert_stmt, (
chat_id, message_id, client_msg_id,
sender_id, content, created_at
))
# [applied] column is True only on first write
return result.one().applied
This snippet shows how IF NOT EXISTS makes the Cassandra write idempotent. A Kafka consumer can safely call deliver_message on re-delivery — the second call simply returns False and the consumer ACKs without corrupting data.
Presence Heartbeat with Redis TTL
## Redis presence pattern: client sends heartbeat every 30s.
## Key expires automatically after 45s — no explicit "offline" event needed.
import redis
import time
r = redis.Redis(host='redis-presence', port=6379, decode_responses=True)
PRESENCE_TTL_SECONDS = 45 # 1.5x heartbeat interval for jitter tolerance
def record_heartbeat(user_id: str, device_id: str) -> None:
"""
Called by the WebSocket server each time a heartbeat frame arrives.
Sets or refreshes the TTL so the key stays alive while the client is online.
"""
presence_key = f"presence:{user_id}:{device_id}"
pipe = r.pipeline()
pipe.hset(presence_key, mapping={
"status": "online",
"last_seen": int(time.time()),
"device_id": device_id
})
pipe.expire(presence_key, PRESENCE_TTL_SECONDS)
pipe.execute()
def get_user_presence(user_id: str) -> str:
"""
Returns 'online' if any device key for this user exists (not expired),
otherwise returns 'offline'.
"""
# Scan for any device key belonging to the user
keys = r.keys(f"presence:{user_id}:*")
return "online" if keys else "offline"
The pipeline() call batches the HSET and EXPIRE into a single round-trip, halving the Redis latency per heartbeat. At millions of connected clients, this matters.
Snowflake-Style Message ID Generation
## Simplified Snowflake ID generator — produces 64-bit, time-sortable, unique IDs.
## Critical for Cassandra clustering key ordering without a centralized sequence.
import time
import threading
class SnowflakeIDGenerator:
"""
Bit layout (64 bits total):
- 1 bit : sign (always 0 for positive)
- 41 bits : milliseconds since custom epoch (~69 year range)
- 10 bits : machine/worker ID (supports 1024 nodes)
- 12 bits : sequence number per millisecond (4096 IDs/ms/node)
"""
EPOCH = 1700000000000 # Custom epoch: Nov 2023 (reduces ID magnitude)
WORKER_ID_BITS = 10
SEQUENCE_BITS = 12
MAX_SEQUENCE = (1 << SEQUENCE_BITS) - 1 # 4095
def __init__(self, worker_id: int):
assert 0 <= worker_id < (1 << self.WORKER_ID_BITS), "Invalid worker ID"
self.worker_id = worker_id
self.sequence = 0
self.last_timestamp = -1
self._lock = threading.Lock()
def next_id(self) -> int:
with self._lock:
now = int(time.time() * 1000)
if now == self.last_timestamp:
self.sequence = (self.sequence + 1) & self.MAX_SEQUENCE
if self.sequence == 0: # Sequence exhausted — wait for next ms
while now <= self.last_timestamp:
now = int(time.time() * 1000)
else:
self.sequence = 0
self.last_timestamp = now
# Pack bits together
return ((now - self.EPOCH) << 22) | (self.worker_id << 12) | self.sequence
## Usage: each chat server instance gets a unique worker_id at startup
generator = SnowflakeIDGenerator(worker_id=42)
msg_id = generator.next_id() # Sortable by creation time, globally unique
The Snowflake generator produces IDs that are monotonically increasing within a millisecond, which is exactly what Cassandra needs as a clustering key to sort messages chronologically within a partition.
Extension Topics to Mention Proactively
If you finish your core design early or the interviewer asks "what else would you think about?", these three topics demonstrate depth and will often spark the most interesting part of the conversation.
End-to-End Encryption Architecture
WhatsApp uses the Signal Protocol. Each client generates a public/private key pair on-device. The server stores only public keys. When Alice messages Bob, her client fetches Bob's public key from the server, encrypts the message locally, and sends the ciphertext. The server relays bytes it mathematically cannot read. Key insight for the interview: the server becomes a routing layer, not a trust boundary. This has storage implications (encrypted blobs are not searchable) and backup implications (key loss = permanent data loss).
Multi-Device Sync
When a user adds a second device, every new message must be encrypted and delivered to each device independently. The server must maintain a device registry (user_id → [device_key_1, device_key_2, ...]). Message fan-out now applies even for 1:1 chats: one logical message becomes N encrypted copies, one per registered device. Mention that this is why adding a new device to WhatsApp requires a QR code scan — it is a cryptographic key exchange ceremony, not just a login.
Spam and Abuse Detection Pipelines
A Kafka consumer group streams all messages into an async ML scoring pipeline. Messages are scored for spam signals (mass sending, link patterns, reported content hashes) without reading content on E2E-encrypted systems. Instead, metadata signals are used: message velocity per sender, forward count, identical hash across many senders (PhotoDNA for CSAM detection operates on the client before encryption). Rate limiting at the chat server layer — N messages per user per minute backed by a Redis token bucket — is the first line of defense and is worth mentioning explicitly.
💡 Pro Tip: Mentioning abuse detection unprompted signals that you think about the full product lifecycle, not just the happy path. Interviewers at companies like Meta, Google, and Snap specifically look for this systems-thinking extension.
What You Now Understand That You Did Not Before
Let's be specific about the conceptual leaps this lesson delivered:
🧠 Before: "Chat needs a database and a server."
After: You can specify Cassandra with (chat_id, message_id) partition + clustering keys and explain why this schema gives O(1) conversation reads at petabyte scale.
📚 Before: "WebSocket is for real-time stuff." After: You understand that WebSocket creates stateful connection affinity, requires a service registry for server-to-server message routing, and must coexist with HTTP endpoints for non-real-time operations.
🔧 Before: "Messages should be delivered once." After: You can explain at-least-once + idempotency key as the practical path to effectively-once semantics, and why true exactly-once is an operational cost not worth paying at this scale.
🎯 Before: "Online/offline is just a flag in the database." After: You can design a Redis TTL heartbeat system that handles millions of concurrent presence updates, degrades gracefully on partition, and avoids the thundering-herd problem of synchronized client reconnections.
🔒 Before: "Security is someone else's problem." After: You can articulate the Signal Protocol key exchange model and explain why E2E encryption shifts the trust boundary from the server to the client device.
Practical Next Steps
1. Build a minimal version. Implement a WebSocket server in Node.js or Go that routes messages between two clients through a Redis pub/sub channel. You will immediately discover the connection affinity problem when you try to run two server instances. This 4-hour project teaches more than 40 hours of reading.
2. Read the WhatsApp Engineering Blog and the Signal Protocol specification. WhatsApp has published posts on their move from Ejabberd to their custom Erlang server, their media infrastructure, and their encryption model. Reading primary sources fills in the details that no interview prep guide can cover.
3. Practice the narrative out loud with a timer. Use the five-phase framework above and give yourself 27 minutes total (matching most system design interview slots). Record yourself. You will notice where you stall, what numbers you forgot, and which tradeoffs you glossed over. Fix those gaps before the interview, not during it.
⚠️ Final critical point: The single most common failure mode in this interview is spending 20 minutes on high-level architecture and running out of time before reaching message delivery guarantees and storage schema — which is where interviewers make their hire/no-hire decision. Practice the deep dive phase until it is reflexive.
⚠️ Second critical point: Never say "I would use Kafka for everything" or "Cassandra for everything." Every technology recommendation in this lesson came with an explicit tradeoff. An interviewer who hears unconditional enthusiasm for a single tool will probe hard for the weaknesses you seem unaware of. Always pair your choice with what you gave up.
You now have the vocabulary, the numbers, the schema decisions, the tradeoff arguments, and a structured interview framework to walk into a chat system design interview with the confidence of someone who has built it before. The difference between a good answer and a great one is the tradeoffs. Name them, own them, and defend them.