Key Concepts & Terminology
Master essential vocabulary and concepts used in every system design discussion.
Why System Design Vocabulary Is Your Interview Superpower
Imagine you're sitting across from a senior engineer at a company you've dreamed of joining. The interviewer leans forward and asks: "How would you design a system to handle millions of users uploading photos?" You know the concepts. You've built things. You understand — at some level — what needs to happen. But when you open your mouth, the words come out vague, imprecise, almost apologetic. You say "store the images somewhere fast" instead of caching. You say "spread the load across machines" instead of load balancing. You say "make sure it doesn't go down" instead of discussing fault tolerance or high availability. Before you know it, the interview has slipped away — not because you lacked knowledge, but because you lacked vocabulary. Grab the free flashcards at the end of each section to drill these terms until they're second nature.
This is more common than most candidates realize, and it's one of the most fixable problems in technical interview preparation. System design interviews are fundamentally conversations, and like any conversation, the words you choose send powerful signals about who you are as an engineer. Mastering the language of system design isn't about memorizing jargon for its own sake — it's about developing the fluency to think clearly, communicate precisely, and demonstrate the kind of depth that gets you hired at the senior level.
This lesson is your foundation. We'll explore why terminology matters so deeply, what it costs you to get it wrong, and how the key concepts we'll cover in this lesson connect to everything else in the roadmap.
How Interviewers Read Between the Lines
When a senior engineer interviews you, they're not just evaluating whether you can name the right components. They're building a mental model of where you've been — what systems you've touched, what tradeoffs you've lived through, and how you think under pressure. Terminology is one of the clearest signals they have.
Think about it from the interviewer's perspective. They've spent years designing, debugging, and scaling real systems. They have an intuitive feel for the vocabulary that comes naturally to engineers at different levels. When a candidate says "partition" instead of "split the database up somehow," or describes a "write-ahead log" when explaining data durability, those word choices register immediately. They signal: this person has been in the trenches.
🎯 Key Principle: Interviewers use your vocabulary as a proxy for your experience. The more precisely you speak, the more experienced you appear — even before you've described a single design decision.
This works in reverse, too. Vague language creates doubt. If you say "we need some kind of buffer between the services," an experienced interviewer thinks: Do they mean a message queue? A cache? An API gateway? Do they actually know the difference? That doubt — even if unfounded — costs you. The interviewer now has to spend time probing instead of exploring deeper design territory with you.
Here's the hierarchy of how seniority typically maps to language in system design discussions:
Junior Engineer
└── "Store the data somewhere"
└── "Make it faster"
└── "Add more servers"
Mid-Level Engineer
└── "Use a database for persistence"
└── "Cache the responses"
└── "Scale horizontally"
Senior Engineer
└── "Use an append-only log with periodic compaction"
└── "Apply read-through caching with a TTL tuned to data volatility"
└── "Scale out stateless service tiers behind a load balancer using consistent hashing"
Notice that the senior engineer isn't using more words — they're using more precise words. Each term does real work, carrying specific meaning and implying awareness of tradeoffs.
The Real Cost of Imprecision
Let's be direct: speaking imprecisely in a system design interview doesn't just make you sound junior — it actively costs you points on the evaluation rubric. Most tech companies score system design interviews across several dimensions, and communication clarity is almost always one of them.
⚠️ Common Mistake — Mistake 1: Using vague descriptors instead of technical terms. "Fast storage" means nothing to an evaluator. Redis, Memcached, SSDs with NVMe, or an in-memory data grid — these mean something. Each one implies a different set of tradeoffs you'd need to navigate.
Beyond the scoring rubric, imprecision derails the conversation itself. System design interviews are collaborative — you're meant to think out loud, take feedback, and evolve your design. But if you and the interviewer aren't using shared vocabulary, you end up in a fog of misunderstanding:
Interviewer: "How would you handle consistency here?" Candidate: "We'd make sure the data is always correct." Interviewer: "Right, but are you going for strong consistency or eventual consistency?" Candidate: "...strong?" Interviewer: "And what does that mean for your partition strategy?"
At this point, the candidate is lost — not because they don't understand the concepts, but because they never built the vocabulary bridges that would let them engage with the question confidently. The interviewer has to slow down, simplify, and essentially re-teach, which burns precious time and leaves a poor impression.
❌ Wrong thinking: "I'll explain the concept in plain English and the interviewer will understand what I mean." ✅ Correct thinking: "I'll use precise technical terms because they carry meaning that plain English can't efficiently convey — and they prove I've worked at this level."
💡 Real-World Example: A staff engineer at a major tech company once described interviewing a candidate who clearly understood distributed systems deeply — they'd built one in a previous role. But they kept saying "the main server" and "the backup server" instead of primary and replica, and "when things go wrong" instead of failure scenarios or fault tolerance. The candidate was passed over at the staff level and offered a senior role instead — one level lower. The technical substance was there. The vocabulary wasn't.
A Tale of Two Interviews
Let's make this concrete. Below is the same design question answered twice — once with vague language and once with precise terminology. Both candidates have roughly the same underlying idea. Watch how the language changes the entire character of the exchange.
The Interview with Vague Language
Interviewer: How would you design a system where users can post messages and other users can see those messages quickly?
Candidate: So, I'd store the messages in a database. When someone posts, it goes to the database. When someone reads, we pull it out. To make it fast, we'd cache things. And if there are too many people, we add more computers.
This answer isn't wrong, exactly. But it raises ten questions for every one it answers. What kind of database? What's the caching strategy? How do you invalidate the cache? What does "add more computers" mean — more database nodes? More application servers? The interviewer is left doing all the intellectual work.
The Interview with Precise Terminology
Interviewer: How would you design a system where users can post messages and other users can see those messages quickly?
Candidate: Sure — this looks like a fan-out on write versus fan-out on read tradeoff. For a system with a high read-to-write ratio, I'd lean toward pre-computing the feed on write — when a user posts, we push the message to a message queue and a set of workers asynchronously write to each follower's feed cache in Redis. Reads become a simple cache lookup, which keeps latency low. For users with millions of followers — celebrity accounts — we'd switch to a hybrid model to avoid the thundering herd problem on write.
Same candidate knowledge, dramatically different signal. The second answer demonstrates awareness of real system tradeoffs (fan-out strategies), distributed systems patterns (message queues, async workers), performance characteristics (latency, thundering herd), and practical engineering judgment (the hybrid model for edge cases).
🧠 Mnemonic: P.A.T. — Precise language = Appear senior, Avoid ambiguity, Take control of the conversation.
What We'll Cover in This Lesson — and How It All Connects
System design vocabulary can feel like an ocean. Where do you start? The good news is that the terminology clusters naturally into a handful of categories, and once you understand the categories, individual terms are much easier to learn and remember.
Here's a map of the key concept categories we'll cover across this lesson and its sub-topics:
SYSTEM DESIGN VOCABULARY MAP
═══════════════════════════════════════════════════════════
1. STRUCTURAL COMPONENTS
├── Clients & Servers
├── APIs (REST, GraphQL, gRPC)
├── Load Balancers
└── Proxies (Forward & Reverse)
2. DATA & STORAGE
├── Databases (SQL vs. NoSQL)
├── Caching (Strategies & Invalidation)
├── Replication & Sharding
└── CAP Theorem
3. COMMUNICATION & COORDINATION
├── Synchronous vs. Asynchronous
├── Message Queues & Event Streaming
├── Pub/Sub Patterns
└── Service Discovery
4. RELIABILITY & PERFORMANCE
├── Latency & Throughput
├── Availability & Fault Tolerance
├── Rate Limiting
└── Circuit Breakers
5. SCALE & ARCHITECTURE
├── Horizontal vs. Vertical Scaling
├── Microservices vs. Monoliths
├── CDNs
└── Consistent Hashing
═══════════════════════════════════════════════════════════
Each of these categories feeds directly into the sub-topics that follow in this lesson. In Section 2, we'll dig into structural components — clients, servers, and how they communicate. In Section 3, we'll explore data and storage terminology. Section 4 applies everything to a real design scenario. And Section 5 will specifically address the terminology mistakes that cost candidates jobs — because knowing what not to say matters just as much as knowing what to say.
📋 Quick Reference Card: How Vocabulary Categories Connect to Interview Topics
| 🧠 Category | 🔧 Key Terms | 🎯 Interview Context |
|---|---|---|
| 🔒 Structural | Load balancer, API gateway, reverse proxy | "Walk me through the request lifecycle" |
| 📚 Data & Storage | Sharding, replication, cache eviction | "How do you handle data at scale?" |
| 🔧 Communication | Message queue, event-driven, idempotent | "How do services talk to each other?" |
| 🎯 Reliability | SLA, SLO, fault tolerance, retry logic | "How do you handle failures?" |
| 🔒 Scale | Horizontal scaling, CDN, consistent hashing | "How does your design handle 10x traffic?" |
💡 Pro Tip: Don't try to memorize every term in every category at once. Focus first on having one precise term for every vague concept you currently use. "Fast" → low-latency. "Handles failures" → fault-tolerant. "Spreads the load" → load-balanced. These substitutions alone will immediately elevate how you sound.
The Vocabulary Mindset: Think in Systems, Speak in Systems
There's a deeper reason why vocabulary matters beyond interview performance. The words you use shape the way you think. Engineers who have internalized system design terminology don't just sound more experienced — they are more effective in their daily work, because they have precise cognitive tools for the problems they face.
Consider how a doctor thinks about a patient's symptoms. A medical student might see "the patient seems tired and pale." An experienced physician sees anemia — and with that single word comes an entire mental model: causes, diagnostic pathways, treatment options, contraindications. The word does cognitive work. System design terms do the same thing.
When you say sharding, you're not just naming a technique — you're invoking an entire set of associated concepts: shard keys, hotspot risks, cross-shard queries, rebalancing strategies. When you say circuit breaker, you're invoking failure detection, fallback behavior, and recovery windows. The vocabulary is load-bearing. It holds up entire structures of understanding.
This is why we're investing a full lesson in key concepts and terminology before we get into design patterns, case studies, or deep dives. You need the language before you can use it fluently.
Here's a simple code analogy that illustrates why precise naming matters even in day-to-day engineering:
## ❌ Vague naming — forces readers to reverse-engineer intent
def process(data, thing, flag):
if flag:
temp = do_something(data)
return update(temp, thing)
return data
## ✅ Precise naming — communicates intent immediately
def apply_discount_to_order(order, discount_code, is_eligible):
if is_eligible:
discounted_order = calculate_discount(order, discount_code)
return save_order(discounted_order)
return order
The second version isn't longer in terms of logic — it's the same operations. But it communicates what is happening and why, without requiring the reader to infer intent. Precise vocabulary in system design conversations works exactly the same way. It reduces cognitive load for the interviewer and signals that you think in clear, structured terms.
Now look at how the same principle scales up to a system design conversation. Below is a simplified simulation of a service health-check endpoint — the kind of component you'd discuss when talking about reliability and observability in a system design interview:
import time
import requests
## Simulates a circuit breaker pattern with three states:
## CLOSED (normal), OPEN (failing fast), HALF-OPEN (testing recovery)
class CircuitBreaker:
CLOSED = "CLOSED" # All requests pass through
OPEN = "OPEN" # Requests fail immediately (no calls made)
HALF_OPEN = "HALF_OPEN" # One test request allowed through
def __init__(self, failure_threshold=3, recovery_timeout=10):
self.state = self.CLOSED
self.failure_count = 0
self.failure_threshold = failure_threshold # Failures before opening
self.recovery_timeout = recovery_timeout # Seconds before retrying
self.last_failure_time = None
def call(self, func, *args, **kwargs):
if self.state == self.OPEN:
# Check if enough time has passed to try again
if time.time() - self.last_failure_time > self.recovery_timeout:
self.state = self.HALF_OPEN
else:
raise Exception("Circuit is OPEN — failing fast")
try:
result = func(*args, **kwargs)
self._on_success()
return result
except Exception as e:
self._on_failure()
raise e
def _on_success(self):
self.failure_count = 0
self.state = self.CLOSED
def _on_failure(self):
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.state = self.OPEN
This code block illustrates the circuit breaker pattern — a concept you'll want to be able to name and explain fluently in system design interviews. Notice how the three named states (CLOSED, OPEN, HALF_OPEN) make the code readable. If you were describing this to an interviewer, you'd say: "I'd implement a circuit breaker that transitions from CLOSED to OPEN after three consecutive failures, then enters a HALF-OPEN state after a recovery timeout to test whether the downstream service has recovered." Every word is doing work.
A Note on Learning Strategy for This Lesson
As you move through this lesson and the ones that follow, you'll notice that terms rarely appear in isolation. Replication leads naturally to discussions of consistency. Sharding connects to partition tolerance in the CAP theorem. Caching requires understanding cache invalidation and eviction policies. The vocabulary is a web, not a list.
The best way to learn it is contextually — see the terms in action, in examples, in code, and in realistic interview dialogues. That's exactly what this lesson is designed to do. By the time you reach the cheat sheet in Section 6, the terms won't feel like a list you memorized — they'll feel like tools you've actually used.
🤔 Did you know? Research in cognitive science shows that vocabulary acquisition is dramatically more effective when terms are learned in context rather than in isolation. Flashcards help with retention, but encountering a term in a realistic scenario — like a mock interview dialogue — is what builds the deep, transferable understanding that shows up under pressure.
💡 Mental Model: Think of system design vocabulary as a toolkit. A carpenter doesn't just have a hammer — they know when to reach for it, what it's best suited for, and when a different tool would do a better job. Your goal isn't to collect terms — it's to build a toolkit where every term has a clear place and purpose.
With that foundation in place, let's start filling that toolkit. The next section takes you into the core building blocks of any distributed system: clients, servers, and the communication patterns that connect them. By the end of this lesson, you won't just know what these terms mean — you'll know how to use them in a sentence that makes an interviewer sit up and take notice.
📋 Quick Reference Card: Vague Language → Precise Terminology (Quick Swaps)
| ❌ What You Might Say | ✅ What You Should Say | 🎯 Why It Matters |
|---|---|---|
| 🔧 "Make it faster" | Low-latency architecture | Implies awareness of measurement |
| 📚 "Store it somewhere" | Persistent storage / in-memory cache | Signals you know the tradeoff |
| 🧠 "Handle failures" | Fault tolerance / graceful degradation | Shows you've thought about failure modes |
| 🎯 "Spread the load" | Load balancing / horizontal scaling | Demonstrates architectural vocabulary |
| 🔒 "Avoid going down" | High availability / SLA of 99.9% | Connects design to business requirements |
| 🔧 "Send messages between services" | Asynchronous messaging via a message queue | Implies understanding of decoupling |
🎯 Key Principle: System design vocabulary is not about impressing people with jargon. It's about communicating with precision — so that you and your interviewer can explore complex tradeoffs together, efficiently, without misunderstanding. The goal is clarity at speed. Every term you master is a tool for thinking faster and communicating better.
You've already taken the most important step: recognizing that vocabulary matters and committing to building it intentionally. The rest of this lesson will give you exactly that foundation.
The Core Building Blocks: Clients, Servers, and Communication
Before you can talk intelligently about sharding databases, designing fault-tolerant microservices, or explaining why Netflix uses multiple CDN regions, you need a rock-solid mental model of the basic units that every distributed system is built from. Think of this section as learning the grammar before writing sentences — without it, even your most sophisticated ideas will come out garbled in an interview setting.
Let's build that foundation deliberately, from the ground up.
The Client-Server Model: The Foundation of Everything
At its heart, almost every software system you will ever design follows some variation of the client-server model. The concept is elegantly simple: one party (the client) makes a request, and another party (the server) responds to it.
💡 Mental Model: Think of a restaurant. You (the client) sit down and ask a waiter for a cheeseburger. The kitchen (the server) receives that order, prepares the food, and sends it back to your table. The waiter is the communication layer — the protocol — that carries messages between you and the kitchen. You don't need to know how the kitchen works internally. You just need to know how to place an order.
In software, this plays out through a request-response cycle. A client sends a request to a server, the server processes it, and the server returns a response. This interaction is the heartbeat of the internet.
CLIENT SERVER
| |
|------- HTTP Request --------->|
| GET /users/42 |
| |
| (server processes) |
| |
|<------ HTTP Response ---------|
| 200 OK |
| { "name": "Alice" } |
| |
The client is any entity that initiates a request. This could be a web browser, a mobile app, a command-line tool, or even another server. The server is any entity that listens for and handles those requests. The line between client and server is a role, not a type of machine — the same physical computer can act as a server to one service and a client to another simultaneously.
A Concrete HTTP Example
Here's what this looks like in actual code. Below is a minimal Python server built with Flask that responds to a simple HTTP request, and a client that talks to it:
## server.py — A minimal HTTP server
from flask import Flask, jsonify
app = Flask(__name__)
## This is an "endpoint" — a specific URL path the server listens on
@app.route('/users/<int:user_id>', methods=['GET'])
def get_user(user_id):
# In a real system, this would query a database
# For now, we simulate it with a static response
users = {
42: {"id": 42, "name": "Alice", "role": "admin"},
99: {"id": 99, "name": "Bob", "role": "viewer"}
}
user = users.get(user_id)
if user:
return jsonify(user), 200 # 200 = OK
return jsonify({"error": "User not found"}), 404 # 404 = Not Found
if __name__ == '__main__':
app.run(port=5000)
## client.py — A simple HTTP client making a request to the server
import requests
## The client sends a GET request to the server's endpoint
response = requests.get('http://localhost:5000/users/42')
## The server responds with a status code and a body (the "payload")
print(f"Status Code: {response.status_code}") # 200
print(f"Response Body: {response.json()}") # {'id': 42, 'name': 'Alice', 'role': 'admin'}
This small example quietly introduces several critical vocabulary terms we'll formalize shortly. Notice: the server exposes a specific URL path (/users/<user_id>), the client sends data to that path, and the response comes back with both a status code (200) and a payload (the JSON body). Every system design conversation at scale is, at its core, a more elaborate version of this exchange.
⚠️ Common Mistake: Candidates often describe clients and servers as if they are always physically separate machines. In reality, during development, both often run on the same laptop. What matters is the role each plays in the communication, not their physical location.
Stateless vs. Stateful Systems
One of the most important axes along which distributed systems differ is whether they maintain state between interactions. This distinction has profound consequences for scalability, reliability, and complexity.
A stateless system treats every request as an entirely independent transaction. The server retains no memory of previous interactions. If you send two requests to a stateless server, it handles each one from scratch, with no knowledge that they came from the same client or that the first one ever happened.
A stateful system, by contrast, maintains information across requests. The server remembers context — who you are, what you did last, where you left off.
STATELESS (each request is self-contained):
Client Server
|-- Request 1 (carries all needed info) -->|
|<-------------- Response 1 --------------| (server forgets)
|
|-- Request 2 (carries all needed info) -->|
|<-------------- Response 2 --------------| (server forgets)
STATEFUL (server remembers between requests):
Client Server
|-- Request 1 (login) ------------------>|
|<-------------- Response 1 --------------| (server remembers session)
|
|-- Request 2 ("what's in my cart?") --->|
| (server uses memory from Request 1) |
|<-------------- Response 2 --------------| (server remembers session)
🎯 Key Principle: HTTP itself is a stateless protocol. Each HTTP request is independent. When websites feel stateful (like staying logged in), it's because the application layer adds statefulness on top of HTTP using tools like cookies, tokens, or server-side sessions — not because HTTP itself remembers anything.
The Trade-offs in Practice
Stateless designs are almost universally preferred for the core logic of scalable web services, and here's why: if a server holds no state, any server in a pool of identical servers can handle any request. You can add or remove servers freely without worrying about which client was "connected" to which server.
Stateful designs are more complex to scale. If Server A holds your session data and a load balancer sends your next request to Server B, Server B doesn't know who you are. You either need sticky sessions (always routing a user to the same server — fragile) or externalized state (moving session data to a shared database or cache — more robust but adds complexity).
| Stateless | Stateful | |
|---|---|---|
| 🔧 Scalability | Easy — any server handles any request | Hard — state must be synchronized or externalized |
| 🔒 Resilience | High — server failure loses nothing | Lower — server failure may lose in-memory state |
| 📚 Complexity | Lower — simpler server logic | Higher — must manage state lifecycle |
| 🎯 Use Cases | REST APIs, CDN servers, microservices | Multiplayer game sessions, live video streaming, order processing workflows |
💡 Real-World Example: A payment processing workflow is intentionally stateful. The system must track that Step 1 (charge card) succeeded before executing Step 2 (dispatch order). Losing that state mid-transaction causes real problems. Meanwhile, serving a static image from a CDN is blissfully stateless — the image doesn't care who asked for it.
APIs as Contracts Between System Components
As systems grow, they are broken into multiple components that need to communicate with each other. The agreement about how those components talk — what messages they can send, what format they use, what responses to expect — is called an API (Application Programming Interface).
💡 Mental Model: An API is like a legal contract between two parties. The server promises: "If you send me a request in this format, I promise to respond in that format." The client agrees to follow those rules. Neither party needs to know anything about the other's internal implementation. The contract is all that matters.
There are three dominant communication patterns you'll encounter in system design discussions:
REST (Representational State Transfer)
REST is the most common API style for public-facing web services. It uses standard HTTP methods (GET, POST, PUT, DELETE) mapped to operations on resources — which are things your system cares about, like users, orders, or products. REST APIs communicate using URLs as resource identifiers and typically exchange data as JSON.
RESTful API for a blog system:
GET /posts → retrieve all posts
GET /posts/7 → retrieve post with ID 7
POST /posts → create a new post
PUT /posts/7 → update post with ID 7
DELETE /posts/7 → delete post with ID 7
REST's strength is its simplicity and universality. Any HTTP client can talk to a REST API. Its weakness is that it can be inefficient for complex operations — for example, fetching a user along with all their posts and comments might require multiple round trips.
RPC (Remote Procedure Call)
RPC is a different philosophy. Instead of thinking in terms of resources and HTTP verbs, RPC lets you call a function on a remote server as if it were a local function in your code. Modern systems often use gRPC (Google's implementation), which uses Protocol Buffers for efficient binary serialization.
## With RPC, your client code looks like a local function call
## even though it's executing on a remote server
## gRPC client pseudocode
channel = grpc.insecure_channel('payment-service:50051')
stub = PaymentServiceStub(channel)
## This looks like a local call, but it executes on a remote machine
result = stub.ProcessPayment(
PaymentRequest(user_id=42, amount=99.99, currency="USD")
)
print(result.transaction_id) # Response from the remote server
RPC is favored for internal service-to-service communication where performance matters and you control both ends. REST is favored for public APIs where simplicity and broad compatibility matter more.
Message-Based Communication
Both REST and RPC are synchronous — the client sends a request and waits for a response. Message-based communication (also called async messaging) breaks this pattern. A client publishes a message to a queue or message broker (like Kafka or RabbitMQ) and immediately continues doing other things. The server (or multiple servers) reads and processes that message whenever it's ready.
SYNCHRONOUS (REST/RPC):
Client ----request----> Server
Client <---response---- Server
(Client is blocked, waiting the whole time)
ASYNCHRONOUS (Message-Based):
Client ----message----> Queue <----reads---- Server
(Client moves on immediately; server processes later)
Message-based systems excel when work is slow (like sending emails or processing video), when you need to decouple producers from consumers, or when you want to smooth out traffic spikes by buffering work.
⚠️ Common Mistake: Candidates often propose REST for everything. In an interview, recognizing when to suggest async messaging (long-running jobs, event-driven architectures, high-throughput pipelines) signals strong architectural judgment.
Horizontal vs. Vertical Scaling: A First Look
No discussion of servers is complete without touching on how we handle more load. There are two fundamental strategies, and you need to understand the vocabulary even if the deep mechanics come later.
Vertical scaling ("scaling up") means making a single server more powerful — adding more CPU, RAM, or faster storage. It's the simplest approach and requires no architectural changes, but it has a hard ceiling: there's only so big a single machine can get, and the cost grows non-linearly.
Horizontal scaling ("scaling out") means adding more servers and distributing load across them. It's more complex — now you need load balancers, stateless designs, and strategies for data consistency — but it's how every large-scale system achieves massive capacity.
VERTICAL SCALING (one big server):
[Small Server] → [Medium Server] → [Big Server] → ??? (limit reached)
HORIZONTAL SCALING (many servers):
[Server 1]
[Server 2] ← Load Balancer ← Clients
[Server 3]
[Server N...]
🎯 Key Principle: Most production systems use a combination of both. You vertically scale until it becomes cost-prohibitive, then you horizontally scale. The design decisions you make early — especially around statefulness — determine how easy horizontal scaling will be later.
We'll return to scaling in depth in the Scalability lesson. For now, lock in the vocabulary: scale up = vertical, scale out = horizontal.
🧠 Mnemonic: "Up is one, Out is many." Scale UP = one bigger machine. Scale OUT = many machines spreading out.
Essential Vocabulary: Five Terms Every Designer Must Know
Let's crystallize five terms that will appear constantly in system design conversations. Interviewers use these casually, and fluency with them signals that you're speaking the same language.
Node
A node is any single computing unit in a distributed system — a server, a virtual machine, a container, or even a process. When you hear "a cluster of 50 nodes," it means 50 individual computing units working together.
💡 Mental Model: Think of nodes as individual workers in a warehouse. Each worker is a node. The warehouse (the system) is the sum of all the workers working in coordination.
Service
A service is a self-contained unit of functionality that exposes its capabilities via an API. In a modern microservices architecture, you might have a User Service, a Payment Service, and a Notification Service — each running independently, each with its own API.
🔧 Practical note: "Service" and "server" are related but not identical. A single physical server might run multiple services. A single service might run across dozens of servers.
Endpoint
An endpoint is a specific URL (or address) at which a service can be reached to perform a specific function. It's the precise "door" you knock on to trigger a specific behavior.
API Base URL: https://api.yourapp.com
Endpoints:
/users ← endpoint for user operations
/users/42 ← endpoint for a specific user
/users/42/orders ← endpoint for a user's orders
/payments/process ← endpoint for payment processing
Payload
The payload is the actual data being transmitted in a request or response — everything beyond the routing and metadata. When you POST a new user to an API, the JSON body containing the user's name and email is the payload. When the server responds with user details, that JSON object is the payload.
⚠️ Common Mistake: Don't confuse the payload with the headers. HTTP headers carry metadata (content type, authentication tokens, caching directives). The payload is the content itself — the message in the envelope, not the envelope.
Protocol
A protocol is a set of rules governing how two parties communicate. It defines the format of messages, the order of operations, and how errors are handled. HTTP, TCP, WebSocket, and gRPC are all protocols — they're the agreed-upon languages that clients and servers use to understand each other.
💡 Mental Model: A protocol is like the rules of a formal business meeting. There are rules about who speaks when, what format reports must take, and how disagreements are resolved. Without those rules, the meeting devolves into chaos. Without a protocol, computers can't communicate reliably.
📋 Quick Reference Card:
| 🔧 Term | 📚 Definition | 🎯 Example |
|---|---|---|
| 🖥️ Node | A single computing unit in a system | One server in a 10-server cluster |
| ⚙️ Service | A self-contained unit of functionality | A Payment Service or Auth Service |
| 🚪 Endpoint | A specific URL/address for one function | POST /api/v1/orders |
| 📦 Payload | The actual data in a request or response | {"name": "Alice", "email": "..." } |
| 📜 Protocol | Rules governing communication between parties | HTTP, TCP, WebSocket, gRPC |
Putting It All Together
These concepts don't exist in isolation — they interlock. A client (node) sends an HTTP request (protocol) to a specific endpoint of a service (running on another node). The request contains a payload. The service, designed to be stateless, processes the request without needing memory of prior interactions, sends back a response payload, and any server in its horizontal pool could have handled it just as well.
That one paragraph describes the architecture of most modern web APIs. The vocabulary you've just built is the lens through which every system design conversation is conducted. When an interviewer asks "how would you design Instagram's feed service?", they want you to speak fluently in exactly these terms — clients, servers, stateless services, endpoints, protocols, and payloads — before diving into the sophisticated patterns built on top of them.
💡 Pro Tip: In system design interviews, explicitly naming these components as you diagram a system — "here, the mobile client sends a REST request to this endpoint of our User Service" — demonstrates clarity of thought and communication precision. Interviewers notice the difference between candidates who gesture vaguely at boxes and arrows versus those who label every component with intention.
With this vocabulary locked in, you're ready to move into the language of data — how systems store, retrieve, and manage information at scale. That's exactly where Section 3 takes us.
Data, Storage, and the Language of System Internals
Every system you will ever design has one fundamental challenge at its core: data needs to go somewhere, stay there reliably, be retrieved quickly, and sometimes travel between services. The vocabulary you build in this section is not just theoretical — it is the precise language engineers use when sketching systems on whiteboards, debating architecture choices, and, crucially, impressing interviewers. When you can say "we should partition this table horizontally and cache reads at the application layer to reduce latency" without hesitation, you signal deep fluency. Let's build that fluency.
Databases Demystified: Relational vs. Non-Relational
At the most fundamental level, a database is an organized collection of data that can be stored, retrieved, and modified efficiently. But the word "database" covers an enormous spectrum of technologies, and in system design interviews, the first vocabulary decision you must make is whether to use a relational database or a non-relational database (also called a NoSQL database).
A relational database (RDBMS) organizes data into tables — structured grids of rows and columns — and uses SQL (Structured Query Language) to interact with that data. The defining characteristic is that relationships between tables are enforced explicitly: a row in an orders table can reference a row in a users table via a foreign key. Examples include PostgreSQL, MySQL, and SQLite. These systems excel when your data is highly structured, when relationships between entities matter deeply, and when you need strong consistency guarantees.
A non-relational database takes a different approach by abandoning the rigid table structure in favor of flexibility. This family includes:
- 🧠 Document stores (like MongoDB): store data as JSON-like documents, ideal for content with variable attributes
- 📚 Key-value stores (like Redis, DynamoDB): store data as simple key → value pairs, optimized for blazing-fast lookups
- 🔧 Column-family stores (like Cassandra): organize data by columns rather than rows, built for write-heavy, large-scale analytics
- 🎯 Graph databases (like Neo4j): model data as nodes and edges, perfect for social networks and recommendation engines
💡 Real-World Example: Consider designing Instagram. User profiles have fixed fields like username, email, and bio — a relational table works perfectly. But each user's photo metadata might have wildly different EXIF data, tags, filters, and location fields. A document store is more natural here because the schema varies per document.
🎯 Key Principle: The choice between relational and non-relational is not about which is "better" — it is about matching the data model to the problem. Most large-scale systems use both.
Relational (SQL) Non-Relational (NoSQL)
───────────────────── ──────────────────────────
| users table | { _id: "u001",
|─────────────────| name: "Alice",
| id | name | age | age: 30,
|────|──────|─────| posts: ["p1","p2"],
| 1 | Alice| 30 | preferences: {
| 2 | Bob | 25 | theme: "dark"
└─────────────────┘ }
}
Strict schema, Flexible schema,
JOINs between tables nested/embedded data
Caching: The Art of Remembering the Answer
Caching is one of the most powerful and frequently discussed concepts in system design. The core idea is beautifully simple: if fetching a piece of data is expensive (slow, CPU-intensive, or costly), store a copy of that data in a faster location so future requests can be served without repeating the expensive work.
Think of caching like a chef's mise en place — instead of fetching ingredients from the storage room for every dish, common ingredients sit right at the workstation, ready to grab instantly.
The vocabulary around caching is precise and interviewers listen for it:
- A cache hit occurs when the requested data is found in the cache. The system returns the cached value immediately without touching the underlying database. Fast. Cheap. Good.
- A cache miss occurs when the requested data is NOT in the cache. The system must fetch it from the original source, return it to the requester, and typically store it in the cache for next time.
- Cache eviction is the process of removing data from the cache to make room for new data. Because caches have limited memory, they can't store everything forever.
Eviction Policies
This is where interviews often go deeper. An eviction policy determines which data gets removed when the cache is full. The most common policies you must know:
- 🔒 LRU (Least Recently Used): Evicts the data that hasn't been accessed for the longest time. The assumption is that recently accessed data will be accessed again soon. Used in most general-purpose caches.
- 📚 LFU (Least Frequently Used): Evicts the data that has been accessed the fewest total times. Better for workloads where some data is perennially popular.
- 🧠 TTL (Time To Live): Data expires after a set duration, regardless of usage. Every cache entry has a timestamp; expired entries are evicted. Common in web caching (think browser caches and CDNs).
⚠️ Common Mistake: Assuming caching always helps. If your data changes frequently and you cache it aggressively, users may see stale data — outdated information that hasn't been refreshed. Cache invalidation (deciding when to remove or update a cache entry) is famously one of the hardest problems in computer science.
Request → Check Cache
│
┌─────┴──────┐
│ │
Cache HIT Cache MISS
│ │
Return cached Fetch from DB
value ✅ │
Store in cache
│
Return value
💡 Mental Model: Think of caching as a sticky note. The first time someone asks you a complex question, you research the answer (expensive). You write the answer on a sticky note (caching). The next time someone asks, you read the sticky note instantly (cache hit). Eventually, the sticky note might become outdated — that's when you need to invalidate it and research again.
Replication and Partitioning: Making Data Scale
When a single database server can't handle all your traffic or all your data, you need strategies for distributing the load. Two fundamental strategies are replication and partitioning, and they solve different problems.
Replication means keeping multiple copies of the same data across different servers. The server that accepts writes is called the primary (or master), and the servers that hold copies are called replicas (or secondaries). Reads can be served from replicas, spreading the load. If the primary fails, a replica can be promoted — this is the basis of high availability.
┌──────────────┐
│ Primary DB │ ← All WRITES go here
└──────┬───────┘
│ replicates data
┌──────┴───────────────┐
│ │
┌──────▼──────┐ ┌──────▼──────┐
│ Replica 1 │ │ Replica 2 │
│ (reads) │ │ (reads) │
└─────────────┘ └─────────────┘
Partitioning (also called sharding) divides the data itself across multiple servers. Instead of every server having the full dataset, each server owns a shard — a subset of the data. For example, users with IDs 1–1,000,000 live on shard A, users with IDs 1,000,001–2,000,000 live on shard B, and so on. This is horizontal partitioning. Vertical partitioning means splitting a table by columns — putting frequently accessed columns on one server and archival columns on another.
Horizontal Partitioning (Sharding by user ID):
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Shard A │ │ Shard B │ │ Shard C │
│ Users 1-1M │ │ Users 1M-2M │ │ Users 2M-3M │
└───────────────┘ └───────────────┘ └───────────────┘
🤔 Did you know? The term "sharding" comes from an old database system called "UO" (Ultima Online), where game data was split across parallel "shards" of a fictional crystal. The gaming metaphor stuck and is now standard infrastructure terminology.
💡 Pro Tip: In interviews, when you suggest sharding, interviewers will often ask "how do you choose your shard key?" The shard key is the attribute used to determine which shard stores a given record. A bad shard key creates hotspots — one shard receiving far more traffic than others — which defeats the purpose entirely.
Message Queues and Event-Driven Architecture
Not all communication in a distributed system needs to be immediate. Sometimes, services need to communicate asynchronously — one service sends a message and moves on without waiting for a response. This is the domain of message queues and event-driven architecture.
A message queue is a buffer that stores messages sent by a producer until a consumer is ready to process them. This decouples the sender from the receiver: the producer doesn't need to know who will handle the message or when. Popular message queue systems include RabbitMQ, Apache Kafka, and AWS SQS.
In an event-driven architecture, components communicate by emitting events (things that happened) rather than making direct requests. A payment service emits a PaymentCompleted event; an email service, an analytics service, and an inventory service each listen for that event and react independently.
Producer/Consumer Pattern:
┌──────────┐ publish ┌──────────────┐ consume ┌──────────┐
│ Producer │ ──────────► │ Message Queue │ ──────────► │ Consumer │
│(Order │ │ [ msg1 ] │ │(Email │
│ Service) │ │ [ msg2 ] │ │ Service) │
└──────────┘ │ [ msg3 ] │ └──────────┘
└──────────────┘
│
└──────────► ┌──────────┐
│ Consumer │
│(Analytics│
│ Service) │
└──────────┘
Here is a Python pseudo-code example illustrating the producer-consumer pattern:
import queue
import threading
import time
## A simple in-memory queue simulating a message broker
message_queue = queue.Queue()
def producer(order_id: int):
"""Simulates an order service placing a message on the queue."""
message = {"event": "OrderPlaced", "order_id": order_id}
print(f"[Producer] Publishing: {message}")
message_queue.put(message) # Non-blocking: producer moves on immediately
def consumer(service_name: str):
"""Simulates a downstream service consuming messages asynchronously."""
while True:
# Block until a message is available, then process it
message = message_queue.get()
print(f"[{service_name}] Processing: {message}")
time.sleep(0.5) # Simulate work (sending email, updating analytics, etc.)
message_queue.task_done()
## Start a consumer running in the background (e.g., Email Service)
consumer_thread = threading.Thread(
target=consumer,
args=("EmailService",),
daemon=True
)
consumer_thread.start()
## Producer publishes several order events without waiting for consumers
for order_id in range(1, 4):
producer(order_id)
time.sleep(0.1) # Orders come in rapidly
message_queue.join() # Wait for all messages to be processed
print("All messages processed.")
This code demonstrates the key decoupling principle: the producer function places messages on the queue and immediately returns. The consumer runs in a separate thread, processing messages at its own pace. If the email service is slow, messages simply queue up — the order service is never blocked.
## More realistic: multiple consumers for parallel processing
import queue
import threading
task_queue = queue.Queue()
def worker(worker_id: int, task_queue: queue.Queue):
"""Each worker pulls tasks independently — horizontal scaling of consumers."""
while True:
task = task_queue.get()
if task is None: # Sentinel value to signal shutdown
break
print(f"[Worker {worker_id}] Handling task: {task['event']}")
task_queue.task_done()
## Spawn 3 concurrent consumers — this is how real queues scale throughput
workers = []
for i in range(3):
t = threading.Thread(target=worker, args=(i, task_queue))
t.start()
workers.append(t)
## Publish 9 events to be distributed across workers
for n in range(9):
task_queue.put({"event": f"OrderPlaced", "order_id": n})
## Graceful shutdown: send one None sentinel per worker
for _ in workers:
task_queue.put(None)
for t in workers:
t.join()
This second example shows how adding more consumers is how you scale message processing — a pattern directly analogous to services like Kafka consumer groups in production systems.
⚠️ Common Mistake: Confusing message queues with direct API calls. If Service A calls Service B's API directly, they are tightly coupled — if B is down, A fails. With a message queue, A publishes an event and continues; B processes it when it's available. This is the resilience benefit of async communication.
The Essential Vocabulary: ACID, BASE, and Data Internals
This is where system design vocabulary becomes truly precise. The following terms appear in almost every serious database discussion, and knowing them confidently separates intermediate developers from senior-level candidates.
ACID: The Reliability Promise of Relational Databases
ACID is an acronym describing four properties that guarantee reliable database transactions:
🧠 Mnemonic: Think of ACID as a promise a bank makes when transferring money — if anything goes wrong, nothing changes.
| Property | Definition | Bank Transfer Analogy |
|---|---|---|
| 🔒 Atomicity | A transaction either fully completes or fully rolls back — no partial states | Either both accounts update, or neither does |
| 🎯 Consistency | A transaction moves the database from one valid state to another valid state | Account balances never go negative |
| 🔧 Isolation | Concurrent transactions don't interfere with each other | Two simultaneous transfers don't create phantom balances |
| 📚 Durability | Once committed, a transaction persists even after a system crash | A completed transfer survives a server reboot |
Durability specifically means that committed data is written to non-volatile storage (disk). This is distinct from persistence, which is the broader concept that data survives beyond the process that created it.
BASE: The Pragmatic Trade-Off of Distributed Systems
When you distribute a database across multiple servers, maintaining strict ACID guarantees becomes extremely expensive in terms of latency. Many NoSQL systems instead follow BASE properties:
- 🧠 Basically Available: The system guarantees availability, though not necessarily the most current data
- 🔧 Soft State: The state of the system may change over time, even without input, as data propagates
- 📚 Eventually Consistent: Given enough time without new updates, all replicas will converge to the same value
❌ Wrong thinking: "BASE databases are broken because they're inconsistent." ✅ Correct thinking: "BASE databases trade immediate consistency for availability and partition tolerance — a deliberate, well-reasoned engineering choice for certain workloads."
💡 Real-World Example: When you like a post on social media, your friend in another country might not see that like for a fraction of a second. The system is eventually consistent — the data will propagate, but perfect instant consistency across all global replicas would be too slow and expensive.
Indexes: The Table of Contents for Your Data
An index is a data structure that improves the speed of data retrieval at the cost of additional storage and write overhead. Without an index, finding a user by email in a table of 100 million rows requires scanning every single row — a full table scan. With an index on the email column, the database can find the row in milliseconds.
Think of an index exactly like a book's index: instead of reading every page to find where "sharding" appears, you look it up alphabetically in the index and jump directly to page 247.
⚠️ Common Mistake: Over-indexing. Every index speeds up reads but slows down writes (because the index must also be updated on every insert/update/delete). Adding indexes to every column is an anti-pattern that cripples write-heavy systems.
Schema: The Blueprint
A schema is the formal definition of a database's structure — the tables, their columns, the data types of each column, and the constraints (like foreign keys and unique constraints). In relational databases, the schema is strict: every row in a table must conform to it. In many NoSQL databases, the schema is flexible or non-existent (schema-less), allowing each document to have different fields.
📋 Quick Reference Card: Core Data Vocabulary
| Term | One-Line Definition | Remember It As |
|---|---|---|
| 🔒 ACID | Reliability guarantees for transactions | The bank's promise |
| 📚 BASE | Availability-first consistency model | The distributed trade-off |
| 🎯 Index | Lookup structure for fast retrieval | Book's index |
| 🔧 Schema | Structure definition for data | Blueprint of a table |
| 🧠 Persistence | Data survives process termination | Outlives the program |
| 📚 Durability | Committed data survives crashes | Survives power outage |
| 🔒 Replication | Copies of data across servers | Mirror for reliability |
| 🎯 Sharding | Splitting data across servers | Divide to conquer |
| 🔧 Cache Hit | Data found in fast storage | Lucky draw |
| 🧠 Eviction | Removing old data from cache | Clearing the sticky notes |
Putting It All Together
These concepts don't exist in isolation. In a real system design conversation, you might say: "For user profiles, I'd use a relational database with an index on the email column for fast lookups. To reduce read latency, I'll add a Redis cache with an LRU eviction policy and a 5-minute TTL. When a user places an order, the order service publishes an event to a message queue, allowing the notification service to consume it asynchronously — this keeps our services loosely coupled. Since we'll have millions of users, we'll shard the orders table horizontally by user ID, and use read replicas to scale query throughput."
Every italicized concept in that paragraph is terminology from this section. That fluency is what transforms a junior engineer's vague description of "storing stuff" into a senior engineer's precise architectural narrative. In the next section, you'll see exactly this kind of vocabulary put to work in a real system design walkthrough — designing a URL shortener from scratch.
Reading and Writing System Design: Patterns in Practice
Knowing individual terms is useful. Knowing how to use them together — fluidly, accurately, and confidently — is what separates candidates who get offers from those who don't. In this section, we're going to slow down and walk through a single, classic system design problem — the URL shortener — and label every decision with the vocabulary you've been building. Think of it as annotated engineering: we design the system and narrate it simultaneously, so you can see exactly how terminology appears in context.
By the end of this section, you'll have a template for how real system design conversations flow, a code-level anchor for abstract storage concepts, and the language to discuss difficult trade-offs without sounding like you're guessing.
Why the URL Shortener Is the Perfect Teaching Example
The URL shortener — think bit.ly or tinyurl.com — is a staple of system design interviews for good reason. It's simple enough that you can hold the whole thing in your head, but complex enough to touch nearly every core concept: read/write patterns, storage decisions, caching, scalability, availability, and fault tolerance. It's also a system most developers have used, which makes it easier to reason about intuitively.
Here's the core behavior we're designing for:
- A user submits a long URL (e.g.,
https://example.com/some/very/long/path?with=params) - The system returns a short code (e.g.,
https://short.ly/aB3x9) - When anyone visits the short URL, they are redirected to the original long URL
Simple to describe. Full of interesting engineering decisions.
The Annotated Design: A Walkthrough
Let's build this system piece by piece, naming every component as we introduce it.
Client (Browser)
|
v
[ Load Balancer ] <-- distributes traffic
|
v
[ Application Servers ] <-- handle business logic
/ \
v v
[Cache] [Database]
(Redis) (PostgreSQL)
This is our starting architecture. It looks deceptively simple — five boxes and some arrows — but every connection in this diagram represents a deliberate decision. Let's walk through each one.
The Client Makes a Request
A client is any entity that initiates a request. In this case, it's a browser or a mobile app. The client sends an HTTP POST request with the long URL in the request body when creating a short link, and an HTTP GET request when visiting a short link.
The client doesn't know — and shouldn't care — which specific server handles its request. That's the job of the next layer.
The Load Balancer Distributes Traffic
Before the request reaches any application logic, it passes through a load balancer. The load balancer's job is to distribute incoming requests across multiple servers so that no single server becomes overwhelmed. In an interview, you'd say something like:
"We'd put a load balancer in front of our application tier. This lets us scale horizontally by adding more servers as traffic grows. It also gives us redundancy — if one application server goes down, the load balancer routes traffic to the healthy ones."
Notice two terms appear naturally: horizontal scaling (adding more servers rather than making one server bigger) and redundancy (having backup capacity so the system keeps running when a component fails).
💡 Pro Tip: In interviews, you don't need to specify the exact load balancing algorithm (round-robin, least connections, etc.) unless the interviewer asks. Mentioning that you'd use one and why is enough to start.
The Application Servers Handle Logic
Behind the load balancer sit multiple application servers. These are stateless — they don't store any user session data locally. Every piece of persistent information lives in the database or cache layer. Statelessness is what makes horizontal scaling possible: because no server holds unique state, you can add or remove servers freely without affecting correctness.
🎯 Key Principle: Stateless application servers are a prerequisite for effective horizontal scaling. If a server holds local state, you can't freely route requests to any server in the pool.
The Database Stores Persistent Data
When a user creates a short link, we need to permanently store the mapping between the short code and the long URL. This is the job of the database. For a URL shortener, a simple relational schema works well:
-- Creating the core table in a relational database
CREATE TABLE url_mappings (
short_code VARCHAR(10) PRIMARY KEY, -- the key, e.g. "aB3x9"
long_url TEXT NOT NULL, -- the full destination URL
created_at TIMESTAMP DEFAULT NOW(), -- when it was created
access_count BIGINT DEFAULT 0 -- optional: for analytics
);
-- Writing a new mapping (happens when a user creates a short link)
INSERT INTO url_mappings (short_code, long_url)
VALUES ('aB3x9', 'https://example.com/some/very/long/path?with=params');
-- Reading a mapping (happens on every redirect)
SELECT long_url FROM url_mappings WHERE short_code = 'aB3x9';
This table is our source of truth — the authoritative record of all URL mappings. Reads from this table happen every time someone visits a short link, which in a popular system could be millions of times per day.
🤔 Did you know? URL shorteners are extremely read-heavy systems. For every person who creates a short link, potentially thousands of people click it. This read/write asymmetry has a huge influence on the architecture.
The Cache Accelerates Reads
Hitting the database on every single redirect would be slow and expensive. Instead, we introduce a cache — a fast, in-memory data store that holds frequently accessed data. Redis is a common choice. Here's how the lookup pattern works in code:
import redis
import psycopg2
## Connect to Redis (cache) and PostgreSQL (database)
cache = redis.Redis(host='cache-host', port=6379, decode_responses=True)
db = psycopg2.connect("dbname=urldb user=app password=secret host=db-host")
def get_long_url(short_code: str) -> str | None:
"""
Look up a short code and return the original long URL.
Uses a cache-aside pattern: check cache first, fall back to DB.
"""
# Step 1: Check the cache first (fast path)
cached_url = cache.get(f"url:{short_code}")
if cached_url:
return cached_url # Cache hit — return immediately
# Step 2: Cache miss — query the database (slower path)
cursor = db.cursor()
cursor.execute(
"SELECT long_url FROM url_mappings WHERE short_code = %s",
(short_code,)
)
result = cursor.fetchone()
if result is None:
return None # Short code doesn't exist
long_url = result[0]
# Step 3: Populate the cache so future lookups are fast
# TTL of 3600 seconds (1 hour) — cache entry expires automatically
cache.setex(f"url:{short_code}", 3600, long_url)
return long_url
This is the cache-aside pattern (also called lazy loading). The application checks the cache first; on a cache miss, it queries the database and then writes the result into the cache for future requests. The cache key url:{short_code} follows a common namespacing convention to avoid collisions.
In an interview, you'd narrate this as:
"Reads will be far more frequent than writes, so we'll add a Redis cache in front of the database. We'll use a cache-aside pattern — check the cache first, fall back to the database on a miss, and then populate the cache so subsequent requests are fast. The TTL — time-to-live — on cache entries ensures stale data eventually expires."
Talking Through Trade-Offs: The Language of System Design
A strong system design candidate doesn't just describe what the system does — they explain why each decision was made and what was sacrificed to get there. This is trade-off language, and it requires vocabulary precision.
Let's apply it to three specific decisions in our URL shortener.
Trade-Off 1: SQL vs. NoSQL for Storage
We chose a relational database (PostgreSQL). Why not a key-value store like DynamoDB?
"For a URL shortener, our data model is extremely simple — it's essentially a key-value mapping. A NoSQL key-value store would offer better write throughput and simpler horizontal scaling. However, we chose a relational database initially because it gives us stronger consistency guarantees and is easier to reason about. If we needed to scale writes dramatically, we'd revisit this — potentially sharding the database or migrating to a NoSQL store."
Consistency here means that when a short link is created, any subsequent read anywhere in the system will see that new link — there's no window where the data is in an ambiguous state. Some NoSQL systems offer eventual consistency instead, where data changes propagate across nodes over time rather than immediately.
⚠️ Common Mistake: Saying "NoSQL is faster than SQL" as a blanket statement. The truth is nuanced. NoSQL databases trade certain consistency guarantees and query flexibility for scalability and speed in specific access patterns. Always frame it in terms of the specific use case.
Trade-Off 2: Cache Staleness vs. Freshness
Our cache has a TTL of one hour. What if a user updates a short link's destination during that hour?
"Our cache introduces a potential staleness window. If someone updates a short URL's destination, cached entries will still point to the old URL for up to an hour. We have two options: lower the TTL (fresher data, more database pressure) or implement cache invalidation — explicitly deleting the cache entry when the underlying data changes. Cache invalidation is more complex but gives us immediate consistency. For a URL shortener, since updates are rare, a moderate TTL is probably acceptable."
This is availability versus consistency framed at the cache layer. A longer TTL makes the cache more available (fewer cache misses, less database load) at the cost of occasionally serving stale data.
💡 Mental Model: Think of a cache TTL like a newspaper. Yesterday's newspaper is still mostly accurate, but a few things have changed. How often you need a fresh edition depends on how often the news changes and how much it matters if a reader gets outdated information.
Trade-Off 3: Short Code Generation
How do we generate unique short codes? We could use a random string, a hash of the long URL, or a sequential ID encoded in base62.
"Random generation is simple but requires checking for collisions — we might generate a code that's already taken. Hashing the URL is deterministic, meaning the same long URL always gets the same short code, which could be useful or problematic depending on requirements. Sequential IDs encoded in base62 are guaranteed unique and easy to implement, but they reveal information about how many links have been created. The right choice depends on our privacy and correctness requirements."
Here's a quick illustration of base62 encoding:
import string
## Base62 character set: 0-9, a-z, A-Z
BASE62_CHARS = string.digits + string.ascii_lowercase + string.ascii_uppercase
def encode_base62(num: int) -> str:
"""
Convert an integer (e.g., a database auto-increment ID)
into a short base62 string suitable for use as a URL code.
Example: encode_base62(125000) -> 'Wbm'
"""
if num == 0:
return BASE62_CHARS[0]
result = []
while num > 0:
result.append(BASE62_CHARS[num % 62])
num //= 62
return ''.join(reversed(result))
## Example usage
print(encode_base62(1)) # '1'
print(encode_base62(62)) # 'a' (62 in base62 is '10', not shown here for brevity)
print(encode_base62(125000)) # A short, unique alphanumeric code
In an interview, proposing this approach demonstrates that you understand the relationship between storage (the database ID), encoding (base62), and the resulting user-facing interface (the short code). It shows you're thinking in systems.
Recognizing Design Patterns by Name
Beyond individual components, experienced engineers recognize recurring patterns — structural solutions that appear across many different systems. Naming these patterns in an interview signals architectural fluency.
Single Point of Failure
A single point of failure (SPOF) is any component whose failure would bring down the entire system. In our initial diagram, if we only had one application server and it crashed, all users would see errors. The load balancer itself, if there's only one, is also a SPOF.
❌ Fragile Architecture:
Client --> [ Single App Server ] --> Database
^
|
SPOF: if this crashes, everything breaks
✅ Resilient Architecture:
Client --> [ Load Balancer (Primary) ]
|
[ Load Balancer (Standby) ] <-- failover
/ | \
[Server 1] [Server 2] [Server 3]
|
[ Database ]
(with replica)
In an interview:
"We need to identify and eliminate single points of failure. Our load balancer is currently a SPOF — we'd want a standby instance ready to take over if the primary fails. This is failover — automatic switching to a backup when the primary becomes unavailable."
Redundancy
Redundancy is the practice of duplicating critical components so the system continues functioning when one copy fails. Our multiple application servers are redundant. A database replica is a redundant copy of the database that can serve reads or be promoted to primary if the original fails.
🎯 Key Principle: Redundancy costs money but buys fault tolerance — the system's ability to keep operating correctly even when components fail. The question isn't whether to have redundancy but how much fault tolerance your system requires versus how much you're willing to spend.
Separation of Concerns
Separation of concerns is the design principle that different responsibilities should be handled by different components. In our URL shortener, the application server handles business logic, the cache handles fast reads, and the database handles persistence. None of these layers bleeds into the others.
❌ Wrong thinking: "I'll just have the application server cache things in memory locally — it's simpler."
✅ Correct thinking: "Local in-memory caching couples the cache to specific server instances, making it incompatible with horizontal scaling and creating inconsistency between servers. A separate caching layer maintains separation of concerns."
🧠 Mnemonic: "SPoF, ReDundancy, SoC" — think Something Properly Fixed Requires Duplicate Stuff or Crashes. Silly, but the three patterns will stick: SPOF, Redundancy, SoC.
Putting It All Together: Interview Language in Action
Let's see what a fluent, vocabulary-rich system design response sounds like when you combine everything from this walkthrough. Here's how a strong candidate might open their URL shortener design:
"For a URL shortener, I'd start with a three-tier architecture: a load balancer distributing traffic across stateless application servers, backed by a relational database for persistent storage and a Redis cache for read acceleration. Since this is a read-heavy system — potentially 100:1 read/write ratio — cache hit rate will be critical to keep latency low. I'd use a cache-aside pattern with a TTL of about an hour for most entries.
For the database, I'd lean toward a single primary with read replicas initially. Writes go to the primary; reads can fan out to replicas. This eliminates the database as a single point of failure for reads and gives us fault tolerance through redundancy. If write throughput becomes a bottleneck down the line, we could look at sharding the database by short code prefix.
Short codes would be generated by encoding the database's auto-increment ID in base62, giving us collision-free uniqueness without extra lookups. I'd want to think through a few trade-offs here: this approach leaks the approximate number of links created, which might matter for business confidentiality. An alternative is random generation with a collision check, which is private but adds a database read on every write..."
Count the terms: three-tier architecture, stateless, load balancer, read-heavy, cache-aside, TTL, primary with read replicas, single point of failure, fault tolerance, redundancy, sharding, base62, trade-offs. Every term is used accurately and in natural, flowing prose. That's the target.
📋 Quick Reference Card: URL Shortener Components and Their Vocabulary
| 🔧 Component | 📚 Key Terms | 🎯 What to Say in Interviews |
|---|---|---|
| 🌐 Load Balancer | horizontal scaling, redundancy, failover | "Distributes traffic, enables scaling, eliminates SPOF" |
| ⚙️ App Servers | stateless, separation of concerns | "Stateless so any server can handle any request" |
| 🗄️ Database | consistency, persistence, replicas, sharding | "Source of truth; replicas for fault tolerance" |
| ⚡ Cache | TTL, cache-aside, cache miss, staleness | "Reduces DB load; trade-off is potential staleness" |
| 🔢 Short Code Gen | throughput, collision, base62 | "Trade-off between uniqueness guarantees and privacy" |
From Annotated Diagram to Interview Fluency
The URL shortener walkthrough does something important: it shows you that system design vocabulary isn't just a list of words to memorize. Each term is tied to a decision, a reason, and a trade-off. When you name a load balancer, you're also invoking the concepts of horizontal scaling and eliminating single points of failure. When you mention a cache, you're implicitly raising questions about TTL, staleness, and the cache-aside pattern.
This is why building vocabulary in context — through examples like this — is far more effective than memorizing definitions in isolation. The terms don't exist independently; they're part of a web of interconnected concepts that describe how real systems are built and why.
In the next section, we'll look at the terminology mistakes that trip up even experienced candidates — because knowing the right terms is only half the battle. Knowing what not to say, and what common misconceptions to avoid, is equally important.
Common Terminology Mistakes That Cost Candidates the Job
You have prepared for weeks. You understand distributed systems conceptually, you have practiced on whiteboards, and you feel ready. Then the interviewer asks you to design a system and something subtle goes wrong — not in your architecture, but in your language. You say "just add more servers" when you mean horizontal scaling. You say "use a database" without specifying what kind. You say "caching will fix the latency" without mentioning invalidation. None of these mistakes feel catastrophic in the moment, but together they signal to an interviewer that your mental models are fuzzy. That fuzziness is what costs candidates the job.
This section is a targeted intervention. We will walk through the five most consequential vocabulary and conceptual errors that appear in system design interviews, explain precisely why each one matters, and give you concrete tools to correct them before they become habits.
Mistake 1: Confusing Scalability with Performance ⚠️
This is the single most common conflation in system design interviews, and it is understandable why it happens — the two concepts are related, they both affect user experience, and improving one often improves the other. But they are not the same thing, and treating them as synonyms will immediately raise a red flag for experienced interviewers.
Performance refers to how fast your system responds to a single request under normal conditions. It is about latency, throughput for a given workload, and efficiency. If your API endpoint takes 800ms to return a response, that is a performance problem. Optimizing a database query, adding an index, or moving computation closer to the user are all performance improvements.
Scalability, on the other hand, refers to your system's ability to handle growing amounts of work — more users, more data, more requests — without degrading. A system can be performant at low load but collapse under high load. That collapse is a scalability failure, not a performance failure.
PERFORMANCE vs. SCALABILITY
Single Request View (Performance)
┌────────────────────────────────────┐
│ Request ──► System ──► Response │
│ 800ms │
│ ← this is what performance │
│ measures ───────────────────► │
└────────────────────────────────────┘
Load Growth View (Scalability)
┌───────────────────────────────────────────────┐
│ 10 users/sec → Response: 100ms ✅ │
│ 100 users/sec → Response: 120ms ✅ │
│ 1000 users/sec → Response: 4500ms ❌ │
│ │
│ ← the degradation curve is scalability ──► │
└───────────────────────────────────────────────┘
A critical nuance: you can have a highly scalable system that performs poorly. Imagine a horizontally scaled architecture that adds nodes easily but every request requires three slow database round-trips. It scales, but it performs badly. The inverse is also true — a finely tuned single-server system might be blazingly fast up to a point, then fall over entirely.
❌ Wrong thinking: "Our system is slow, so we need to scale it." ✅ Correct thinking: "Our system is slow at low load — that is a performance problem, likely in the query path. Scaling would not help until we fix the underlying bottleneck."
💡 Pro Tip: When an interviewer says "the system needs to handle more traffic," pause and clarify. Ask whether the concern is latency at peak load (which may be a performance bottleneck that scaling will not fix) or throughput capacity (which horizontal scaling directly addresses). This single question signals architectural maturity.
🎯 Key Principle: Performance is about the speed of one thing. Scalability is about what happens when you have many things. Optimize performance first, then design for scalability — otherwise you scale your inefficiencies.
Mistake 2: Using 'Database' as a Monolithic Term ⚠️
When a candidate says "store it in the database," an experienced interviewer immediately wants to know: which kind? The word "database" in modern system design is not a single thing — it is a category containing dozens of profoundly different tools with different consistency guarantees, query patterns, and trade-offs. Using the word without qualification is like saying "use a vehicle" when the choice between a bicycle and a cargo plane matters enormously.
Here are the major categories you should be able to distinguish and deploy correctly in conversation:
Type |
🔧 Best For | ⚠️ Trade-off |
|---|---|---|
| 🗃️ Relational (SQL) | Structured data, ACID transactions | Harder to scale horizontally |
| 📄 Document Store | Flexible schemas, JSON-like objects | Weaker joins, eventual consistency |
| 🔑 Key-Value Store | High-speed lookups, session data | No complex queries |
| 📊 Column-Family | Time-series, wide rows, analytics | Complex data modeling |
| 🕸️ Graph DB | Relationship-heavy data, social graphs | Niche use case, steep learning curve |
| 🔍 Search Engine | Full-text search, faceted queries | Not a primary data store |
The interview mistake is not just using the wrong word — it is failing to justify your choice. When you say "I would use PostgreSQL here," you should be able to follow it with: "because we need ACID transactions for financial records, and the data has a well-defined relational structure." When you say "I would use Redis," you should be able to add: "because we need sub-millisecond lookups for session tokens and the data fits comfortably in memory."
Here is how this distinction plays out in code. Consider storing user session data. Compare the two approaches and notice how different the access patterns are:
## ❌ Vague: "store it in the database"
## This forces a SQL query for every request check — expensive
import psycopg2
def get_session(session_id: str) -> dict:
conn = psycopg2.connect("dbname=app user=postgres")
cursor = conn.cursor()
# Full SQL round-trip for a simple token lookup
cursor.execute(
"SELECT user_id, expires_at FROM sessions WHERE token = %s",
(session_id,)
)
row = cursor.fetchone()
return {"user_id": row[0], "expires_at": row[1]} if row else None
## ✅ Specific: "store sessions in a key-value store like Redis"
## O(1) lookup, TTL built-in, no SQL overhead
import redis
import json
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
def get_session(session_id: str) -> dict:
# Direct O(1) hash lookup with automatic TTL expiry
data = r.get(f"session:{session_id}")
return json.loads(data) if data else None
def store_session(session_id: str, user_id: int, ttl_seconds: int = 3600):
payload = json.dumps({"user_id": user_id})
# SETEX atomically sets value AND expiry — perfect for sessions
r.setex(f"session:{session_id}", ttl_seconds, payload)
The second version is not just faster — it also demonstrates that you understand why the tool fits the problem. That understanding is exactly what interviewers are evaluating.
💡 Mental Model: When you hear yourself say the word "database," treat it as a placeholder that must be replaced. Ask: What is the read/write ratio? Does order matter? Do I need transactions? Do I need to query by multiple fields? Each answer narrows the field of appropriate tools.
🧠 Mnemonic: SPARK — Structure (SQL for structured), Patterns (query patterns drive choice), Access speed (key-value for speed), Relationships (graph for relationships), Keep it justified (always explain why).
Mistake 3: Overloading the Word 'Server' ⚠️
In early programming education, a "server" is often taught as a single machine that responds to requests. In system design interviews, this simplification becomes a liability. Modern architectures decompose what a layperson calls "the server" into multiple specialized components, and using the generic term signals that you are not thinking at the right level of resolution.
Here are the distinct components that candidates collapse into the word "server":
- Web server: Handles HTTP/HTTPS, serves static assets, terminates SSL. Examples: Nginx, Apache.
- Application server: Runs business logic, processes requests, talks to databases. Examples: a Node.js process, a Django app, a Spring Boot service.
- Load balancer: Distributes incoming requests across multiple application servers. Examples: Nginx (also doubles as web server), HAProxy, AWS ALB.
- Database server: Hosts the database engine and manages data persistence.
- Cache server: Holds frequently accessed data in memory to reduce downstream load.
- Message broker / queue server: Buffers and routes asynchronous messages between services.
The Anatomy of "The Server" (what candidates should say vs. what they do say)
What candidates say:
┌─────────────────┐
│ Client │──────────────► "The Server"
└─────────────────┘
What they should mean:
┌─────────────┐ ┌──────────────┐ ┌───────────────┐
│ Client │────►│ Load Balancer│────►│ App Server 1 │──► DB Server
└─────────────┘ └──────────────┘ ┌─►│ App Server 2 │──► Cache Server
│ │ App Server 3 │
│ └───────────────┘
│ │
└─────────┴── (health checks)
Why does this matter in an interview? Because the interviewer is evaluating whether you can reason about failure domains, bottlenecks, and scaling axes independently. A load balancer fails differently than an application server. You scale them differently. You monitor them differently. When you say "server," you collapse all of that into a single undifferentiated blob.
❌ Wrong thinking: "If we get more traffic, we add more servers." ✅ Correct thinking: "If we get more traffic, we can horizontally scale the application servers behind the load balancer. If the load balancer itself becomes a bottleneck, we can use DNS-based load balancing or an Anycast setup to distribute at the network layer."
💡 Real-World Example: Netflix's architecture separates its API gateway (handles auth, rate limiting, routing), microservice application servers (each owning a domain like recommendations or playback), and CDN edge servers (serve video content close to users). Calling any of these "the server" in a Netflix system design interview would immediately reveal a shallow understanding of their architecture.
Mistake 4: Treating Caching as a Silver Bullet ⚠️
Caching is the most enthusiastically over-applied solution in system design interviews. The moment a candidate hears "the system is slow," the reflex fires: "add a cache." And caching is powerful — but dropping it into a design without discussing cache invalidation, consistency trade-offs, and eviction policies is the equivalent of saying "just use machine learning" without discussing training data or inference costs. It signals a surface-level understanding.
The classic computer science quip, attributed to Phil Karlton, captures it perfectly: "There are only two hard things in computer science: cache invalidation and naming things." Caching is hard precisely because of invalidation — and interviewers know this.
Here is what a shallow cache discussion looks like versus a thorough one:
## Shallow cache implementation — what most candidates show
import redis
import json
r = redis.Redis()
def get_user_profile(user_id: int) -> dict:
cached = r.get(f"user:{user_id}")
if cached:
return json.loads(cached)
# Fetch from DB (simulated)
profile = fetch_from_database(user_id)
r.set(f"user:{user_id}", json.dumps(profile)) # ← No TTL! Cache never expires
return profile
## ─────────────────────────────────────────────────────────────────
## Thorough cache implementation — what strong candidates show
CACHE_TTL_SECONDS = 300 # 5 minutes — explicit trade-off decision
def get_user_profile_v2(user_id: int) -> dict:
cached = r.get(f"user:{user_id}")
if cached:
return json.loads(cached)
profile = fetch_from_database(user_id)
# TTL ensures staleness is bounded — discuss this trade-off explicitly
r.setex(f"user:{user_id}", CACHE_TTL_SECONDS, json.dumps(profile))
return profile
def update_user_profile(user_id: int, new_data: dict) -> None:
# Write-through: update DB first, then invalidate cache
# This prevents stale reads after writes
update_in_database(user_id, new_data)
r.delete(f"user:{user_id}") # ← Explicit invalidation on write
# Alternative: r.setex(...) with new data (write-through vs. cache-aside)
Notice what the second version forces you to discuss: the 5-minute TTL is a deliberate choice with a trade-off. Users might see slightly stale profile data for up to 5 minutes after an update. Is that acceptable? For a profile picture, probably yes. For an account balance, absolutely not. This is the conversation an interviewer wants to have.
The key caching questions you must address proactively:
🔧 What is the eviction policy? LRU (Least Recently Used) is most common, but LFU (Least Frequently Used) is better for skewed access patterns like celebrity social media profiles.
🔧 What is the invalidation strategy? Cache-aside (lazy loading), write-through (update cache on every write), or write-behind (async write to DB after caching)?
🔧 What is the consistency requirement? Can the system tolerate stale reads? For how long? This is a product question as much as a technical one.
🔧 What happens on a cache miss storm? If the cache goes down and every request hits the database simultaneously, you have a thundering herd problem. Consider probabilistic early expiration or request coalescing.
⚠️ Common Mistake: Candidates propose caching for write-heavy workloads. Caching provides the most benefit for read-heavy workloads with relatively stable data. For write-heavy systems, caching adds complexity without proportional benefit — and introduces consistency risk.
🎯 Key Principle: Every time you say "cache," follow it immediately with: what gets invalidated, when, and what the consistency trade-off is. That one habit separates candidates who know caching from candidates who know caching well enough to use it safely.
Mistake 5: Jumping to Solutions Before Defining Requirements ⚠️
This is the most structurally damaging mistake on the list, because it does not just use the wrong word — it uses the right words in the wrong context. A candidate who immediately starts designing microservices and Kafka pipelines for a URL shortener that needs to handle 100 requests per day has not misunderstood the technology; they have misunderstood the process. And system design is fundamentally a process of translating ambiguous requirements into concrete technical decisions.
The vocabulary problem here is subtle but important: when you jump to solutions, you are implicitly defining terms for yourself without aligning with the interviewer. You assume "high availability" means 99.99% uptime when the interviewer meant 99.9%. You assume "user" means anonymous web visitors when the interviewer had authenticated enterprise users in mind. You assume "fast" means under 100ms when the product team is fine with 500ms.
Requirements vocabulary is its own domain of system design terminology:
- Functional requirements: What the system does — the features and behaviors.
- Non-functional requirements (NFRs): How the system performs — latency, availability, durability, consistency.
- Scale requirements: How many users, requests per second, data volume, geographic distribution.
- Consistency model: Strong, eventual, or causal consistency — and which trade-offs are acceptable.
- SLA (Service Level Agreement): The contractual performance target, e.g., 99.9% uptime.
- SLO (Service Level Objective): The internal target you set to meet the SLA, typically more conservative.
- SLI (Service Level Indicator): The actual measured metric, e.g., request success rate.
## This is a mental model, not runtime code — but it demonstrates
## how requirements should constrain design decisions.
## Before designing anything, establish these parameters explicitly:
class SystemRequirements:
"""
Template for requirements gathering in a system design interview.
Ask the interviewer to fill in these values before sketching architecture.
"""
# Functional
core_features: list[str] # What must the system do?
out_of_scope: list[str] # What are we explicitly NOT building?
# Scale
daily_active_users: int # e.g., 10_000_000
read_write_ratio: str # e.g., "100:1 reads to writes"
peak_requests_per_second: int # e.g., 50_000
data_retention_years: int # e.g., 5 — affects storage estimates
# Non-functional
latency_p99_ms: int # e.g., 200 — 99th percentile target
availability_slo: float # e.g., 0.999 — three nines
consistency_requirement: str # e.g., "eventual" or "strong"
geo_distribution: bool # Single region or multi-region?
# Constraints
budget_sensitivity: str # "cost is no object" vs. "optimize spend"
existing_infrastructure: list # What tech stack is already in place?
The reason vocabulary alignment matters here is that once you know the answers to these questions, the right vocabulary for the rest of the interview changes. A system requiring strong consistency across regions leads you to talk about distributed transactions and consensus protocols. A system tolerating eventual consistency leads you to talk about conflict-free replicated data types (CRDTs) and vector clocks. A system needing 99.999% availability leads you to discuss active-active multi-region failover. You cannot use the right words if you have not established the right context.
💡 Pro Tip: In the first two to three minutes of any system design question, ask three questions: (1) "What scale are we designing for?" (2) "Are there consistency requirements I should know about?" (3) "Are there parts of the system that are out of scope today?" These questions do not make you look unprepared — they make you look like a senior engineer who knows that the solution space is completely different depending on the answers.
🤔 Did you know? Google's Site Reliability Engineering team codified the SLI/SLO/SLA framework precisely because different parts of the organization were using "availability" to mean different things. The vocabulary itself became a systemic problem at scale — which is exactly why vocabulary precision is treated as a professional skill, not a pedantic concern.
📋 Quick Reference Card: The Five Mistakes and Their Fixes
| ❌ Mistake | ✅ Correction | 🎯 Signal to Interviewer |
|---|---|---|
| 🔀 "Scale it" to fix slowness | Distinguish performance bottleneck vs. capacity limit | You understand root cause analysis |
| 🗄️ "Use a database" | Name type + justify: "PostgreSQL for ACID" or "Redis for O(1) lookups" | You know tool-problem fit |
| 🖥️ "Add more servers" | Specify: load balancer, app server, cache server, DB replica | You think in components, not blobs |
| ⚡ "Just cache it" | Add: TTL, invalidation strategy, consistency trade-off | You understand caching is a contract |
| 🏃 Jump to architecture | Gather scale, consistency, and scope requirements first | You design for the actual problem |
Putting It All Together
These five mistakes are not isolated bad habits — they often chain together in a single interview response. A candidate might say: "For high traffic, I would scale the server, put a cache in front of the database, and that should handle the load." In one sentence, they have conflated scalability with performance, used "server" generically, proposed caching without any invalidation strategy, and implied a design without establishing what "high traffic" means.
The corrected version of that sentence might sound like: "Before we design for scale, can I confirm the expected read/write ratio and the peak RPS? Assuming this is read-heavy at around 50,000 requests per second, I would horizontally scale the application servers behind a Layer 7 load balancer. For the data layer, given the access patterns you described, a Redis cache with an LRU eviction policy and a 5-minute TTL on user profile data would reduce database load significantly — though we should discuss whether 5 minutes of staleness is acceptable for this use case."
That is the same idea, communicated with precision. The difference in perceived competence between these two answers is enormous — not because the architecture changed, but because the vocabulary carried the underlying thinking accurately.
💡 Remember: Interviewers are not testing whether you know the "right" architecture. In most cases, there is no single right answer. They are testing whether you can reason clearly under ambiguity, communicate trade-offs precisely, and demonstrate that your vocabulary reflects genuine understanding. Fixing these five mistakes does not just make you sound better — it makes you think better, because precise language and precise thinking reinforce each other.
Key Takeaways and Your System Design Vocabulary Cheat Sheet
You started this lesson without a shared language for talking about distributed systems. You finish it with one. That shift — from vague intuition to precise vocabulary — is exactly what separates candidates who "know how to code" from candidates who "think like engineers." In this final section, we consolidate everything into a reference you can return to before every practice session, every mock interview, and every real one.
Let's lock it all in.
Recapping the System Design Mental Model
Every system design conversation, no matter how complex the final architecture, starts with the same skeleton. Before you touch databases, caching strategies, or consistency models, you need to visualize the core mental model clearly.
[ Client ] ──── HTTP/WebSocket ────► [ Load Balancer ]
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
[ Server A ] [ Server B ] [ Server C ]
│ │ │
└────────┬────────┘ │
▼ ▼
[ Primary DB ] ◄──── Replication ──┘
│
[ Cache Layer ]
(Redis / Memcached)
This diagram encodes the five conceptual layers you now command:
🧠 Clients — any device or program that initiates a request and consumes a response. A browser, a mobile app, a CLI tool — all clients.
📡 Communication layer — the protocols (HTTP, WebSocket, gRPC) and infrastructure (DNS, CDN, load balancers) that route requests from clients to servers reliably and efficiently.
🔧 Application servers — stateless or stateful processes that contain business logic. They receive requests, execute logic, and return responses. They do not store long-lived data.
🗄️ Data stores — the persistent memory of the system. Relational databases, NoSQL stores, object storage, and message queues all live here.
⚡ Caching layer — a speed-optimization tier that stores frequently accessed data in fast memory to reduce load on data stores and decrease latency.
💡 Mental Model: Think of this as a restaurant. The client is the diner. The load balancer is the host who seats people. The servers are the waitstaff executing your order. The database is the kitchen with all the ingredients. The cache is the pre-made mise en place the chef grabs first before going to the pantry.
Your System Design Vocabulary Cheat Sheet
This glossary is your quick-reference companion for every subsequent lesson in this roadmap. Each term is defined in plain language and paired with a one-line analogy to make it stick.
📋 Quick Reference Card: Core System Design Terminology
| Term | Plain-Language Definition | One-Line Analogy |
|---|---|---|
| 🖥️ Client | Any system that sends requests and consumes responses | A customer placing an order |
| 🌐 Server | A process that receives requests and returns responses | A waiter fulfilling the order |
| ⚖️ Load Balancer | Distributes incoming traffic across multiple servers | An airport gate agent assigning planes to gates |
| 🗄️ Database | Persistent, structured storage for application data | A filing cabinet that never forgets |
| ⚡ Cache | Fast, temporary storage for frequently read data | A chef's prep station vs. the walk-in pantry |
| 🔁 Replication | Copying data across multiple nodes for redundancy | Making backup copies of an important document |
| 🔀 Sharding | Splitting a database horizontally across multiple machines | Dividing a phone book by last name across volumes |
| 📬 Message Queue | A buffer that decouples producers from consumers asynchronously | A postal system — sender drops a letter, receiver picks it up later |
| 🌍 CDN | A distributed network of servers that delivers static content from edge nodes close to users | A local distribution warehouse instead of shipping from HQ |
| 🔗 API | A defined contract for how services communicate | A restaurant menu — it tells you what you can order and how |
| ↕️ Vertical Scaling | Adding more resources (CPU, RAM) to a single machine | Upgrading from a sedan to an SUV |
| ↔️ Horizontal Scaling | Adding more machines to a pool | Adding more lanes to a highway |
| ⏱️ Latency | Time taken for a single request to travel from client to server and back | How long it takes to send and receive one text message |
| 🚿 Throughput | Number of requests a system can handle per unit of time | How many cars can pass through a toll booth per hour |
| 🩺 Availability | Percentage of time a system is operational and responding to requests | A store's "open" sign — how often it's actually open |
| 🤝 Consistency | Every read receives the most recent write | Everyone in a meeting seeing the same version of a shared doc |
| 🧱 Monolith | A single deployable application containing all functionality | One big Swiss Army knife |
| 🧩 Microservices | An architecture where functionality is split across independent, small services | A toolbox with individual specialized tools |
🎯 Key Principle: You don't need to memorize these definitions word-for-word. You need to be able to use them correctly mid-conversation. The goal is fluency, not recitation.
How These Concepts Feed Into What's Coming Next
This lesson was not a standalone unit — it was the foundation poured before the structure goes up. Every major topic in the rest of this roadmap builds directly on what you've learned here.
Scalability (Next Lesson)
Scalability is entirely built on the vocabulary distinction between vertical scaling (scaling up) and horizontal scaling (scaling out), and it depends on your understanding of stateless servers, load balancers, and database replication. When an interviewer asks "how would you scale this system?", your answer must immediately reference these building blocks.
## Simplified example: stateless server design enables horizontal scaling
## A stateless server doesn't hold session data locally — it reads from shared storage
import redis
import json
from flask import Flask, request, jsonify
app = Flask(__name__)
## Shared session store — not in-process memory
## This is what makes the server stateless and horizontally scalable
session_store = redis.Redis(host='redis-cluster.internal', port=6379, db=0)
@app.route('/get-cart', methods=['GET'])
def get_cart():
user_id = request.args.get('user_id')
# Session lives in Redis, not in this server's memory
# Any server instance can handle this request — true horizontal scaling
cart_data = session_store.get(f'cart:{user_id}')
if cart_data:
return jsonify(json.loads(cart_data))
return jsonify({'items': []})
## If this server crashes, another instance handles the next request seamlessly
## because NO state is stored locally
This code demonstrates why stateless servers are a prerequisite concept for scalability: by offloading session state to a shared cache (Redis), any server in the pool can handle any request. That's horizontal scaling in action.
Latency vs. Throughput (Upcoming Lesson)
You've already been introduced to both terms in this lesson. The upcoming lesson takes them further — exploring the tradeoffs between optimizing for one vs. the other, and how architectural choices (CDN placement, caching strategy, database indexing) shift the balance. Without understanding what latency and throughput are, you cannot reason about how to tune them.
CAP Theorem (Upcoming Lesson)
The CAP Theorem is impossible to discuss without a firm grasp of consistency, availability, and what distributed systems actually mean. You now have all three. When the CAP lesson explains why you must choose two of three guarantees, you'll have the vocabulary to understand exactly what's being traded away and why it matters for your data store choice.
💡 Pro Tip: Before each upcoming lesson, reopen this cheat sheet. Read the 4–5 terms most relevant to that lesson's topic. It takes 90 seconds and meaningfully improves how quickly you absorb new material by activating prior knowledge.
The Self-Assessment Checklist
Before moving on to the Scalability lesson, you should be able to answer each of the following questions confidently — in your own words, without referencing notes. If any of these give you pause, revisit the corresponding section of this lesson.
Conceptual Understanding
- Can you explain the difference between a client and a server without using either word in the definition?
- Can you describe what a load balancer does and name one real-world scenario where it's essential?
- Can you articulate the difference between vertical scaling and horizontal scaling, and explain when you'd choose one over the other?
- Can you define latency and throughput and explain how they relate to each other?
- Can you explain why caching improves performance and name one risk of using a cache?
- Can you describe the difference between replication and sharding?
Vocabulary Precision
- Can you use the terms idempotent, stateless, and eventual consistency in a sentence without looking them up?
- Can you explain the difference between a monolith and microservices and name one tradeoff of each?
- Can you explain what a message queue is and why you'd use one instead of a direct service-to-service call?
Interview Readiness
- If an interviewer says "the system needs to be highly available," can you immediately translate that into architectural choices?
- Can you sketch a basic 3-tier architecture (client → server → database) and label every component correctly?
- Can you explain the difference between SQL and NoSQL databases and name a use case for each?
⚠️ If you checked fewer than 8 of these 12 boxes confidently, invest another 20–30 minutes reviewing sections 2 and 3 of this lesson before proceeding. These concepts are load-bearing — everything else rests on them.
Practice Exercise: Sketch the Social Media Feed Architecture
The single most effective way to solidify system design vocabulary is to use it in context. Here is your practice exercise.
Scenario: Design a basic architecture for a social media feed — the kind you see on Twitter/X or Instagram. Users can post content. Other users can see a feed of posts from people they follow.
Step 1: Identify the Components
Before drawing anything, list out what you'll need using the vocabulary from this lesson:
- 🖥️ Clients — mobile apps and web browsers
- ⚖️ Load balancer — routes incoming feed and post requests
- 🔧 Application servers — handles feed generation logic and post creation
- 🗄️ Primary database — stores users, posts, and follower relationships (likely relational)
- ⚡ Cache layer — stores pre-computed feeds for active users (Redis)
- 📬 Message queue — processes "fan-out" events when a user posts (so millions of followers' feeds update asynchronously)
- 🌍 CDN — serves images and videos from edge nodes
Step 2: Sketch the Diagram
[Mobile/Web Clients]
│
▼
[ CDN ] ◄──── Static assets (images, videos)
│
▼
[ Load Balancer ]
│
┌────┴────┐
▼ ▼
[Server] [Server] ◄── Stateless application servers
│ │
└────┬────┘
│
┌────┴──────────────────┐
▼ ▼
[Cache Layer] [Primary DB]
(Pre-built feeds) (Posts, Users,
Follows)
│
▼
[Message Queue]
(Fan-out on post)
Step 3: Label Every Decision
Now write one sentence for each component explaining why it's there using correct terminology:
## Social Feed Architecture — Annotated Decision Log
Decisions:
1. Load Balancer → Enables horizontal scaling of app servers;
distributes traffic evenly; eliminates single point of failure.
2. Stateless App Servers → Any server can handle any request;
session state lives in the cache, not in-process memory.
3. Cache Layer (Redis) → Pre-computed feeds reduce read latency;
most users read feeds far more than they post (read-heavy workload).
4. Primary DB (PostgreSQL) → Relational model suits structured data
(users, follows); strong consistency needed for follower counts.
5. Message Queue (Kafka) → Decouples post creation from feed updates;
fan-out to millions of followers is async — improves write throughput.
6. CDN → Reduces latency for static assets by serving from edge nodes
geographically close to users.
This exercise is not about getting the architecture "right" — there is no single correct answer. It's about getting comfortable talking about systems using precise language. The moment you can label every arrow and box with the correct term and explain why it's there, you are interview-ready at the vocabulary level.
💡 Real-World Example: Twitter's original architecture was a Rails monolith serving feeds generated in real time from the database. As they scaled to hundreds of millions of users, they migrated to a fan-out-on-write model with Redis caching pre-built timelines — exactly the cache + message queue pattern described above. The vocabulary you used to label that diagram is the same vocabulary Twitter engineers used in their architecture reviews.
What You Now Understand That You Didn't Before
Let's make the growth explicit. Here's a before-and-after summary of this lesson's impact:
📋 Quick Reference Card: Before vs. After This Lesson
| 🔴 Before This Lesson | 🟢 After This Lesson |
|---|---|
| 🔴 "The website sends data to a server" | 🟢 "The client makes an HTTP request; the load balancer routes it to a stateless app server" |
| 🔴 "We store data in the database" | 🟢 "We use a relational DB for structured writes and a Redis cache to reduce read latency" |
| 🔴 "The system needs to handle more users" | 🟢 "We need to scale horizontally by adding app server instances behind the load balancer" |
| 🔴 "We should make it faster" | 🟢 "We can reduce latency by adding a CDN for static assets and caching hot database reads" |
| 🔴 Freezing when asked "how does this scale?" | 🟢 Immediately reaching for replication, sharding, and caching as vocabulary |
🎯 Key Principle: System design interviews are not tests of whether you can build the perfect system. They are tests of whether you can think out loud in the language of systems engineering. That language is what you just learned.
⚠️ Final critical point to remember: Vocabulary without application is trivia. Application without vocabulary is guessing. The only path to interview confidence is using these terms in practice — out loud, in diagrams, in writing — until they are automatic. Start with the practice exercise above, and do it again with a different system (a ride-sharing app, a streaming service, an e-commerce checkout) until labeling components feels natural.
Your Next Steps
🔧 Immediate action: Complete the social media feed diagram exercise. Time yourself — aim to produce a labeled sketch in under 10 minutes.
📚 Before the next lesson: Re-read the glossary table once. Pay special attention to the distinction between vertical and horizontal scaling — it's the central concept of the Scalability lesson.
🎯 Ongoing practice: After every lesson in this roadmap, add new terms to your personal version of this cheat sheet. Build your own living glossary. By the end of the roadmap, it will be your most valuable interview artifact.
🤔 Did you know? Research on technical interviews at top-tier companies consistently shows that candidates who use precise domain vocabulary are rated significantly higher on "senior-level thinking" — even when their proposed solutions are functionally similar to candidates who use vague language. The words you choose signal how you think.
The foundation is built. The language is yours. Let's go build systems.