Foundations of System Design
Build the core knowledge required before tackling any system design interview question.
Why System Design Matters: The Big Picture
Imagine you've just shipped your most elegant piece of code. The logic is tight, the tests pass, and your pull request gets approved with glowing comments. You feel like a real engineer. Then the product launches, ten thousand users sign up in an hour, and your beautifully crafted application collapses like a house of cards in a thunderstorm. The database chokes. The servers time out. Users see error pages. The on-call engineer — maybe you — is scrambling at 2 a.m. wondering what went wrong. Sound familiar? Even if you haven't lived this nightmare personally, you've almost certainly witnessed it from a distance. And the root cause, more often than not, isn't bad code. It's bad system design. Grab our free flashcards at the end of this section to lock in the vocabulary before moving forward — these terms will follow you throughout the entire roadmap.
System design is the discipline of thinking beyond individual functions and classes to ask: How does the whole thing hold together at scale? It's the difference between building a toy and building infrastructure. And mastering it is what separates developers who can build features from engineers who can build systems — the kind that serve millions of users, survive traffic spikes, and keep running when individual components fail.
The Cost of Poor System Design: When Giants Fall
Let's ground this in reality, because system design failures aren't academic thought experiments — they're expensive, embarrassing, and sometimes career-defining moments for the engineers involved.
Twitter's Fail Whale Era
For years, Twitter's iconic "Fail Whale" error image was so common that it became a cultural meme. The company had built its initial architecture as a monolithic Rails application — a single, tightly coupled codebase where every feature lived together. This made perfect sense for a startup with a handful of engineers moving fast. But as Twitter grew from thousands to tens of millions of users, the monolith became a bottleneck. A single slow database query could cascade into a site-wide outage. The timeline feature — showing you tweets from people you follow — required so many real-time database joins that the system regularly buckled under load.
The fix took years of painful re-architecture: decomposing the monolith into services, pre-computing timelines, and introducing caching layers. The engineering cost was enormous. The reputational cost was visible to every user who hit that whale.
💡 Real-World Example: Twitter's migration from a monolithic architecture to a service-oriented one is one of the most-studied examples in the industry. Their engineering blog posts from 2012–2015 document what happens when you defer architectural decisions until they become emergencies.
Netflix and the Great Outage
In 2008, Netflix suffered a major database corruption incident that took their DVD shipping service offline for three days. Rather than patch the problem, they made a bold architectural decision: migrate entirely to Amazon Web Services and redesign their system around fault tolerance — the principle that any individual component should be able to fail without bringing down the whole system.
This decision gave birth to their famous Chaos Engineering practice, where they intentionally kill production services to ensure the system can survive real failures. The Netflix architecture that emerged — distributed, microservices-based, with aggressive redundancy — is now a template that the entire industry studies.
🤔 Did you know? Netflix's "Chaos Monkey" tool, which randomly terminates production instances, was intentionally designed to run during business hours — not at 3 a.m. — so engineers would be awake to respond to failures and learn from them in real time.
Amazon's $1.6 Million Per Minute Problem
Amazon has publicly stated that a one-second delay in page load time costs them 1% in sales. During peak events like Prime Day, their systems handle millions of requests per second. A poorly designed caching layer, an under-provisioned database, or a single point of failure in payment processing can translate to losses measured in millions of dollars per minute.
In 2013, an Amazon Web Services outage took down not just Amazon properties but a significant portion of the internet — Netflix, Instagram, and Pinterest among them — because so many products had been built on top of AWS without designing for regional failover. The lesson was stark: your dependencies have dependencies, and system design must account for failure at every level of the stack.
🎯 Key Principle: Failure is not an edge case — it is a certainty. Good system design assumes that hardware will fail, networks will partition, and software will have bugs. The question is never if something will break, but when, and whether your architecture can survive it gracefully.
How System Design Interviews Differ From Coding Interviews
If you've prepared for software engineering interviews before, you've probably spent hours on LeetCode — grinding through dynamic programming problems, memorizing graph traversal algorithms, and learning to reverse a linked list in your sleep. Coding interviews test a specific, narrow skill: can you solve a well-defined algorithmic problem with an optimal solution?
System design interviews are fundamentally different, and developers who approach them with a coding-interview mindset almost always struggle.
What Evaluators Are Actually Looking For
When an interviewer asks you to "design Twitter" or "build a URL shortener that handles a billion users," they are not looking for the correct answer. There is no correct answer. What they're evaluating is your engineering judgment — your ability to:
🧠 Identify and clarify requirements before jumping to solutions 📚 Make explicit trade-offs between competing concerns (consistency vs. availability, latency vs. throughput) 🔧 Reason about scale — what works for 1,000 users, what breaks at 1 million, what needs to be redesigned at 1 billion 🎯 Communicate your thinking clearly and defend your decisions under questioning 🔒 Recognize failure modes and design for resilience
💡 Mental Model: Think of a system design interview like an architectural review meeting with a senior engineer. You're not being graded on finding the answer — you're being evaluated on whether you think like someone who has built real systems and learned hard lessons from them.
The contrast with coding interviews is sharp:
| 🖥️ Coding Interview | 🏗️ System Design Interview | |
|---|---|---|
| 📋 Problem scope | Well-defined, bounded | Deliberately ambiguous |
| ✅ Success criteria | Correct output, optimal complexity | Sound reasoning, good trade-offs |
| ⏱️ Time structure | Mostly heads-down coding | Heavy discussion and dialogue |
| 🎯 What's tested | Algorithmic thinking | Engineering judgment |
| 📊 Right answer? | Usually yes | Almost never |
❌ Wrong thinking: "I need to figure out the correct architecture and present it confidently." ✅ Correct thinking: "I need to ask the right questions, reason through trade-offs out loud, and show that I understand why design decisions matter."
⚠️ Common Mistake — Mistake 1: Jumping straight into solutions before understanding requirements. In a real system design interview, failing to ask clarifying questions in the first few minutes is a significant red flag. Senior engineers always start with "What are we optimizing for?"
The Lifecycle of a System: From Napkin Sketch to Production Infrastructure
Every production system you've ever used started as something embarrassingly simple. Understanding this lifecycle is crucial, because system design is not about building the perfect architecture on day one — it's about making the right decisions for your current stage and designing in a way that allows you to evolve.
Stage 1: The Prototype (One Server, No Scale)
Let's make this concrete. Imagine you're building a URL shortening service — the kind that turns https://www.example.com/very/long/path into https://sho.rt/abc123. In the beginning, your entire architecture might look like this:
[User Browser] ──► [Single Web Server + App + Database]
And your code might look something like this:
## Simple URL shortener - prototype stage
import hashlib
import sqlite3
def shorten_url(long_url: str) -> str:
# Generate a short code from the URL using MD5 hash
hash_code = hashlib.md5(long_url.encode()).hexdigest()[:6]
# Store in a local SQLite database (fine for prototypes!)
conn = sqlite3.connect('urls.db')
cursor = conn.cursor()
cursor.execute(
'INSERT OR IGNORE INTO urls (short_code, long_url) VALUES (?, ?)',
(hash_code, long_url)
)
conn.commit()
conn.close()
return f'https://sho.rt/{hash_code}'
def resolve_url(short_code: str) -> str:
conn = sqlite3.connect('urls.db')
cursor = conn.cursor()
# Simple lookup - works perfectly at small scale
cursor.execute('SELECT long_url FROM urls WHERE short_code = ?', (short_code,))
result = cursor.fetchone()
conn.close()
return result[0] if result else None
This code works. It's readable, testable, and ships fast. For your first hundred users, it's completely appropriate. The mistake would be over-engineering it on day one.
Stage 2: Early Growth (Adding Basic Infrastructure)
As your service grows to thousands of users, you start hitting predictable problems. The SQLite file can't handle concurrent writes. Every request hits the database even for URLs that are resolved a thousand times a day. Now your architecture evolves:
[User Browser]
│
▼
[Load Balancer]
/ \
[App Server] [App Server] ← Multiple instances for availability
\ /
│
▼
[Cache Layer (Redis)] ← Frequently resolved URLs cached
│
▼
[Primary Database (Postgres)] ← Write operations
│
▼
[Read Replica] ← Read-heavy operations distributed
Notice what changed: not the core logic, but the infrastructure around it. Your URL shortening function might barely change, but how it talks to data storage changes dramatically.
## URL shortener - growth stage with caching
import redis
import psycopg2
## Initialize connections (in production, use connection pooling)
cache = redis.Redis(host='cache.internal', port=6379, decode_responses=True)
def resolve_url_with_cache(short_code: str) -> str:
# Check cache first - cache hits are ~1ms vs ~10ms for DB
cached_url = cache.get(f'url:{short_code}')
if cached_url:
return cached_url # Cache hit: fast path
# Cache miss: go to database
conn = psycopg2.connect(dsn='postgresql://readonly_user@read-replica/urls')
cursor = conn.cursor()
cursor.execute('SELECT long_url FROM urls WHERE short_code = %s', (short_code,))
result = cursor.fetchone()
conn.close()
if result:
long_url = result[0]
# Store in cache for 24 hours to reduce future DB hits
cache.setex(f'url:{short_code}', 86400, long_url)
return long_url
return None
This code introduces caching — a pattern so fundamental to system design that you'll encounter it in nearly every architecture discussion. The key insight is the comment: cache hits are ~1ms vs ~10ms for a database call. At 10 million requests per day, that difference is the gap between a system that barely survives and one that handles load gracefully.
💡 Pro Tip: The evolution from Stage 1 to Stage 2 isn't just about performance — it's about identifying bottlenecks. The database is almost always the first bottleneck in a growing web service. Understanding why this is true (disks are slow, locks are expensive, connections are limited) is more valuable than memorizing specific solutions.
Stage 3: Production Scale
At millions of daily active users, the architecture transforms further — database sharding, CDN integration, message queues for async processing, geographic distribution for latency, circuit breakers for fault isolation. Each evolution introduces new complexity and new trade-offs. This is the territory that system design interviews are actually probing.
🎯 Key Principle: Every architectural decision is a trade-off. Caching improves read speed but introduces cache invalidation complexity. Microservices improve scalability but introduce distributed systems problems. Database sharding improves write throughput but complicates queries. Understanding these trade-offs — not memorizing "right" answers — is the core skill.
What This Roadmap Covers and How It Builds
This lesson is the first of six in the Foundations of System Design module, and each section is deliberately sequenced to build on the last. Understanding the architecture of this roadmap itself will help you navigate it more effectively.
🧠 Section 1 (this section): Establishes why system design matters and creates the mental stakes. You now understand that poor design has real consequences, that interviews test judgment not memorization, and that systems evolve through predictable stages.
📚 Section 2 — Core Components: Before you can design systems, you need vocabulary. We'll introduce the fundamental building blocks — load balancers, databases, caches, message queues, CDNs — that appear in virtually every large-scale architecture. Think of this as learning the alphabet before writing prose.
🔧 Section 3 — Thinking in Systems: A structured mental framework for approaching design problems. Requirements gathering, back-of-the-envelope estimation, trade-off reasoning — this is the process that separates strong system design candidates from weak ones.
🎯 Section 4 — Design Walkthrough: We apply everything to a concrete example: designing a URL shortener from scratch, walking it through multiple stages of scale. Seeing abstract concepts applied to a real problem cements understanding in a way that theory alone cannot.
🔒 Section 5 — Common Pitfalls: The most frequent mistakes developers make — both in real systems and in interviews. Forewarned is forearmed.
📋 Section 6 — Key Takeaways: A quick-reference summary and bridge to the deeper sub-topics that follow in this course.
🧠 Mnemonic: "VATDC" — Vocabulary, Approach, Trade-offs, Design, Common mistakes. That's the arc of these six sections. If you can internalize this flow, you'll always know where you are in the learning journey.
Why This Matters for Your Career
Let's be direct about something: system design skills have an asymmetric impact on your career trajectory. Junior developers are evaluated almost entirely on coding ability — can you implement features, write clean code, fix bugs? But as you move into senior engineering roles, the ability to design systems becomes the primary differentiator.
Senior engineers and staff engineers are expected to make architectural decisions that teams will live with for years. The engineer who can look at a proposed design and immediately spot that "this will fall apart when you hit 10 million writes per day because you haven't accounted for write amplification on this index" is worth dramatically more to an organization than one who cannot.
In interviews specifically, system design rounds are almost universally present at FAANG and equivalent companies for any role at the senior level or above. Candidates who struggle here — even those with exceptional coding skills — frequently don't pass. The inverse is also true: candidates who demonstrate sophisticated system design thinking can sometimes compensate for slightly weaker algorithmic performance.
💡 Pro Tip: System design skills compound. Every production system you work on teaches you something — where the bottlenecks appeared, what design decisions caused pain, which abstractions held up under pressure. The developers who grow fastest are those who actively study the architecture of the systems they work on, not just their individual feature area.
📋 Quick Reference Card: Why System Design Matters
| Area | 🔍 What it affects | 📊 Stakes |
|---|---|---|
| 🔧 Production reliability | Uptime, user experience | Revenue, reputation |
| 🎯 Interview performance | Senior+ role offers | Career velocity |
| 🧠 Team effectiveness | Architectural decisions | Years of productivity |
| 📚 Personal growth | Engineering judgment | Senior → Staff path |
⚠️ Common Mistake — Mistake 2: Treating system design as something to "cram" before interviews rather than a continuous practice. Developers who truly excel at system design think architecturally all the time — when reading engineering blog posts, when reviewing pull requests, when using products and wondering how they work under the hood.
The engineers who built Twitter's infrastructure, who designed Netflix's chaos engineering practice, who architected Amazon's distributed systems — they didn't learn these skills from a checklist. They built them through deliberate study, hands-on experience, and the expensive lessons that come from real failures.
You're starting that journey now. And unlike them, you have the benefit of studying their documented failures, their published architectures, and the accumulated wisdom of a field that has spent decades learning what works and what doesn't.
That's an enormous advantage. Let's use it.
The Building Blocks: Core Components of Any System
Before you can design a system, you need to speak its language. Every large-scale application — whether it's Netflix streaming video to 200 million households or Twitter processing 500 million tweets per day — is built from the same fundamental components. These building blocks are the shared vocabulary of system design, and mastering them gives you the mental model to reason about any architecture you encounter in the wild or in an interview room.
Think of this section as learning the periodic table before studying chemistry. You don't need to memorize every element on day one, but you need to understand what atoms are and how they bond before you can explain molecules. In the same way, understanding clients, servers, databases, caches, load balancers, message queues, and CDNs gives you the atoms of distributed systems. The rest of this course — and your career — is about combining them intelligently.
Clients, Servers, and the Request-Response Model
Every distributed system begins with a conversation. At its core, the internet is an enormous collection of machines asking each other questions and sending back answers. This interaction is formalized in what we call the request-response model.
A client is any entity that initiates a request. A server is any entity that receives that request, processes it, and returns a response. Critically, these are roles, not physical machines — the same computer can be a client in one interaction and a server in another. Your laptop is a client when it requests a webpage from Google's servers, but a developer's local machine running a web server is a server when it responds to curl commands from the terminal.
CLIENT SERVER
┌────────┐ HTTP Request ──► ┌──────────┐
│ Browser│ │ Web App │
│ App │ ◄── Response │ Server │
└────────┘ └──────────┘
│ │
│ GET /api/users/42 │
│ ─────────────────────────► │
│ │
│ 200 OK { "name": "Alice" } │
│ ◄───────────────────────── │
The HTTP protocol is the dominant language of this conversation on the web. When your browser navigates to https://example.com, it sends an HTTP GET request to a server, which responds with HTML, CSS, and JavaScript. The protocol defines how messages are structured — not what they contain.
💡 Real-World Example: When you open the Instagram app on your phone, your phone (the client) sends a request to Instagram's servers asking for your feed. The server queries a database, assembles the response, and sends back a JSON payload containing posts, usernames, and image URLs. Your app then renders this data as the scrollable feed you see. This entire round trip, from tap to rendered image, ideally takes under 200 milliseconds.
🎯 Key Principle: The request-response model introduces latency — the time delay between sending a request and receiving a response. Every design decision you make will either increase or decrease latency. Understanding this is the foundation of performance thinking in system design.
Stateless vs. Stateful Servers
One of the most important properties of a server is whether it is stateful or stateless. A stateless server does not store any information about past client interactions. Each request contains all the information the server needs to fulfill it. A stateful server remembers previous interactions and uses that context to handle new requests.
❌ Wrong thinking: "Stateful servers are better because they remember the user, which feels more personalized."
✅ Correct thinking: "Stateless servers are generally preferred at scale because any server in a cluster can handle any request, making horizontal scaling dramatically simpler."
This distinction becomes critical when you add load balancers into the picture — which we will explore shortly.
Databases, Caches, and Storage Layers
If the client-server model explains how data moves, the storage layer explains where data lives — and why that distinction shapes everything about your system's behavior.
Databases: The Source of Truth
A database is a persistent, organized store of data. "Persistent" is the key word: when your server restarts, the data in the database survives. Databases come in two broad families:
Relational databases (SQL) organize data into tables with rows and columns. They enforce strict schemas and support powerful query languages. Examples include PostgreSQL, MySQL, and SQLite. When you store a user's account information, their order history, and the products they've purchased, a relational database lets you join those tables to answer complex questions like "show me all orders for users who signed up in California last month."
Non-relational databases (NoSQL) sacrifice some of that query power in exchange for flexibility, horizontal scalability, or specialized access patterns. MongoDB stores data as JSON-like documents. Redis stores key-value pairs in memory. Cassandra optimizes for write-heavy workloads across many nodes. DynamoDB powers Amazon's retail platform and prioritizes availability and scale.
⚠️ Common Mistake — Mistake 1: Defaulting to a relational database for every problem. If you're building a leaderboard that needs the top 10 scores updated hundreds of times per second, a relational database is the wrong tool. A sorted set in Redis handles this with microsecond-level performance. Match the database to the access pattern, not the other way around.
Caches: Trading Space for Speed
A cache is a high-speed storage layer that holds a subset of data, typically the most recently or frequently accessed items. The fundamental trade-off is simple: memory is expensive but fast; disk is cheap but slow. Caches keep hot data in memory so you don't have to hit the database on every request.
REQUEST
│
▼
┌─────────┐ Cache HIT? ┌───────────┐
│ Server │─────────────────►│ Cache │
│ │◄─────────────────│ (Redis) │
│ │ Return data └───────────┘
│ │ │
│ │ Cache MISS? │ No
│ │ ▼
│ │◄────────────────┌───────────┐
│ │ Store + return│ Database │
└─────────┘ └───────────┘
💡 Mental Model: Think of a cache like the physical desk in a library. The entire collection is in the stacks (the database), but the books you're currently working with are on your desk (the cache). Retrieving a book from your desk takes seconds; walking to the stacks takes minutes.
The most widely used caching technology in production systems is Redis (Remote Dictionary Server). It stores data as key-value pairs in memory and supports rich data structures: strings, hashes, lists, sets, and sorted sets.
Here is a simple demonstration of caching a database query result with Redis using Python:
import redis
import json
import time
## Connect to Redis (your cache layer)
cache = redis.Redis(host='localhost', port=6379, db=0)
def get_user_profile(user_id: int, db_client) -> dict:
"""
Fetch a user profile, checking the cache first.
Cache miss: query the database and store result for 5 minutes.
Cache hit: return the cached result directly.
"""
cache_key = f"user:profile:{user_id}"
# Step 1: Check the cache
cached_data = cache.get(cache_key)
if cached_data:
print(f"Cache HIT for user {user_id}")
return json.loads(cached_data) # Deserialize and return
# Step 2: Cache miss — query the database
print(f"Cache MISS for user {user_id} — querying database")
user = db_client.query("SELECT * FROM users WHERE id = %s", user_id)
# Step 3: Store in cache with a 300-second (5 min) TTL
cache.setex(cache_key, 300, json.dumps(user))
return user
This pattern — check cache, fall back to database on a miss, populate the cache for next time — is called the cache-aside (or lazy loading) pattern. It is one of the most common caching strategies in production systems.
🤔 Did you know? Facebook's Memcached cluster at peak served over a billion requests per second, avoiding hundreds of millions of database queries every minute. Caching is not an optimization — at scale, it is a survival mechanism.
Load Balancers, Message Queues, and CDNs
Once your system grows beyond a single server, you need infrastructure to coordinate traffic, decouple components, and deliver content efficiently. These three components — load balancers, message queues, and CDNs — appear in virtually every large-scale architecture.
Load Balancers: Distributing the Work
A load balancer sits in front of a pool of servers and distributes incoming requests across them. Its job is to ensure no single server becomes a bottleneck while others sit idle, and to route traffic away from servers that are unhealthy.
┌─────────────────┐
Client ─────────►│ Load Balancer │
Client ─────────►│ │
Client ─────────►│ (e.g. Nginx, │
│ AWS ALB) │
└────────┬────────┘
│
┌──────────────┼──────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Server 1 │ │ Server 2 │ │ Server 3 │
└──────────┘ └──────────┘ └──────────┘
Load balancers use various routing algorithms to distribute traffic. Round-robin cycles through servers in order. Least connections sends new requests to the server with the fewest active connections. IP hashing always routes a given client to the same server — useful when you need soft session affinity.
Load balancers also perform health checks, periodically pinging each server. If a server fails to respond, the load balancer stops sending traffic to it until it recovers. This is a cornerstone of high availability.
Message Queues: Decoupling with Asynchrony
A message queue is a buffer that temporarily stores messages (jobs, events, tasks) produced by one component until another component is ready to consume them. The producer and consumer are decoupled — they don't need to be running simultaneously, and the producer doesn't wait for the consumer to finish.
💡 Real-World Example: When you upload a video to YouTube, the upload completes almost instantly. But the transcoding (converting your video into dozens of formats and resolutions) takes minutes. YouTube doesn't make you wait — it places a transcoding job on a message queue and returns a "processing" status immediately. Worker servers pick jobs off the queue and process them asynchronously. Popular message queue systems include Apache Kafka, RabbitMQ, and Amazon SQS.
User Upload
│
▼
┌─────────┐ Enqueue job ┌──────────────┐ Dequeue
│ API │─────────────────►│ Message Queue│◄──────────────┐
│ Server │ │ (Kafka/SQS) │ │
└─────────┘ └──────────────┘ ┌───────────────┐
│ │ Video Encoder │
│ 202 Accepted │ Workers │
▼ └───────────────┘
Client sees
"Processing..."
🎯 Key Principle: Message queues transform tight synchronous coupling into loose asynchronous coupling. This makes systems more resilient (a slow consumer doesn't block the producer), more scalable (you can add more worker instances independently), and more fault-tolerant (if a worker crashes mid-job, the message can be redelivered).
Content Delivery Networks: Bringing Data Closer
A CDN (Content Delivery Network) is a geographically distributed network of servers — called edge nodes or points of presence (PoPs) — that cache and serve static content close to end users. Instead of every request for your company's logo traveling from a user's browser in Tokyo all the way to your origin server in Virginia, a CDN serves it from a node in Tokyo.
The physics of the internet are unforgiving: light travels through fiber optic cables at roughly two-thirds the speed of light in a vacuum, and a round trip from Tokyo to Virginia adds about 150–200ms of latency before a single byte of work is done. CDNs eliminate this for static assets (images, CSS, JavaScript, video) by caching them at the edge.
🧠 Mnemonic: Think of a CDN as a chain of warehouses (edge nodes) stocking popular goods close to customers, so that the central factory (origin server) only needs to ship items when a warehouse runs out of stock (cache miss).
Providers like Cloudflare, AWS CloudFront, and Fastly operate hundreds of PoPs worldwide. For a global application, a CDN is not a luxury — it is table stakes.
How These Components Interact: An End-to-End Architecture
Isolated components are just vocabulary. The real insight comes from understanding how they compose into a working system. Let's trace a single request through a realistic architecture — say, loading a profile page on a social network.
┌────────────────────────────────────────────────────────────────┐
│ THE INTERNET │
└────────────────────────────────────────────────────────────────┘
│ │
│ Static assets (images, JS, CSS) │ Dynamic API requests
▼ ▼
┌─────────────┐ ┌──────────────┐
│ CDN │ │ Load Balancer│
│ (Cloudflare)│ │ (AWS ALB) │
└─────────────┘ └──────┬───────┘
│
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ App │ │ App │ │ App │
│ Server 1│ │ Server 2│ │ Server 3│
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└────────────┼────────────┘
│
┌──────────────┼──────────────┐
▼ ▼
┌────────────┐ ┌────────────┐
│ Cache │ │ Database │
│ (Redis) │ │ (Postgres) │
└────────────┘ └────────────┘
│
┌────────────────┐
│ Message Queue │
│ (async tasks) │
└────────────────┘
Here is what happens when a user visits a profile page:
- 🌐 The browser requests the page's JavaScript and CSS. These are served instantly by the CDN from a nearby edge node — no origin server involved.
- 📡 The browser makes an API call to
/api/users/42/profile. This request hits the load balancer, which routes it to one of three available app servers using round-robin. - ⚡ The app server checks Redis for a cached version of user 42's profile. It finds it (cache hit). Response time: 2ms.
- 📬 The user views the profile and clicks "follow." The app server enqueues a "send follow notification" job to the message queue and immediately returns a 200 OK to the client. The notification is processed asynchronously by a worker.
- 🗄️ When the cache TTL expires and the next request is a cache miss, the app server queries PostgreSQL for the fresh user record, stores it in Redis, and returns it.
This is not a hypothetical architecture — it is a simplified version of how companies like Twitter, LinkedIn, and GitHub serve millions of users daily.
## Simplified example: An API endpoint using the full stack
from flask import Flask, jsonify
import redis
import json
app = Flask(__name__)
cache = redis.Redis(host='redis-host', port=6379)
@app.route('/api/users/<int:user_id>/profile')
def get_profile(user_id):
"""
Demonstrates the cache-aside pattern in an API endpoint.
The load balancer routes requests here; we check cache before DB.
"""
cache_key = f"user:{user_id}:profile"
# 1. Check Redis cache first
if cached := cache.get(cache_key):
return jsonify(json.loads(cached)), 200 # Fast path: ~2ms
# 2. Cache miss: query the database (slow path: ~20-50ms)
user = database.find_user(user_id) # Abstracted DB call
if not user:
return jsonify({"error": "User not found"}), 404
# 3. Populate cache for next request (TTL: 5 minutes)
cache.setex(cache_key, 300, json.dumps(user))
return jsonify(user), 200
This snippet illustrates how a single API endpoint interacts with the cache layer — a pattern that is replicated across thousands of endpoints in a mature system.
A Brief Introduction to Networking Concepts
The components above don't exist in a vacuum — they communicate over networks governed by protocols and addressing schemes. A full treatment of networking belongs to the dedicated child lessons ahead, but you need a working mental model now to reason about the architectures we've already discussed.
Every device on a network has an IP address — a numeric identifier that lets routers direct traffic to the right destination. The TCP/IP protocol stack is the foundation of internet communication: IP handles routing packets between machines, while TCP ensures reliable, ordered delivery of those packets. When a client connects to your web server, they're establishing a TCP connection over the HTTP (or HTTPS) protocol.
DNS (Domain Name System) translates human-readable names (like api.yourapp.com) into IP addresses. When you deploy a load balancer, you point your DNS record at the load balancer's IP. Clients resolve the domain, get that IP, and connect — never knowing or caring which of your three app servers actually handles the request.
Ports are logical endpoints on a machine. A server can run multiple services simultaneously — a web server on port 443 (HTTPS), a database on port 5432, a Redis cache on port 6379. When a client connects, it specifies both an IP address and a port to reach the right service.
📋 Quick Reference Card: Core Networking Terms
| 🔑 Term | 📖 What It Does | 💡 Example |
|---|---|---|
| 🌐 IP Address | Uniquely identifies a device on a network | 192.168.1.1, 10.0.0.42 |
| 🚪 Port | Identifies a specific service on a device | HTTP → 80, HTTPS → 443 |
| 📡 DNS | Translates domain names to IP addresses | api.app.com → 52.10.4.12 |
| 🔗 TCP | Guarantees reliable, ordered packet delivery | File downloads, API calls |
| ⚡ UDP | Fast, connectionless delivery (no guarantee) | Video streaming, gaming |
| 🔒 TLS/HTTPS | Encrypts data in transit | All modern web traffic |
⚠️ Common Mistake — Mistake 2: Assuming all traffic uses TCP. Real-time applications like video calls (Zoom, FaceTime) and online games frequently use UDP because it sacrifices reliability for speed. A dropped video frame is barely noticeable; waiting for a TCP retransmission to deliver it creates jarring lag. Understanding when to choose UDP over TCP is a common system design interview question.
💡 Pro Tip: In system design interviews, when you mention that services communicate over the network, briefly noting the protocol (REST over HTTP, gRPC, WebSockets) signals to the interviewer that you think in complete systems, not just abstract boxes. It's a small detail that separates good candidates from great ones.
Putting It All Together: Your New Mental Model
You now have a working vocabulary for discussing distributed systems. These building blocks — clients and servers exchanging requests and responses, databases persisting data, caches accelerating reads, load balancers distributing traffic, message queues decoupling work, and CDNs serving static content at the edge — appear in virtually every system you will ever design or interview about.
The critical insight is that no component exists in isolation. A cache only makes sense in relation to the database it fronts. A load balancer only matters when you have multiple servers to balance across. A message queue only adds value when a producer and consumer have different throughput characteristics or reliability requirements.
As you move through the rest of this lesson and into deeper sub-topics, you will revisit each of these components in detail. But you already have what you need to read an architecture diagram, ask intelligent clarifying questions, and reason about trade-offs. That is the foundation everything else is built on.
Thinking in Systems: How to Approach Design Problems
A blank whiteboard is one of the most intimidating things a software engineer can face. The interviewer says, "Design Twitter," and suddenly your mind races through a thousand possible starting points — databases, caches, load balancers, microservices — all clamouring for attention at once. The engineers who perform well in these moments aren't the ones who know the most buzzwords. They're the ones who have internalized a structured thinking process that lets them transform an overwhelming, ambiguous question into a manageable series of smaller decisions. That process is what this section teaches.
Thinking in systems is a learnable skill. It combines requirements gathering, estimation, trade-off analysis, and iterative refinement into a repeatable workflow you can apply to any design problem, whether you're in an interview room or architecting a production service at work.
Functional vs. Non-Functional Requirements: What the System Does vs. How Well It Does It
Every design conversation should begin in the same place: clarifying requirements. Skipping this step is the single most common way candidates derail a system design interview, and it's equally dangerous in real engineering. Requirements fall into two fundamental categories.
Functional requirements describe what the system does — the observable behaviors and features that users or clients directly interact with. For a URL shortener, functional requirements might include: users can submit a long URL and receive a short one; clicking the short URL redirects to the original; users can optionally specify a custom alias.
Non-functional requirements describe how well the system performs those functions. These are the quality attributes that constrain your design choices. Common non-functional requirements include:
- 🎯 Availability: What percentage of the time must the system be operational? (e.g., 99.9% means ~8.7 hours of downtime per year; 99.99% means ~52 minutes)
- 🎯 Latency: What is the maximum acceptable response time for reads? For writes?
- 🎯 Throughput: How many requests per second must the system handle?
- 🎯 Durability: If data is written, what is the acceptable risk of losing it?
- 🎯 Consistency: If multiple users read the same data concurrently, must they always see identical values?
- 🎯 Scalability: How should the system behave as load grows by 10x or 100x?
💡 Real-World Example: When Netflix designs a video streaming pipeline, a functional requirement is "users can play any title in their region." A non-functional requirement is "video playback must begin within 2 seconds for 95% of users." The first tells you what to build; the second forces you to think about CDN placement, adaptive bitrate encoding, and pre-fetching — decisions that never surface if you only think functionally.
⚠️ Common Mistake: Jumping to solutions before requirements are clear. If an interviewer says "design a messaging app," they might mean WhatsApp (1 billion users, end-to-end encryption, offline delivery) or they might mean an internal notification service for 1,000 employees. These require radically different architectures. Always ask clarifying questions first.
A practical framework for requirements gathering is to ask about scale, consistency needs, and read/write ratio before drawing a single box on the whiteboard:
Scale questions:
- How many users (total vs. daily active)?
- How many requests per second at peak?
- What is the expected data volume over 5 years?
Consistency questions:
- Is it acceptable for users to briefly see stale data?
- What happens if a write is lost? (e.g., a lost tweet vs. a lost bank transfer)
Read/write ratio questions:
- Is this read-heavy, write-heavy, or balanced?
- Are reads and writes on the same data, or different?
The answers to these questions should fundamentally change your design. A read-heavy system with tolerable staleness points toward aggressive caching and eventual consistency. A write-heavy system with strict durability requirements points toward write-ahead logs, synchronous replication, and potentially sacrificing some read performance.
The Art of Estimation: Back-of-the-Envelope Calculations
Once you understand the requirements, you need to translate them into numbers. Back-of-the-envelope (BOTE) calculations are rough quantitative estimates that help you understand the order of magnitude of your problem. They don't need to be precise — they need to be directionally correct enough to drive meaningful design decisions.
The goal of estimation is to answer questions like: Do I need one database server or one hundred? Can a single application server handle the traffic? How much storage do I need to provision for the next five years? Will my network bandwidth become a bottleneck before my CPU does?
Essential Numbers Every Engineer Should Know
Before you can estimate, you need a mental library of reference values:
| Resource | Approximate Value |
|---|---|
| L1 cache reference | 0.5 ns |
| Main memory reference | 100 ns |
| SSD random read | 100 µs |
| HDD seek | 10 ms |
| Network round trip (same datacenter) | 0.5 ms |
| Network round trip (cross-continent) | 150 ms |
| Data read from SSD (sequential, 1MB) | 1 ms |
| Data read from HDD (sequential, 1MB) | 20 ms |
🤔 Did you know? Memory is roughly 200,000x faster than a disk seek. This single fact explains why caching is one of the highest-leverage architectural decisions in system design.
A Worked Estimation Example
Let's say you're designing a photo-sharing service and the interviewer tells you it has 100 million daily active users (DAU), with each user viewing an average of 20 photos per day and uploading an average of 2 photos per day.
## Back-of-the-envelope estimation in Python
## These calculations are illustrative — you'd do this on paper/whiteboard
DAU = 100_000_000 # 100 million daily active users
photos_viewed_per_user = 20
photos_uploaded_per_user = 2
avg_photo_size_mb = 3 # compressed JPEG, typical smartphone photo
## --- Traffic Estimation ---
total_views_per_day = DAU * photos_viewed_per_user
total_uploads_per_day = DAU * photos_uploaded_per_user
## Convert to requests per second (there are 86,400 seconds in a day)
## Rule of thumb: divide by 100,000 for a rough RPS from daily count
read_rps = total_views_per_day / 86_400
write_rps = total_uploads_per_day / 86_400
print(f"Read RPS: {read_rps:,.0f}") # ~23,148 reads/sec
print(f"Write RPS: {write_rps:,.0f}") # ~2,315 writes/sec
print(f"Read/Write ratio: {read_rps/write_rps:.0f}:1") # ~10:1
## --- Storage Estimation ---
storage_per_day_gb = (total_uploads_per_day * avg_photo_size_mb) / 1024
storage_per_year_tb = (storage_per_day_gb * 365) / 1024
storage_5_years_tb = storage_per_year_tb * 5
print(f"Storage per day: {storage_per_day_gb:,.0f} GB")
print(f"Storage per year: {storage_per_year_tb:,.0f} TB")
print(f"Storage 5 years: {storage_5_years_tb:,.0f} TB") # ~1,068 TB ≈ 1 PB
This simple calculation immediately surfaces several design implications. A 10:1 read-to-write ratio strongly suggests read optimization — caching, CDN distribution, and read replicas become priorities. A ~1 petabyte of storage over 5 years tells you that you'll need distributed object storage (like S3 or Google Cloud Storage) rather than a simple filesystem. A ~23,000 reads per second tells you that a single database server (typically handling 1,000–5,000 complex queries per second) will be a bottleneck, pointing toward caching layers and horizontal scaling.
💡 Pro Tip: During an interview, narrate your estimation process out loud. Say "I'm assuming 100 million DAU, with about 20 reads and 2 writes per user per day. That gives me roughly 23,000 RPS reads and 2,300 RPS writes, so this is about a 10:1 read-heavy workload." This shows the interviewer how you think, not just what answer you arrive at.
Bandwidth Estimation
Bandwidth is often forgotten until it becomes a crisis in production. Estimate it early:
## Bandwidth estimation
read_rps = 23_148
write_rps = 2_315
avg_photo_size_mb = 3
avg_thumbnail_size_kb = 50 # Users see thumbnails in feeds, not full photos
## Most reads are thumbnail views (feeds/explore), full photo loads are less frequent
full_photo_reads_rps = read_rps * 0.10 # ~10% load full resolution
thumbnail_reads_rps = read_rps * 0.90 # ~90% see thumbnails
## Incoming bandwidth (uploads)
incoming_bandwidth_mbps = write_rps * avg_photo_size_mb * 8 # convert MB to Mb
print(f"Incoming bandwidth: {incoming_bandwidth_mbps / 1000:.1f} Gbps")
## ~55.6 Gbps — this requires a distributed ingestion layer!
## Outgoing bandwidth (downloads)
outgoing_thumbnail_mbps = thumbnail_reads_rps * (avg_thumbnail_size_kb / 1024) * 8
outgoing_full_mbps = full_photo_reads_rps * avg_photo_size_mb * 8
outgoing_total_gbps = (outgoing_thumbnail_mbps + outgoing_full_mbps) / 1000
print(f"Outgoing bandwidth: {outgoing_total_gbps:.1f} Gbps")
## Outgoing >> Incoming → CDN becomes essential
The moment you see outgoing bandwidth dwarf incoming bandwidth, your architecture must include a Content Delivery Network (CDN). You cannot serve 55+ Gbps of photo data from a single origin server — that number alone justifies a distributed edge-caching strategy.
🧠 Mnemonic: "STRAW" — Storage, Traffic (RPS), Read/write ratio, Availability target, Bandwidth Width. Run through STRAW for every system you design to ensure you haven't missed a key estimation dimension.
Trade-Off Thinking: Why No Design Is Universally Correct
Once you have requirements and rough numbers, the real intellectual work of system design begins: making trade-offs. Every architectural decision you make optimizes for some properties at the expense of others. The engineer who understands this produces better designs than the one who searches for a "correct" answer.
The most famous formalization of this idea is the CAP theorem, introduced by Eric Brewer in 2000 and later proven by Gilbert and Lynch. It states that any distributed data store can guarantee at most two of the following three properties simultaneously:
Consistency
/\
/ \
/ \
/ \
/________\
Availability -- Partition
Tolerance
CAP Theorem: Pick any two.
In practice, network partitions are unavoidable,
so the real choice is between C and A during a partition.
- Consistency (C): Every read returns the most recent write, or an error. All nodes see the same data at the same time.
- Availability (A): Every request receives a response (though it might not be the most recent data). The system stays operational.
- Partition Tolerance (P): The system continues operating even when network messages between nodes are dropped or delayed.
Since network partitions are an unavoidable reality in distributed systems, you're almost always choosing between CP (consistent but may become unavailable during a partition) and AP (always available but may serve stale data during a partition).
💡 Mental Model: Think about a bank vs. a social media feed. Your bank balance must be consistent — you cannot accept "eventually we'll figure out if that charge went through." That's a CP system. Your Twitter feed can show you slightly stale data — seeing a tweet from 5 seconds ago instead of right now is fine. That's an AP system. Let the use case drive the choice, not the technology preference.
The Consistency Spectrum
CAP is a useful starting point, but real systems operate on a spectrum of consistency models:
| Consistency Model | Guarantee | Example Use Case |
|---|---|---|
| Linearizability | All operations appear instantaneous and in order | Financial transactions |
| Sequential consistency | Operations appear in some global order | Multi-player game state |
| Causal consistency | Causally related ops seen in order | Comment replies on a post |
| Eventual consistency | All replicas converge given no new writes | DNS propagation, shopping carts |
Applying Trade-Off Thinking Beyond CAP
Trade-off thinking extends far beyond the CAP theorem. Almost every architectural decision involves a similar tension:
- 🔧 Latency vs. Consistency: A cache gives you faster reads but stale data. How stale is acceptable?
- 🔧 Throughput vs. Durability: Writing asynchronously to disk gives higher throughput but risks data loss on crash. Kafka's
acks=allvs.acks=1is this exact trade-off. - 🔧 Cost vs. Performance: More replicas improve read performance and fault tolerance but multiply storage costs.
- 🔧 Simplicity vs. Scalability: A monolith is simpler to develop and operate; microservices enable independent scaling but add network complexity, distributed tracing overhead, and operational burden.
🎯 Key Principle: In a system design interview, the right answer to most architectural questions is not "yes" or "no" but "it depends, and here's what it depends on." Interviewers are evaluating your ability to reason through trade-offs, not recall the one correct architecture.
⚠️ Common Mistake: Defaulting to microservices, Kubernetes, and Kafka for every problem because they sound sophisticated. A system handling 1,000 requests per day is probably better served by a well-structured monolith. Premature distribution is a source of massive operational complexity for no benefit.
❌ Wrong thinking: "I should use a NoSQL database because it's more scalable." ✅ Correct thinking: "My write patterns are document-oriented with variable schema, and I need horizontal write scaling. A document store fits better here than a relational DB, though I'll lose ACID transactions across entities."
Iterative Design: Starting Simple and Evolving Toward Complexity
One of the most effective mental shifts you can make is to treat system design as an iterative process, not a one-shot declaration. Start with the simplest possible system that satisfies the core functional requirements, then identify bottlenecks and evolve.
Iteration 1: The Naive Design
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Client │────▶│ Server │────▶│ DB │
└──────────┘ └──────────┘ └──────────┘
"This works for 100 users. Now let's stress test it."
Iteration 2: Add a Cache (read bottleneck identified)
┌──────────┐ ┌──────────┐ ┌───────┐ ┌──────┐
│ Client │────▶│ Server │────▶│ Cache │────▶│ DB │
└──────────┘ └──────────┘ └───────┘ └──────┘
Iteration 3: Add Load Balancer + Horizontal Scaling (CPU bottleneck)
┌──────────────┐
│ Load Balancer│
└──────┬───────┘
┌──────────┴──────────┐
┌─────▼────┐ ┌─────▼────┐
│ Server 1 │ │ Server 2 │
└─────┬────┘ └─────┬────┘
└──────────┬──────────┘
┌────▼────┐ ┌────────┐
│ Cache │────▶│ DB │
└─────────┘ └────────┘
Iteration 4: Add DB Replication (read/write split)
┌──────────────┐
│ Load Balancer│
└──────┬───────┘
┌───────────┴───────────┐
┌─────▼────┐ ┌─────▼────┐
│ Server 1 │ │ Server 2 │
└─────┬────┘ └─────┬────┘
└──────────┬───────────┘
│
┌────────▼────────┐
│ Cache Layer │
└────────┬────────┘
┌────────────┴────────────┐
┌────▼─────┐ ┌────▼─────┐
│ DB Primary│────────────▶│ DB Replica│
│ (Writes) │ replication │ (Reads) │
└───────────┘ └──────────┘
This iterative approach has a profound benefit: at each stage, you're solving a specific, identified problem rather than adding complexity speculatively. Real systems evolve this way. Instagram ran on a single server for the first few months. YouTube used MySQL for years longer than anyone expected. Simplicity is a feature.
💡 Pro Tip: When walking through a design iteratively, explicitly name the bottleneck before introducing each complexity. Say "At this scale, the single database becomes our write bottleneck. To address that, I'd introduce primary-replica replication with writes going to the primary and reads distributed across replicas." This demonstrates engineering discipline — adding complexity in response to constraints rather than by instinct.
Structuring Your Thinking in a 45-Minute Interview
All of the above concepts need to fit into a structured, time-boxed conversation. Here is a battle-tested framework for a 45-minute system design interview:
⏱ Minute 0–5: Requirements Clarification
Ask about functional requirements, scale, constraints.
Don't touch the whiteboard yet.
⏱ Minute 5–10: Estimation
Estimate traffic, storage, bandwidth.
Identify the dominant constraint (read-heavy? write-heavy? storage-bound?).
⏱ Minute 10–15: High-Level Design
Draw the simplest end-to-end architecture.
Cover the happy path only.
⏱ Minute 15–35: Deep Dive
Pick 2–3 components and go deep.
Discuss trade-offs, alternatives, failure modes.
Let the interviewer guide which areas to explore.
⏱ Minute 35–45: Scale & Edge Cases
Revisit bottlenecks at higher scale.
Discuss failure scenarios, monitoring, alerting.
Summarize trade-offs made.
🎯 Key Principle: The interviewer controls what you explore; you control how you explore it. Use your framework to stay structured, but remain flexible when the interviewer steers you toward a specific component.
📋 Quick Reference Card: The System Design Interview Checklist
| Phase | 🎯 Goal | ✅ Key Actions |
|---|---|---|
| 🔍 Requirements | Define scope | Ask about scale, consistency, features |
| 📐 Estimation | Quantify the problem | Calculate RPS, storage, bandwidth |
| 🗺 High-Level Design | Sketch the skeleton | Draw boxes and arrows, happy path only |
| 🔬 Deep Dive | Show engineering depth | Trade-offs, data models, API design |
| 📈 Scaling | Demonstrate scale thinking | Bottlenecks, replication, sharding |
The candidates who stand out in system design interviews don't know more facts — they have better mental habits. They slow down before drawing, they quantify before deciding, and they name trade-offs instead of asserting conclusions. These habits are learnable through deliberate practice, which is precisely what the rest of this course will give you.
Putting It All Together
Let's ground everything in a single coherent snapshot. Imagine you're asked to "design a rate limiter" in an interview. Here's how the framework applies in rapid sequence:
Requirements: Functional — block requests exceeding N per second per user. Non-functional — the rate limiter itself must add <1ms latency (it's in the hot path), handle 100,000 RPS globally, be eventually consistent across data centers (slightly stale counts are acceptable).
Estimation: 100,000 RPS × a few bytes per counter update = modest data volume. Latency is the binding constraint, not storage. This pushes us toward in-memory storage (Redis) rather than disk-based storage.
Trade-off: Strict counting across distributed nodes requires coordination (consistency) which adds latency. Eventual consistency allows local counters that sync lazily, accepting occasional over-admission in exchange for speed. For most use cases, the AP trade-off is correct here.
Iterative design: Start with a single Redis instance using the token bucket algorithm. Identify that a single Redis node is a SPOF (single point of failure). Add Redis Cluster. Identify cross-datacenter coordination overhead. Introduce local in-memory counters with periodic Redis sync.
This is systems thinking in action — methodical, quantified, trade-off-aware, and iterative. The next section will take these exact skills and apply them to a full walkthrough of designing a URL shortener from scratch.
From Monolith to Distributed: A Practical Design Walkthrough
The best way to internalize system design concepts is to watch them solve a real problem. Abstract principles become concrete when you see exactly why a caching layer gets added, or what breaks when traffic doubles overnight. In this section, we'll design a URL shortener service from scratch — starting with the simplest possible implementation and evolving it step by step under the pressure of scale. By the end, you'll be able to map every component in the final architecture back to the building blocks introduced earlier in this lesson.
URL shorteners are a classic system design exercise for a reason: they're simple enough to fully understand, yet rich enough to surface nearly every major architectural challenge — read/write asymmetry, caching, database bottlenecks, and horizontal scaling.
Step 1: The Single-Server Monolith
Let's start with the absolute minimum viable system. A URL shortener has two core operations:
- Shorten: Accept a long URL, generate a short code, store the mapping, and return the short URL.
- Redirect: Accept a short code, look up the original URL, and issue an HTTP redirect.
On a single server, the entire application — web server, business logic, and database — lives together. This is a monolithic architecture, where all components are tightly coupled and deployed as one unit.
┌─────────────────────────────────────┐
│ Single Server │
│ │
│ [Client] ──► [App Logic] ──► [DB] │
│ │
└─────────────────────────────────────┘
Here's what the core API contract looks like in Python using Flask:
import hashlib
import string
import random
from flask import Flask, redirect, request, jsonify
from datetime import datetime
app = Flask(__name__)
## In-memory store for simplicity (we'll replace this with a real DB shortly)
url_store = {}
def generate_short_code(long_url: str) -> str:
"""
Generate a 6-character alphanumeric short code.
Uses a hash of the URL + timestamp to reduce collision probability.
"""
raw = long_url + str(datetime.utcnow().timestamp())
hash_digest = hashlib.md5(raw.encode()).hexdigest()
# Take the first 6 characters of the hex digest
return hash_digest[:6]
@app.route('/shorten', methods=['POST'])
def shorten_url():
"""POST /shorten { "url": "https://very-long-url.com/path" }"""
data = request.get_json()
long_url = data.get('url')
if not long_url:
return jsonify({'error': 'URL is required'}), 400
short_code = generate_short_code(long_url)
url_store[short_code] = long_url # Write to our store
return jsonify({'short_url': f'https://short.ly/{short_code}'}), 201
@app.route('/<short_code>', methods=['GET'])
def redirect_url(short_code: str):
"""GET /<short_code> → 301 Redirect to the original URL"""
long_url = url_store.get(short_code) # Read from our store
if not long_url:
return jsonify({'error': 'URL not found'}), 404
# 301 = Permanent redirect (browsers cache this)
# 302 = Temporary redirect (browsers do NOT cache this)
return redirect(long_url, code=302)
if __name__ == '__main__':
app.run(port=8080)
This code captures the essential API contract: a POST /shorten endpoint that writes a mapping, and a GET /<short_code> endpoint that reads it and redirects. Notice the comment about 301 vs. 302 redirects — that's a subtle but real trade-off. A 301 tells browsers to cache the redirect permanently, reducing server load. A 302 forces every request back to your server, giving you accurate click analytics but more traffic.
💡 Pro Tip: In an interview, mentioning the 301 vs. 302 trade-off immediately signals that you think about the downstream consequences of design decisions, not just the happy path.
Step 2: Identifying the Bottlenecks
Our single-server system works beautifully for dozens of users. But imagine a popular marketing campaign puts a shortened link in front of millions of people. What breaks first?
🎯 Key Principle: Every system has a bottleneck — the single constraint that limits overall throughput. Your job as a designer is to find it, fix it, and then find the next one.
Let's trace the failure modes as our traffic grows:
CPU Exhaustion
Each redirect request triggers Python code execution — URL lookup, response formatting, HTTP handling. A single CPU-bound server might handle ~1,000–5,000 requests per second before it starts queuing. As connections pile up, latency spikes and requests start timing out.
Memory Pressure
Our in-memory url_store dictionary grows with every shortened URL. At 100 million stored URLs, even a conservative 100 bytes per entry means ~10 GB just for the mapping store. A single server simply runs out of RAM.
Database Contention
When we replace the in-memory dict with a real database (which we must, for durability), database contention becomes the critical bottleneck. Every redirect hits the database for a read. Every shorten operation does a write. On a single DB instance, these reads and writes compete for the same I/O resources, disk locks, and connection pool slots.
Traffic Pattern (URL Shortener):
Writes (Shorten): ██░░░░░░░░ ~10% of requests
Reads (Redirect): ████████░░ ~90% of requests
→ This is a read-heavy workload. Design accordingly.
🤔 Did you know? Services like Bitly process billions of redirects per month. At that scale, even a 1-millisecond improvement in redirect latency translates to thousands of CPU-hours saved annually.
⚠️ Common Mistake: Developers often assume that upgrading to a bigger server (called vertical scaling or "scaling up") is the natural next step. It works temporarily, but it has hard limits — you can only buy so much RAM and CPU — and it creates a single point of failure. One machine goes down; the entire service goes down.
Step 3: Horizontal Scaling — Adding More Servers
The solution to CPU and traffic exhaustion is horizontal scaling ("scaling out"): adding more application servers and distributing incoming requests across them. But this introduces a new architectural element that must sit in front of them — a load balancer.
┌─────────────┐
│ Load │
[Clients] ──────►│ Balancer │
└──────┬──────┘
│
┌───────────────┼───────────────┐
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ App Server │ │ App Server │ │ App Server │
│ #1 │ │ #2 │ │ #3 │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
└───────────────┼───────────────┘
▼
┌─────────────┐
│ Database │
│ (Primary) │
└─────────────┘
This changes the architecture in two important ways:
1. Application servers become stateless. Our original in-memory url_store can't work anymore — if Server #1 stores a mapping, and the next request for that short code lands on Server #3, it won't find it. Every server must read from and write to the shared database. Statelessness is the property that allows any server to handle any request, which is what makes horizontal scaling possible.
2. The database becomes the new bottleneck. We've solved the CPU problem, but now all three servers are hammering the same database simultaneously. We've shifted the bottleneck downstream.
💡 Mental Model: Think of horizontal scaling like adding checkout lanes at a grocery store. Adding lanes helps, but if all lanes share the same single inventory system in the back, that system becomes the new constraint.
Database Read Replicas
Since our workload is ~90% reads, we can introduce read replicas — copies of the database that serve read traffic. The primary database handles all writes; replicas continuously sync from it and absorb the redirect lookups.
┌─────────────┐
│ DB Primary │◄── All WRITES (shorten)
└──────┬──────┘
│ Replication
┌──────────┼──────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Replica 1│ │Replica 2│ │Replica 3│
└─────────┘ └─────────┘ └─────────┘
└──────────┼──────────┘
All READS (redirect)
⚠️ Common Mistake: Read replicas introduce replication lag — a small delay between a write hitting the primary and that data appearing on replicas. If a user shortens a URL and immediately tries to use it, they might hit a replica that hasn't synced yet and get a 404. This is called eventual consistency. Design your system to acknowledge this trade-off explicitly.
Step 4: Adding a Caching Layer
Even with read replicas, database reads have latency measured in milliseconds. For a redirect service where speed is the entire value proposition, we can do much better by introducing a cache — an in-memory data store that sits between the application and the database.
The fundamental principle here is temporal locality: if a URL was redirected recently, it will very likely be redirected again soon. Popular links follow a power-law distribution — a small fraction of URLs accounts for the vast majority of traffic. Cache those, and you can serve most redirects without touching the database at all.
[App Server] ──► [Cache (Redis)] ──► hit? Return URL
│
│ miss?
▼
[Database Replica] ──► fetch, populate cache, return URL
Here's what the redirect logic looks like with a Redis cache layer added:
import redis
import psycopg2 # PostgreSQL driver
from flask import Flask, redirect, jsonify
app = Flask(__name__)
## Connect to Redis cache
cache = redis.Redis(host='redis-server', port=6379, db=0)
## Connect to PostgreSQL (read replica for lookups)
db_conn = psycopg2.connect(
host='db-replica-1',
dbname='urlshortener',
user='app_user',
password='secret'
)
## Cache TTL: how long we store a URL before evicting it (in seconds)
CACHE_TTL_SECONDS = 3600 # 1 hour
@app.route('/<short_code>', methods=['GET'])
def redirect_url(short_code: str):
"""
Redirect logic with cache-aside pattern:
1. Check cache first (fast, in-memory)
2. On cache miss, check database (slower, disk-backed)
3. Populate cache on database hit
"""
# Step 1: Check the cache
cached_url = cache.get(short_code)
if cached_url:
# Cache HIT: serve directly, no DB query needed
return redirect(cached_url.decode('utf-8'), code=302)
# Step 2: Cache MISS — query the database
cursor = db_conn.cursor()
cursor.execute(
'SELECT long_url FROM urls WHERE short_code = %s',
(short_code,)
)
row = cursor.fetchone()
if not row:
return jsonify({'error': 'URL not found'}), 404
long_url = row[0]
# Step 3: Populate the cache for future requests
# TTL ensures stale/deleted URLs eventually expire from cache
cache.setex(name=short_code, time=CACHE_TTL_SECONDS, value=long_url)
return redirect(long_url, code=302)
This implements the cache-aside pattern (also called lazy loading): the application is responsible for checking the cache, and on a miss, fetching from the database and populating the cache. The alternative is write-through caching, where every write to the database simultaneously writes to the cache — ensuring the cache is always warm, but adding latency to every write operation.
The Read/Write Trade-Off in Caching
No caching strategy is universally correct. Each involves deliberate trade-offs:
| 🔵 Cache-Aside (Lazy Load) | 🟢 Write-Through | |
|---|---|---|
| Cache populated | On first read (miss) | On every write |
| Read latency | High on first request | Always low |
| Write latency | Low (DB only) | Higher (DB + Cache) |
| Stale data risk | Yes (TTL manages it) | Low |
| Best for | Read-heavy, unpredictable access | Predictable, write-then-read patterns |
💡 Real-World Example: Twitter's timeline service uses a write-through cache. When you post a tweet, it's written to both the database and the cache immediately, ensuring your followers see it instantly without a slow cache-miss lookup. URL shorteners, by contrast, are classic cache-aside candidates because most URLs are created once and accessed many times later.
⚠️ Common Mistake: Setting cache TTLs too long. If a user deletes or updates a shortened URL, the old destination will continue to be served from cache until the TTL expires. Always design a cache invalidation strategy — either event-driven invalidation (explicitly delete the cache key on update) or short-enough TTLs that staleness is acceptable.
🧠 Mnemonic: Remember the two hard problems in computer science: cache invalidation, naming things, and off-by-one errors. Cache invalidation is genuinely hard — respect it.
Step 5: The Final Architecture — Mapping It All Together
After applying each evolution — horizontal scaling, read replicas, and caching — here's what our URL shortener looks like at scale:
┌─────────────────────────────────────────────────────────────────┐
│ CLIENTS │
│ (browsers, apps, API consumers) │
└───────────────────────────┬─────────────────────────────────────┘
│ HTTPS
▼
┌─────────────────────────────────────────────────────────────────┐
│ LOAD BALANCER │
│ (distributes traffic, health checks) │
└──────────┬─────────────────┬────────────────────┬──────────────┘
│ │ │
▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────────┐
│ App Server │ │ App Server │ │ App Server │
│ #1 │ │ #2 │ │ #3 │
└─────┬──────┘ └─────┬──────┘ └──────┬─────────┘
│ │ │
└─────────────────┼──────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ CACHE LAYER (Redis Cluster) │
│ (in-memory, sub-millisecond reads, TTL-based) │
└───────────────────────────┬─────────────────────────────────────┘
│ cache miss
▼
┌─────────────────────────────────────────────────────────────────┐
│ DATABASE LAYER │
│ │
│ ┌──────────────┐ ┌──────────────────────────────┐ │
│ │ DB Primary │────────►│ Read Replicas (x3) │ │
│ │ (writes) │replicate│ (redirect lookups) │ │
│ └──────────────┘ └──────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Now let's map every component back to the building blocks vocabulary from earlier in this lesson:
📋 Quick Reference Card: Architecture Component Mapping
| 🔧 Component | 🏗️ Building Block Type | 🎯 Purpose in Our System |
|---|---|---|
| 🌐 Load Balancer | Traffic Distribution | Routes requests across app servers, provides failover |
| 🖥️ App Servers (x3) | Compute / Application Tier | Stateless logic: shorten + redirect |
| ⚡ Redis Cache | Caching Layer | Serves hot URLs without hitting DB |
| 🗄️ DB Primary | Persistent Storage (Write) | Source of truth for all URL mappings |
| 📖 DB Replicas | Persistent Storage (Read) | Scales read throughput for redirects |
| 🔗 HTTPS Endpoint | API Gateway / Network | Secure client-facing interface |
Tracing a Request Through the System
To cement your understanding, let's trace what happens when a user clicks https://short.ly/a3f9bc:
- 🌐 DNS resolves
short.lyto the load balancer's IP. - ⚖️ Load Balancer receives the request and forwards it to App Server #2 (least-connections algorithm).
- 🖥️ App Server #2 extracts the short code
a3f9bcand queries Redis. - ⚡ Cache HIT — Redis returns
https://www.original-very-long-domain.com/some/path. (95% of requests end here.) - 🖥️ App Server #2 returns a 302 redirect to the client.
- 🌐 Client browser follows the redirect to the original URL.
For a cache miss (step 4 returns nothing), the app server queries a read replica, gets the URL, writes it to Redis with a TTL, and then redirects. The next request for the same code will be a cache hit.
💡 Real-World Example: This is architecturally very close to how Bitly and TinyURL operate at their core. Real production systems add more layers — CDN edge nodes for global latency, analytics pipelines for click tracking, rate limiting to prevent abuse — but the spine of the architecture is exactly what we've built here.
🎯 Key Principle: System design is an iterative process. You don't design the final architecture first — you start simple, identify what breaks under load, and add complexity only where it's justified. Every component in our final diagram was added to solve a specific, observable problem. This "earn your complexity" mindset is what separates pragmatic engineers from over-engineers.
What This Walkthrough Taught Us
We started with 30 lines of Python and a dictionary. We ended with a distributed system capable of handling millions of redirects per second. More importantly, every architectural decision was driven by a specific problem:
- 🧠 Stateless app servers → because in-memory state breaks horizontal scaling
- 📚 Load balancer → because one server can't handle all traffic
- 🔧 Read replicas → because our workload is 90% reads
- 🎯 Redis cache → because database latency is too high for redirect use cases
- 🔒 TTL-based eviction → because deleted URLs must eventually disappear
This is the mental motion of system design: observe a constraint, reason about what component addresses it, understand the new trade-offs that component introduces, and repeat. The URL shortener is a microcosm of how every large-scale system in the world got to where it is today.
In the next section, we'll look at the most common mistakes developers make when applying these concepts — both in production and in interviews — so you can avoid the traps before you fall into them.
Common Pitfalls: Mistakes Developers Make in System Design
Every experienced system designer has a collection of hard-won lessons — decisions that seemed brilliant on a whiteboard and catastrophic in production. The encouraging news is that most of these mistakes follow recognizable patterns. Whether you're preparing for a system design interview or architecting a real production service, learning to spot these anti-patterns before they happen is one of the most valuable skills you can develop. This section catalogs the five most common and costly mistakes developers make, explains why they happen, and gives you concrete strategies for avoiding each one.
Pitfall 1: Over-Engineering Before You Understand the Problem
⚠️ Common Mistake: The moment a developer hears "design a messaging system," they reach for microservices, Kafka, distributed tracing, and a service mesh — before they've asked a single clarifying question.
This is perhaps the most seductive trap in system design. Complex architectures feel professional. Drawing a diagram with twelve interconnected boxes gives the impression of thorough thinking. But complexity has a price: it increases operational burden, slows iteration, introduces more failure points, and makes onboarding new engineers significantly harder.
Over-engineering occurs when the chosen architecture is more sophisticated than the actual problem demands. A system designed for 10 million daily active users that currently serves 500 beta testers is not ambitious — it's expensive and premature.
💡 Real-World Example: Amazon's early architecture was a monolith. So was Twitter's. So was Shopify's — and Shopify ran a highly successful monolith serving billions in transactions for years before gradually decomposing specific bottlenecks. The microservices revolution happened after these companies understood their domain boundaries and pain points intimately, not before.
The correct mental model is to design for your current scale with clear extension points for the next order of magnitude. A startup expecting 10,000 users does not need a Kubernetes cluster with auto-scaling across three availability zones.
## OVER-ENGINEERED: A user registration endpoint that introduces
## unnecessary asynchronous complexity for a service with 500 users
import asyncio
from kafka import KafkaProducer
from circuit_breaker import CircuitBreaker
from distributed_tracer import trace_span
@trace_span("user.registration")
async def register_user(user_data: dict) -> dict:
# Publishes to Kafka, consumed by a separate User Service
# which writes to DB — introducing async complexity and
# eventual consistency where none is needed yet
producer = KafkaProducer(bootstrap_servers='kafka:9092')
breaker = CircuitBreaker(failure_threshold=5)
await breaker.call(
producer.send, 'user-registration-events', user_data
)
return {"status": "registration_event_published"}
## APPROPRIATE: A direct, synchronous registration for early stage
def register_user_simple(user_data: dict, db) -> dict:
# Write directly to database — simple, debuggable, reliable
user_id = db.users.insert({
"email": user_data["email"],
"password_hash": hash_password(user_data["password"]),
"created_at": utcnow()
})
return {"user_id": user_id, "status": "created"}
The second function is not naive — it's appropriate. When registration becomes a bottleneck (and you'll know because you'll have metrics), you can extract and optimize that path. Until then, the simpler version ships faster, fails more obviously, and is easier to reason about.
🎯 Key Principle: Let your architecture grow with your understanding of the problem, not ahead of it. A system that evolves from a well-structured monolith to targeted microservices is far healthier than one that begins as a distributed system without the operational maturity to support it.
❌ Wrong thinking: "If I don't design for massive scale from day one, I'll have to rewrite everything later."
✅ Correct thinking: "I'll design clean boundaries and sensible abstractions now. Scaling comes from identifying the actual bottleneck, not from speculating about all possible bottlenecks."
Pitfall 2: Designing Only the Happy Path
Imagine designing a bridge by calculating only how it behaves on a calm, sunny day with average traffic. You'd be omitting the scenarios that actually determine whether the bridge stands — storms, overloads, material fatigue. System design suffers from the same blind spot when engineers optimize for the case where everything works.
The happy path is the execution flow where inputs are valid, all downstream services respond, the network is reliable, and no machine fails. In reality, failure modes are not edge cases — they are inevitabilities. Disks corrupt. Networks partition. Third-party APIs time out. Services crash under load.
⚠️ Common Mistake: Designing a payment processing flow that handles a successful charge but has no defined behavior for network timeouts, duplicate requests, or partial failures.
Consider a classic failure scenario: your service calls a payment provider, the network request succeeds on the provider's side but the response is lost in transit. Your service throws an exception and your retry logic triggers — charging the customer twice. This is not hypothetical; it happens in production regularly.
The solution is idempotency: designing operations so that performing them multiple times has the same effect as performing them once.
import uuid
import hashlib
def process_payment(user_id: str, amount: int, idempotency_key: str, db, payment_provider):
"""
Idempotent payment processing.
If called multiple times with the same idempotency_key,
returns the original result without charging again.
"""
# Check if we've already processed this exact request
existing = db.payments.find_one({"idempotency_key": idempotency_key})
if existing:
# Safe to return — not a double charge
return {"status": existing["status"], "payment_id": existing["payment_id"]}
try:
# Attempt the charge with the provider
result = payment_provider.charge(user_id=user_id, amount=amount)
# Persist the outcome before returning
db.payments.insert({
"idempotency_key": idempotency_key,
"user_id": user_id,
"amount": amount,
"status": "success",
"payment_id": result["payment_id"]
})
return {"status": "success", "payment_id": result["payment_id"]}
except TimeoutError:
# Record the attempt so we can investigate — do NOT retry blindly
db.payments.insert({
"idempotency_key": idempotency_key,
"status": "timeout",
"payment_id": None
})
raise # Propagate so the caller handles the uncertainty
This code demonstrates two failure-aware patterns simultaneously: idempotency guards against duplicate processing, and explicit timeout handling prevents silent failures.
Beyond idempotency, robust designs consider circuit breakers (stopping calls to a failing dependency before it cascades), bulkheads (isolating failures to prevent them from consuming all system resources), and graceful degradation (serving a reduced-quality response when a dependency is unavailable rather than returning an error).
FAILURE MODE THINKING: Questions to ask for every component
┌────────────────────────────────────────────────────────┐
│ For each component, ask: │
│ │
│ 1. What happens if THIS component fails? │
│ └─► Does the whole system fail? (bad) │
│ └─► Can we degrade gracefully? (good) │
│ │
│ 2. What happens if a DEPENDENCY of this │
│ component is slow or unresponsive? │
│ └─► Do we block forever? (bad) │
│ └─► Do we time out and circuit break? (good) │
│ │
│ 3. What if an operation is retried? │
│ └─► Does retrying cause duplicate effects? (bad) │
│ └─► Is the operation idempotent? (good) │
└────────────────────────────────────────────────────────┘
💡 Pro Tip: In a system design interview, explicitly narrating your failure mode thinking signals senior-level maturity. After describing how the happy path works, say: "Now let me walk through what happens when the database write succeeds but the cache update fails..." This kind of reasoning distinguishes candidates who have thought deeply about systems from those who have only memorized architecture patterns.
Pitfall 3: Confusing Vertical Scaling for a Long-Term Strategy
Vertical scaling (scaling up) means adding more resources — CPU, RAM, faster disk — to a single machine. Horizontal scaling (scaling out) means adding more machines and distributing load across them.
Vertical scaling is tempting because it requires no application changes. Feeling database pressure? Upgrade to a larger instance. It's fast, it works, and it buys time. The mistake is treating it as a strategy rather than a tactic.
⚠️ Common Mistake: A developer designs a system where all components are assumed to scale vertically indefinitely, never addressing the architectural changes required for horizontal distribution.
The problem is physics. There is a ceiling to how powerful a single machine can be, and the price-performance curve gets brutally steep at the high end. A machine with 64 cores and 512GB of RAM costs many times more than eight machines with 8 cores and 64GB each — and the eight machines also provide fault tolerance that the single machine cannot.
SCALING COMPARISON
Vertical Scaling:
┌─────────────────┐ ┌───────────────────────┐
│ Small Server │ ──► │ Bigger Server │
│ 4 CPU / 16GB │ │ 64 CPU / 512GB │
└─────────────────┘ └───────────────────────┘
Single point of failure at every stage
Hard ceiling exists (Moore's Law limits)
Expensive at high end
Requires downtime to upgrade
Horizontal Scaling:
┌──────────┐ ┌──────────┐
│ Server 1 │ │ Server 1 │
│4CPU/16GB │ ──────► │4CPU/16GB │
└──────────┘ ├──────────┤
│ Server 2 │
│4CPU/16GB │
├──────────┤
│ Server 3 │
│4CPU/16GB │
└──────────┘
No single point of failure (with load balancer)
Near-linear capacity growth
Commodity hardware pricing
Rolling deploys with no downtime
The practical implication is that your application must be designed with horizontal scaling in mind from early on. This means keeping application servers stateless — no user session data stored locally, no in-memory caches that can't be shared. State lives in shared stores (databases, Redis, S3) that all instances can access equally.
🤔 Did you know? The shift from vertical to horizontal scaling is why stateless design became so fundamental to cloud architecture. If your application server holds session state locally, you can't freely route requests to any instance — you're stuck with "sticky sessions" that break horizontal scalability and complicate deployments.
🎯 Key Principle: Design your stateless application layer for horizontal scaling. Use vertical scaling tactically when you need headroom while implementing architectural improvements, not as the answer to long-term growth.
Pitfall 4: Skipping the Requirements Phase
In a system design interview, the single most reliable way to give a poor answer is to start drawing architecture diagrams before asking any questions. In production engineering, the equivalent is beginning implementation before aligning on constraints. Both mistakes stem from the same impulse: appearing decisive and knowledgeable by jumping straight to solutions.
Requirements gathering is the process of discovering and documenting what a system must do (functional requirements), how well it must do it (non-functional requirements), and what constraints it must respect. Skipping this phase means you're designing a solution to an imagined problem — which may bear little resemblance to the actual one.
⚠️ Common Mistake: A candidate is asked to design Instagram. They immediately begin: "We'll have a user service, a photo storage service, a feed service..." — without asking how many users, what the read/write ratio is, whether Stories are in scope, or what the latency requirements are.
Consider how dramatically requirements change the design:
| 📋 Requirement Variable | 🔧 Low End | 🚀 High End | ⚙️ Design Impact |
|---|---|---|---|
| 🔢 Daily Active Users | 10,000 | 100,000,000 | Single DB vs sharded distributed store |
| 📖 Read/Write Ratio | 1:1 (balanced) | 100:1 (read-heavy) | Caching strategy and replica configuration |
| ⏱️ Latency Tolerance | 5 seconds | 50 milliseconds | CDN necessity, database indexing depth |
| 🌍 Geographic Distribution | Single region | Global | Multi-region replication, data residency laws |
| 💰 Consistency Requirements | Eventual OK | Strong required | Choice of database and replication strategy |
A structured requirements intake looks like this:
REQUIREMENTS GATHERING FRAMEWORK
1. FUNCTIONAL REQUIREMENTS (What must it do?)
├── Core features in scope
├── Explicit out-of-scope items
└── User flows / API surface
2. NON-FUNCTIONAL REQUIREMENTS (How well must it do it?)
├── Scale: DAU, requests per second, data volume
├── Latency: P50, P99 read/write targets
├── Availability: 99.9%? 99.99%? (8.7 hrs vs 52 min downtime/year)
└── Consistency: Strong, eventual, or bounded staleness?
3. CONSTRAINTS
├── Existing infrastructure or technology mandates
├── Budget / cost sensitivity
├── Regulatory / compliance requirements
└── Team expertise and operational capacity
💡 Pro Tip: In interviews, treat requirements gathering as demonstrating thoroughness, not uncertainty. When you ask "What's the expected read-to-write ratio?" you're not stalling — you're showing that you understand the question's answer changes the architecture. Interviewers consistently rate this habit as a strong positive signal.
🧠 Mnemonic: Use FACS to remember your requirements categories: Functional, Availability, Consistency, Scale. Run through FACS before drawing a single box.
Pitfall 5: Treating CAP Theorem as a Checklist
The CAP theorem states that a distributed system can guarantee at most two of three properties simultaneously: Consistency (all nodes see the same data at the same time), Availability (every request receives a response), and Partition Tolerance (the system continues operating despite network partitions).
The common mistake is reducing CAP to a label: "We chose AP" or "This is a CP system" — and stopping there. This treats a nuanced design lens as a binary classification stamp, which obscures the real decisions being made.
⚠️ Common Mistake: Confidently stating "We'll use an AP database because we need availability" without discussing what kind of consistency you're sacrificing, which operations can tolerate staleness, and how stale is acceptable.
The critical insight is that partition tolerance is not optional in distributed systems. Networks partition; it is a physical reality. The real decision is: during a partition, do you prioritize consistency or availability? And crucially, this decision can vary by operation within the same system.
CAP DURING A NETWORK PARTITION: The Real Decision
┌─────────────┐ (partition) ┌─────────────┐
│ Node A │ ═══════════✂══════════════ │ Node B │
│ (has data) │ │ (stale?) │
└─────────────┘ └─────────────┘
Option 1: Choose CONSISTENCY (CP behavior)
─────────────────────────────────────────
Node B refuses reads/writes until partition heals.
✅ Data is never stale
❌ System is partially unavailable
Example use case: bank account balances, inventory levels
Option 2: Choose AVAILABILITY (AP behavior)
────────────────────────────────────────────
Node B serves reads from its potentially stale data.
✅ System remains available
❌ Users may see outdated information
Example use case: social media likes, shopping cart contents
The mature way to apply CAP is to think at the operation level, not the system level. A social platform might:
- Require strong consistency for financial transactions (charging a credit card must not process twice)
- Accept eventual consistency for like counts (showing 10,247 instead of 10,248 briefly is harmless)
- Prioritize availability for feed reads (showing a slightly stale feed is better than showing nothing)
## Illustrating per-operation consistency decisions in a single system
class SocialPlatformDB:
def increment_like_count(self, post_id: str) -> None:
"""
Eventual consistency is acceptable here.
We use an eventually consistent counter (e.g., DynamoDB with
relaxed read consistency). A like count that's off by a few
for seconds causes no real harm.
"""
self.dynamo.update_item(
Key={"post_id": post_id},
UpdateExpression="ADD like_count :inc",
ExpressionAttributeValues={":inc": 1},
# No ConditionExpression — we accept eventual consistency
)
def deduct_account_balance(self, user_id: str, amount: int) -> bool:
"""
Strong consistency is REQUIRED here.
We use a conditional write — this operation only succeeds
if the current balance is sufficient, enforcing consistency.
A race condition here means a user could overdraft.
"""
try:
self.dynamo.update_item(
Key={"user_id": user_id},
UpdateExpression="SET balance = balance - :amt",
ConditionExpression="balance >= :amt", # Enforces consistency
ExpressionAttributeValues={":amt": amount},
)
return True
except self.dynamo.ConditionalCheckFailedException:
return False # Balance insufficient — transaction safely rejected
This code demonstrates that the same application might use the same database system with different consistency settings depending on the operation's requirements — a nuanced application of CAP thinking.
💡 Mental Model: Think of CAP not as a label for your database, but as a conversation about each user-facing guarantee. For every piece of data your system manages, ask: "If two nodes disagree on this value during a network partition, what's the cost of showing the stale value vs. the cost of refusing to respond?"
Bringing It All Together: The Pitfall Prevention Checklist
These five pitfalls share a common root: they all involve reaching for an answer before fully understanding the question. Over-engineering skips requirement validation. Ignoring failures skips scenario analysis. Vertical-only scaling skips distribution thinking. Skipping requirements skips constraints. Misapplying CAP skips nuance.
The antidote in every case is the same: slow down, ask more questions, and reason explicitly about trade-offs.
📋 Quick Reference Card: Pitfall Prevention
| ⚠️ Pitfall | 🔍 Warning Sign | ✅ Prevention Strategy |
|---|---|---|
| 🏗️ Over-engineering | Drawing 10+ services for an MVP | Ask: "What's the simplest thing that could work?" |
| 💥 Ignoring failures | No retry/timeout/fallback logic | Apply failure mode checklist per component |
| 📈 Vertical-only scaling | "We'll just get a bigger server" | Design stateless services from the start |
| ❓ Skipping requirements | Designing before asking questions | Run FACS before touching the whiteboard |
| 🗂️ CAP as checklist | "We chose AP" with no nuance | Apply CAP per operation, not per system |
Every expert system designer was once a developer who made these exact mistakes. What separates them now is not that they avoid complexity — it's that they've learned to recognize when complexity is earned versus when it's premature. You now have the patterns to spot these pitfalls in your own thinking, which is the first and most important step toward avoiding them.
Key Takeaways and What Comes Next
You started this lesson as a developer who writes code. You're ending it as someone who thinks in systems. That's not a small shift — it's the difference between building a feature and designing the infrastructure that makes thousands of features possible at once. Before you move forward into the sub-topics ahead, this section gives you a durable mental scaffold: a checklist, a vocabulary recap, three powerful lenses for evaluating any design, and a daily habit that will compound your skills faster than any other practice.
Let this section be your anchor. Come back to it before mock interviews, before whiteboard sessions, and before you propose a new architecture at work.
The Pre-Diagram Checklist: Questions Every System Designer Must Answer
One of the most common mistakes developers make — covered in detail in the previous section — is jumping straight to drawing boxes and arrows before they understand what they're actually building. The checklist below is your circuit breaker. Internalize it, and you'll never waste whiteboard space designing the wrong system again.
🎯 Key Principle: No diagram before requirements. Ever.
Here is the checklist in full:
=== System Design Pre-Diagram Checklist ===
[ ] 1. SCALE
- How many users? (1K, 1M, 1B?)
- Read-heavy or write-heavy?
- Expected data volume in Year 1 vs. Year 5?
[ ] 2. AVAILABILITY
- What is the acceptable downtime? (99.9% = ~8h/year)
- Are there geographic distribution requirements?
- Is this latency-sensitive? (real-time vs. eventual)
[ ] 3. CONSISTENCY
- Does every user need to see the exact same data at all times?
- Can we tolerate stale reads for better performance?
- What is the cost of showing a user incorrect data?
[ ] 4. CONSTRAINTS
- Budget? Cloud-native or on-prem?
- Team size and expertise?
- Any regulatory requirements (GDPR, HIPAA)?
[ ] 5. CORE USE CASES
- What are the top 3 things users do most often?
- What is the single most critical path in this system?
- What does failure look like for each use case?
[ ] 6. API BOUNDARIES
- Who are the clients? (browser, mobile, other services)
- What are the primary API contracts?
- How does data flow between client and server?
This checklist maps directly to the structured approach you learned in Section 3: Thinking in Systems. Notice how it forces you to answer questions about users, data, and failure modes before you've touched a single building block. That ordering is intentional and critical.
💡 Pro Tip: In a 45-minute system design interview, spend the first 8–10 minutes purely on this checklist. Interviewers consistently rate candidates higher when they demonstrate requirements discipline, even if the final design is imperfect.
Recap: Core Building Blocks and Their Real Infrastructure Counterparts
In Section 2, you were introduced to the components that appear in virtually every large-scale system. Here is a consolidated reference that maps each conceptual building block to the actual infrastructure choices you'll encounter in practice.
📋 Quick Reference Card: Building Blocks → Real Infrastructure
| 🧱 Building Block | 🔧 What It Does | 🌐 Real-World Examples |
|---|---|---|
| 🔀 Load Balancer | Distributes traffic across servers | AWS ALB, NGINX, HAProxy |
| 🗄️ Database | Persists structured or unstructured data | PostgreSQL, MySQL, DynamoDB |
| ⚡ Cache | Serves hot data at low latency | Redis, Memcached, CDN edge cache |
| 📨 Message Queue | Decouples producers and consumers | Kafka, RabbitMQ, AWS SQS |
| 🌍 CDN | Serves static assets close to users | Cloudflare, AWS CloudFront, Akamai |
| 🏗️ API Gateway | Single entry point for client requests | Kong, AWS API Gateway, Traefik |
| 📦 Object Storage | Stores large unstructured files (images, video) | AWS S3, GCS, Azure Blob Storage |
| 🔍 Search Index | Enables full-text and faceted search | Elasticsearch, OpenSearch, Algolia |
| 📊 Monitoring | Observes system health and behavior | Datadog, Prometheus + Grafana |
Each of these building blocks isn't just a technology to name-drop — it's a trade-off surface. When you choose Redis over a database for caching, you're trading durability for speed. When you choose Kafka over a simple HTTP call between services, you're trading simplicity for resilience and decoupling. Every architectural decision is a negotiation.
💡 Real-World Example: Twitter (now X) famously uses a combination of Redis for timeline caching, Kafka for event streaming between services, and a mix of MySQL and Manhattan (their own distributed key-value store) for persistent storage. No single building block handles everything. Real systems are composed.
The Three Lenses of System Design
Every design decision you make in system design can be evaluated through three fundamental lenses. Think of them as the three axes of a coordinate system: any architecture exists somewhere in this three-dimensional space, and your job is to consciously position it based on requirements.
RELIABILITY
|
|
| * Your System Lives Somewhere Here
|
+-------------------> SCALABILITY
/
/
/
MAINTAINABILITY
Lens 1: Scalability
Scalability is the system's ability to handle increasing load without degrading performance. There are two primary axes:
- Vertical scaling (scale up): Add more power to existing machines (more CPU, more RAM). Fast to implement, but has physical limits and creates a single point of failure.
- Horizontal scaling (scale out): Add more machines. Harder to implement correctly, but theoretically unbounded and more resilient.
The URL shortener walkthrough in Section 4 demonstrated this directly: a single server handles early traffic fine, but once you add a load balancer and multiple application servers, you've made a horizontal scaling decision that unlocks a new tier of capacity.
Lens 2: Reliability
Reliability is the system's ability to continue functioning correctly even when components fail — and components will fail. The question is never if but when. Reliability is built through:
- Redundancy: Run multiple instances so one failure doesn't cause an outage.
- Graceful degradation: When a subsystem fails, the system delivers reduced functionality rather than total failure.
- Timeouts and circuit breakers: Prevent a slow dependency from cascading into a full system crash.
🎯 Key Principle: A system that is fast but unreliable is not production-ready. Reliability is not a feature — it's a constraint.
Lens 3: Maintainability
Maintainability is the ease with which engineers can understand, modify, and extend the system over time. It is often the most underweighted lens in interview settings, but it's the one that matters most in real engineering careers. Maintainability is improved by:
- Operability: Can the on-call engineer understand what's happening when the system breaks at 3 a.m.?
- Simplicity: Is the system as simple as it can be while still meeting its requirements?
- Evolvability: Can the system accommodate new requirements without requiring a full rewrite?
🧠 Mnemonic: SRM — Scalability handles growth, Reliability handles failure, Maintainability handles time. A great system engineer never optimizes just one dimension.
The following code snippet illustrates how maintainability thinking affects even low-level implementation choices. Compare two approaches to writing a cache-aside pattern:
## ❌ Unmaintainable: logic scattered, no abstraction
def get_user_profile(user_id):
key = f"user:{user_id}"
cached = redis_client.get(key)
if cached:
return json.loads(cached)
result = db.query(f"SELECT * FROM users WHERE id = {user_id}")
redis_client.setex(key, 300, json.dumps(result))
return result
## ✅ Maintainable: abstracted, configurable, testable
CACHE_TTL_SECONDS = 300
class UserRepository:
def __init__(self, db, cache):
self.db = db
self.cache = cache
def get_by_id(self, user_id: int) -> dict:
"""Fetches user with cache-aside pattern."""
cache_key = f"user:{user_id}"
# Attempt cache read first
cached = self.cache.get(cache_key)
if cached:
return json.loads(cached)
# Cache miss: read from database
user = self.db.find_user(user_id)
if user:
# Populate cache for future reads
self.cache.setex(cache_key, CACHE_TTL_SECONDS, json.dumps(user))
return user
Both snippets implement the same caching strategy. The second is maintainable because it's injectable (you can swap the cache or DB implementation in tests), configurable (TTL is a named constant), and readable (the docstring explains intent). System design thinking applies at every level of the stack.
How the Next Sub-Topics Build on This Foundation
This lesson was the foundation. The two sub-topics that follow are the first floor of the building, and they connect directly to what you've learned here.
Key Concepts & Terminology
Everything in this lesson used vocabulary that the Key Concepts & Terminology sub-topic will formalize. Terms like CAP theorem, eventual consistency, idempotency, sharding, and replication were mentioned or implied throughout — now you'll get precise, rigorous definitions and be able to use them with the confidence that interviewers expect.
❌ Wrong thinking: "I'll pick up the vocabulary as I go. I understand the concepts intuitively." ✅ Correct thinking: "Precise vocabulary lets me communicate trade-offs quickly and clearly. Interviewers and teammates use these terms as a shared language."
Networking Basics
Every building block you learned in Section 2 communicates over a network. The URL shortener in Section 4 relied on DNS to resolve short URLs, HTTP to serve redirects, and TCP to ensure reliable data delivery. Networking Basics will give you the model for understanding how data moves between every component you've now named and placed on a diagram.
Lesson: Foundations Next: Key Concepts Next: Networking Basics
================= ================== ===================
- Why design matters -> - CAP Theorem -> - DNS & Resolution
- Building blocks -> - Consistency models -> - TCP vs UDP
- Mental framework -> - Replication types -> - HTTP/HTTPS
- URL shortener demo -> - Sharding strategies -> - Load balancer protocols
- Common pitfalls -> - Idempotency -> - Network latency math
Think of this foundational lesson as giving you the map. Key Concepts & Terminology gives you the legend. Networking Basics shows you the roads. You cannot navigate effectively with any one of them alone.
The Daily Sketch Habit: Your Most Powerful Practice Tool
Knowledge without practice atrophies. The single most effective habit you can build right now is a daily architecture sketch. Here's the protocol:
🎯 The Daily Sketch Protocol
- Pick a familiar product (Instagram, Spotify, Google Docs, GitHub)
- Run the pre-diagram checklist — write your answers, even briefly
- Sketch the architecture using the building blocks you now know
- Apply the three lenses — mark where your design is strong and where it's weak
- Identify one thing you don't know and research it (this is how your knowledge compounds)
This habit takes 15–20 minutes per day. After 30 days, your pattern recognition will be dramatically sharper. You'll start seeing systems when you use products — "this search must be hitting an Elasticsearch cluster," or "this must be using a CDN because the latency is too consistent across regions."
Here's a minimal Python helper you can use to structure your daily sketching notes programmatically — treating each component as a node in a graph that you can query and review:
from dataclasses import dataclass, field
from typing import List
@dataclass
class Component:
"""Represents a single architectural component in a system sketch."""
name: str
type: str # e.g., 'cache', 'database', 'load_balancer'
connects_to: List[str] = field(default_factory=list)
notes: str = ""
@dataclass
class SystemSketch:
"""A lightweight structure for capturing a daily architecture sketch."""
system_name: str
components: List[Component] = field(default_factory=list)
def add_component(self, component: Component):
self.components.append(component)
def summarize(self):
"""Prints a readable summary of the system sketch."""
print(f"\n=== {self.system_name.upper()} ===")
for c in self.components:
connections = ', '.join(c.connects_to) if c.connects_to else 'none'
print(f" [{c.type.upper()}] {c.name}")
print(f" -> Connects to: {connections}")
if c.notes:
print(f" -> Notes: {c.notes}")
## Example: sketching a simplified Instagram-like feed system
sketch = SystemSketch("Instagram Feed (simplified)")
sketch.add_component(Component(
name="API Gateway",
type="gateway",
connects_to=["Feed Service", "Auth Service"],
notes="Single entry point; handles rate limiting"
))
sketch.add_component(Component(
name="Feed Service",
type="service",
connects_to=["Redis Cache", "Post Database", "Message Queue"],
notes="Generates and serves user feed"
))
sketch.add_component(Component(
name="Redis Cache",
type="cache",
connects_to=[],
notes="Pre-computed feeds for active users; TTL = 60s"
))
sketch.add_component(Component(
name="Post Database",
type="database",
connects_to=[],
notes="Sharded by user_id; read replicas for feed queries"
))
sketch.summarize()
This isn't production code — it's a thinking tool. Use it to externalize your daily sketches, spot missing connections, and build a library of systems you've designed. After 30 sketches, review them: you'll see patterns, gaps, and your own growth.
What You Now Understand That You Didn't Before
Let's be explicit about the transformation this lesson has produced. Before this lesson, you likely thought about software systems as code that runs. After this lesson, you think about systems as components that communicate, fail, scale, and evolve. Here is the full summary of that shift:
📋 Quick Reference Card: Before vs. After This Lesson
| 🔴 Before | 🟢 After |
|---|---|
| 🤷 "I'll design it and add scale later" | 🎯 Requirements first, always |
| 🤷 "A database is a database" | 🧠 Right storage for the right access pattern |
| 🤷 "We'll fix failures with better code" | 🔧 Design for failure from the start |
| 🤷 "Architecture is for senior engineers" | 📚 Every developer makes architectural decisions |
| 🤷 "Caching = making things faster" | ⚡ Caching = a trade-off between consistency and speed |
| 🤷 "Microservices are always better" | 🧩 Start monolith; split when the pain is real |
⚠️ Critical Point to Carry Forward: System design is not about finding the perfect architecture. There is no perfect architecture. System design is about making explicit, justified trade-offs that align with the requirements you gathered before you drew a single line. An interviewer who hears you say "I chose this approach because it favors availability over consistency, which matches our requirement for..." will evaluate you as a senior engineer, regardless of your current title.
⚠️ Critical Point to Carry Forward: The building blocks in this lesson are not a shopping list. Every component you add to a diagram introduces operational complexity, latency, and failure modes. The best system designers are ruthless about not adding components until the requirements demand them.
Practical Next Steps
Here are three concrete actions to take before you move into the next sub-topic:
🔧 Action 1: Do your first daily sketch today. Pick any product you used in the last 24 hours. Spend 15 minutes sketching its architecture using the building blocks from Section 2. Don't worry about getting it right — getting it down is the practice.
📚 Action 2: Review the pre-diagram checklist with a real system in mind. Take a system you've worked on professionally and walk the checklist. You'll likely discover requirements that were never formally stated — and assumptions that drove architectural decisions nobody can remember making.
🎯 Action 3: Begin Key Concepts & Terminology with the three lenses as your filter. As you learn each new term, immediately ask: is this concept primarily about scalability, reliability, or maintainability? Most terms live primarily in one lens, and categorizing them this way will help you apply them faster in design conversations.
🤔 Did you know? Amazon's early engineering teams operated under a "two-pizza rule" for team size — if a team needed more than two pizzas to be fed, it was too large. This organizational constraint directly influenced system design: smaller teams build more loosely coupled services, because tight coupling requires constant coordination. Architecture and organization reflect each other.
You now have the foundation. Everything that follows in this course — load balancing strategies, database selection, caching patterns, distributed consensus algorithms — is a specific instance of what you've learned here. Come back to this section often. The fundamentals are the edge.