URL Shortener (e.g. TinyURL)
Design encoding, redirection, analytics storage, and handle 100M+ URLs with low-latency reads.
SPACED REPETITION · 15 practice questions
Make this lesson stick.
Try 3 questions now. No account needed. Sample answers aren't saved.
or sign in to practice all 15A hash map that ten billion clicks lean on
The interviewer says, "Design TinyURL." You ask a few questions and agree on numbers: 100 million new links a day, each clicked about 100 times over its life, so 10 billion redirects a day. Links never expire unless the owner asks, because people print them on posters and product packaging.
Your first sketch is small and honest. One PostgreSQL server. One table, links(id BIGSERIAL PRIMARY KEY, url TEXT). The short key is the id written in base 62. A redirect runs SELECT url FROM links WHERE id = decode(key) and answers with an HTTP redirect.
It is correct. Now put the load on it. Assume the busiest hour runs at 3× the daily average:
| The box must handle | Arithmetic | Result |
|---|---|---|
| New links | 100,000,000 ÷ 86,400 s | 1,157/s average, about 3,500/s at peak |
| Redirects | 10,000,000,000 ÷ 86,400 s | 115,741/s average, about 347,000/s at peak |
| New data | 36.5 billion rows a year × 500 bytes | about 18 TB a year, kept forever |
| Failures | one machine | every printed link is dead while it is down |
Predict first: which row breaks first, and which one is not a problem at all?
Check your answer
The last row is broken on day one: a single box is a single point of failure for links that were printed and can never be changed. Among the load rows, writes are not the problem: a few thousand inserts per second is ordinary work for one well-provisioned primary. The read path breaks first: 347,000 lookups a second at peak is at or beyond what one primary can serve, and every one of them waits on the same machine. Storage comes next, because 18 TB a year outgrows one machine and makes every backup and rebuild slow.
So the whole system is a hash map from a 7-character key to a URL. The work is making that hash map answer 347,000 times a second, never lose an entry, and never hand out the same key twice. Each requirement puts pressure on the design, and each pressure has a standard move:
| # | Pressure in the requirements | Move | What it costs |
|---|---|---|---|
| 1 | Billions of short keys, minted on many servers at once | Leased counter ranges + base 62, or random keys | Coordination, or collision checks |
| 2 | 100 reads per write, and a few links are very hot | Cache-aside, write-through on create, a small in-process cache | Staleness you must bound |
| 3 | 18 TB a year, forever | Partition by a hash of the key, replicate each partition | Operations work, no range scans |
| 4 | Clicks are counted; destinations change or expire | Choose the status code and Cache-Control on purpose |
More traffic reaches you |
| 5 | Analytics without slowing redirects | Emit click events and aggregate off the hot path | Counts lag, duplicates to handle |
| 6 | Links expire or are taken down | Check expiry on every read, update the database before the cache, purge in batches | A cleanup job |
| 7 | Anyone can shorten anything | Validate, rate-limit, scan, and keep a takedown path | Friction for honest users |
This lesson assumes the estimation basics from Foundations of System Design and the caching and sharding vocabulary from Databases & Storage. The rate limiter this system needs is designed in depth in Design URL Shortener & Rate Limiter.
Before any box: pin down the contract
A URL shortener sounds too simple to need requirements. That is exactly why interviewers use it: the answers to a handful of questions change the design, and they want to see you ask them.
Functional: create a short link for a long URL (optionally with a custom alias and an expiry time); redirect GET /<key> to the long URL; show the owner click counts by day, country and referrer.
Non-functional, as numbers:
- Latency: redirects under 100 ms at p99, measured on our side from the moment the request reaches our edge. The user's own network round trip comes on top; the Tokyo exercise in Move 3 and the CDN section of Move 4 deal with it.
- Availability: 99.99% for redirects, which allows about 53 minutes of downtime a year. Creation can be 99.9%. If nobody can create links for five minutes, no printed poster breaks.
- Durability: once we have returned a link, we never lose it.
- Freshness: a new link works everywhere within a second or two, because people paste links into a chat and their friends click immediately.
The questions that change the design:
| Ask | If the answer is yes, the design changes because… |
|---|---|
| Do we count clicks? | Redirects can no longer be cached freely, and you need an event pipeline |
| Can links be edited, expire or be taken down? | Every cache layer needs a bound on how stale it can be |
| Custom aliases? | A second source of keys shares the namespace and needs validation |
| Must links be unguessable? | You need long random keys, not short sequential ones |
| Same URL twice → same link? | You need a lookup index on the long URL, plus a rule about owners |
| Users worldwide? | TLS at the edge, and regional caches and replicas |
On the deduplication question, the usual answer is no across users. Two people who shorten the same article want their own link, their own click counts and their own expiry. If the product wants "same user, same URL → same link", add a small index on (owner, hash of the URL). Call that deduplication, not idempotency; idempotency is about retries of the same request, and it comes back in Move 7.
The API is two endpoints:
POST /api/v1/links
Idempotency-Key: 7c1e9b52-4f0a-4d8e-9a57-2f64c0d1a3be
Content-Type: application/json
{"url": "https://example.com/spring/catalog?utm_source=poster", "alias": "spring-sale", "expiresAt": "2026-12-31T23:59:59Z"}
HTTP/1.1 201 Created
Location: https://sho.rt/spring-sale
GET /spring-sale
HTTP/1.1 302 Found
Location: https://example.com/spring/catalog?utm_source=poster
Cache-Control: private, max-age=60
Errors: 400 for a URL you refuse, 409 when the alias is taken, 429 when the caller is rate-limited, 404 for a key that never existed and 410 Gone for one that expired or was removed.
The data model is one table. Clicks do not live here:
CREATE TABLE links (
short_key VARCHAR(30) PRIMARY KEY, -- 7 characters when generated, aliases up to 30
long_url TEXT NOT NULL,
owner_id BIGINT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
expires_at TIMESTAMPTZ, -- NULL means never
status SMALLINT NOT NULL DEFAULT 0 -- 0 active, 1 disabled
);
CREATE INDEX links_by_owner ON links (owner_id, created_at);
CREATE INDEX links_by_expiry ON links (expires_at) WHERE expires_at IS NOT NULL;
The owner's "my links" page and the expiry purge (Move 6) each need their own access path, hence the two indexes. In a partitioned key-value store, the owner index becomes a separate table keyed by owner, and expiry uses the store's native TTL or a small table of keys bucketed by expiry day.
The numbers, and what each one sizes
Here is the full back-of-envelope, with every assumption visible:
| Quantity | Assumption and arithmetic | Result |
|---|---|---|
| Creates | 100 M/day ÷ 86,400 | 1,157/s, peak (×3) about 3,500/s |
| Redirects | 100 per create | 115,741/s, peak about 347,000/s |
| Row size | key 7 + URL 200 + owner 8 + timestamps 16 + status 1 = 232 bytes; round up for row headers and the index | 500 bytes on disk |
| Storage | 36.5 billion links a year × 500 bytes | 18.25 TB a year; 54.75 TB with 3 replicas |
| Ten years | × 10 | 182.5 TB; 547.5 TB with replicas |
| Redirect egress | 347,000/s × 500 bytes of headers | 174 MB/s, about 1.4 Gbit/s at peak |
| Cache | 500 M distinct links clicked per day × 300 bytes per entry | 150 GB; 300 GB with a replica |
| Database reads | 347,000/s × 5% misses, if the cache hits 95% | about 17,400/s at peak |
Read the table for what drives each tier. The redirect servers are sized by peak requests per second. The cache is sized by the working set, meaning the links people actually click on a given day. The database is sized by storage: after the cache it sees about 17,000 reads a second, which is not much, but 18 TB a year, forever, will not fit on one machine. Candidates often shard "for throughput" here. The honest reason is size.
Predict first: is a 6-character base-62 key enough for this system?
Check your answer
No. 62⁶ = 56.8 billion keys, and you create 36.5 billion a year, so the space runs out after about 568 days, a year and a half. Seven characters give 62⁷ = 3.52 trillion keys, about 96 years at 100 million a day. One character buys 62× the runway, which is why "just use 7" is the usual answer. Compare the key space with every key you will ever mint over the system's lifetime, not with one day's or one year's volume.
Move 1: Mint keys without a meeting
Pressure: dozens of servers create links at the same time, and each one needs a key nobody else has. Asking a central service for every single key would put a network round trip and a single point of failure on every create.
A key is a number in disguise
A short key is just an integer written with 62 symbols instead of 10:
ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
WIDTH = 7 # every generated key is exactly 7 characters
def encode(n: int, width: int = WIDTH) -> str:
if not 0 <= n < 62 ** width:
raise ValueError("id does not fit in the key width")
digits = []
while n:
n, r = divmod(n, 62)
digits.append(ALPHABET[r]) # least significant digit first
return "".join(reversed(digits)).rjust(width, "0")
def decode(key: str) -> int:
n = 0
for ch in key:
n = n * 62 + ALPHABET.index(ch)
return n
print(encode(125)) # 0000021 (2*62 + 1)
print(encode(62 ** 6)) # 1000000
print(encode(62 ** 7 - 1)) # ZZZZZZZ
assert all(decode(encode(n)) == n for n in range(0, 62 ** 7, 9_876_543_211))
Just as 10⁶ is "1000000" in decimal, 62⁶ is 1000000 in base 62. Padding to a fixed width means every generated key has exactly 7 characters, which will matter when custom aliases share the namespace.
Why 62 and not 64? Standard Base64 adds + and /. A / is a path separator, and + means a space in form-encoded query strings, so both invite escaping bugs. The URL-safe variant, base64url (RFC 4648 §5), uses - and _ and works fine in a path. Most teams pick base 62 anyway: letters and digits are easy to read aloud and type, and a double-click selects the whole key, while a - often splits it.
| Alphabet | 6 characters | 7 characters |
|---|---|---|
| Hex (16) | 16.8 million | 268 million |
| Base 62 | 56.8 billion | 3.52 trillion |
| base64url (64) | 68.7 billion | 4.40 trillion |
Three ways to get a number nobody else has
1. Hash the long URL. Take MD5 or SHA-256 of the URL and keep the first few characters. No coordination, and the same URL always gets the same key. The catch is truncation. MD5 has 128 bits, which is plenty; you throw almost all of them away. By the birthday bound, a space of N keys is more likely than not to contain a collision after about 1.18 × √N keys:
| Key | Space | 50% chance of a collision after | At 1,157 creates/s |
|---|---|---|---|
| 6 hex characters | 16.8 million | about 4,800 keys | 4 seconds; the space is full in 4 hours |
| 6 base-62 characters | 56.8 billion | about 281,000 keys | 4 minutes |
| 7 base-62 characters | 3.52 trillion | about 2.2 million keys | 32 minutes |
So collisions are not a corner case; at this volume they are routine. You must insert only if the key is absent, and on a clash with a different URL, hash again with a salt and retry. Once you retry, "same URL → same key" is no longer a pure function: you have to look up which salt that URL ended up with. And the free deduplication is usually the wrong behavior anyway, for the ownership reasons above. Hashing fits small systems that really want deduplication. At this scale it is the weakest option.
2. A counter. Unique by construction: no two creates get the same number. A single global counter, though, is a round trip and a single point of failure on every create. The move is to lease ranges. An allocator hands each server a block of 100,000 ids, and the server counts through its block locally. Here the allocator starts at 62⁶ so the first key is 1000000:
| Step | Event | Allocator hands out | Keys issued |
|---|---|---|---|
| 1 | Server A starts and asks for a block | block 0: ids from 56,800,235,584 | |
| 2 | Server B starts and asks | block 1: ids from 56,800,335,584 | |
| 3 | A creates two links | 1000000, 1000001 |
|
| 4 | B creates a link | 1000q0U |
|
| 5 | A crashes after 37 links, restarts, asks again | block 2: ids from 56,800,435,584 | next A key: 1000Q1O |
Server A's crash throws away 99,963 unused ids. That is harmless: nobody else will ever receive them, so nothing collides.
Why it works. Uniqueness reduces to one promise: the allocator never hands out the same block twice. Everything else is local counting. That promise costs one durable, atomic increment per block, for example UPDATE id_blocks SET next_id = next_id + 100000 RETURNING next_id - 100000 on a synchronously replicated database, or a compare-and-set in etcd or ZooKeeper. At 1,157 creates a second the allocator gets one request every 86 seconds.
Block size is a trade. With 20 create servers, each uses about 58 ids a second, so a fresh 100,000-id block lasts about 29 minutes. That is not how long the allocator can be down: when it fails, servers are partway through their blocks, and the first one typically runs dry within a minute and a half. The fix is double buffering. Each server holds a spare block and fetches a new spare in the background as soon as it starts using the old one, so every server always has at least one full block in hand: about 29 minutes of allocator outage before any create fails. The price is waste: a restart discards up to two blocks, so if all 20 servers restart once a day, each deploy throws away up to 4 million ids, 4% of daily usage, which trims up to about four years off a runway of roughly 95. Bigger blocks buy more outage tolerance for more waste.
⚠️ The allocator must not forget. If its database acknowledges a lease and then fails over to an asynchronous replica that never received it, the next server can be handed the same block, and two servers mint identical keys. So make every insert into links conditional (INSERT … ON CONFLICT DO NOTHING in PostgreSQL, or a DynamoDB put with attribute_not_exists(short_key)). Then a duplicate becomes a failed write and a retry with the next id, instead of silently overwriting someone's live link.
What about Snowflake-style ids (timestamp, worker id and sequence packed into 64 bits, the layout Twitter published in 2010)? They need no allocator, but in 2026 their values are near 2⁶¹, which is 11 base-62 characters. They are great database ids and too long for short links. Core Building Blocks covers id generators in general.
3. Random keys. Draw 7 random base-62 characters from a cryptographically secure generator and insert only if absent. The chance that a fresh key clashes equals the fraction of the space already used: about 1% after one year and about 10% after ten years at our volume. That means an occasional extra write, not a bottleneck. There is no coordination, and keys reveal nothing about order.
A key generation service (KGS) is the pre-computed version: fill a pool with random unused keys ahead of time and let each server take a batch. Redis's SPOP with a count pops that many random members in one atomic command. Creation never collides at request time. The costs are another service to run and replicate, keys lost when a server crashes holding a batch (harmless), and the same forgetting problem as the allocator: if the pool's store fails over and loses recent pops, keys get handed out twice. Keep the conditional insert as the referee. The pool only needs a few days of keys, not billions.
| Scheme | Unique because | Coordination | The key reveals | Main cost |
|---|---|---|---|---|
| Hash + truncate | retry on clash | none | nothing, but same URL → same key | collisions certain at scale |
| Leased counter ranges | allocator never repeats a block | one call per block | creation order and volume | enumerable keys |
| Random + conditional insert | database rejects duplicates | none | nothing | retries grow as the space fills |
| Pre-generated pool (KGS) | pool hands each key out once | pool service | nothing | another service to run |
Sequential keys leak
With counter keys, 1000000 is followed by 1000001. Anyone can walk the space, scrape every link and estimate your daily volume. Two popular fixes don't work:
- Shuffling the alphabet changes how keys look, not how they are laid out. Consecutive ids still give keys that differ only in the last character, so a handful of your own links reveals the order of the shuffled alphabet, and the rest can be walked.
- XOR with a secret flips the same bits in every id, so ids that agree in their high bits still agree after the XOR. Ten consecutive ids usually land on keys that differ only in their last one or two characters.
What does work is a keyed permutation of the id space: a one-to-one shuffle that only the secret holder can compute, so consecutive ids land far apart and nobody can predict the next key.
Optional: a keyed permutation in 20 lines
A four-round Feistel network is a permutation of 42-bit numbers for any round function. "Cycle-walking" keeps re-applying it until the result lands inside [0, 62⁷), which keeps it a permutation of exactly the 7-character keys.
import hashlib, hmac
SPACE = 62 ** 7 # all 7-character keys
HALF = 21 # 2**42 is the smallest power of two >= 62**7
MASK = (1 << HALF) - 1
def _feistel(secret: bytes, x: int) -> int:
left, right = x >> HALF, x & MASK
for rnd in range(4):
mac = hmac.new(secret, f"{rnd}:{right}".encode(), hashlib.sha256).digest()
left, right = right, left ^ (int.from_bytes(mac[:8], "big") & MASK)
return (left << HALF) | right
def scramble(secret: bytes, n: int) -> int:
"""A keyed one-to-one shuffle of [0, 62**7): distinct ids give distinct keys."""
x = _feistel(secret, n)
while x >= SPACE: # "cycle-walk" back into range
x = _feistel(secret, x)
return x
secret = b"keep-me-in-a-secrets-manager"
print([scramble(secret, n) for n in range(3)]) # three scattered numbers; encode() each one
Since 2⁴² is only 1.25× larger than 62⁷, _feistel runs about 1.25 times per call on average. The same construction checked over a 62³ space produced every value exactly once.
A permutation hides order and volume. It does not make links unguessable. After ten years, 365 billion of the 3.52 trillion possible 7-character keys are in use, so one random guess in ten hits a real link, however well you shuffled. Unguessable means sparse:
| Random key length | Possible keys | One random guess hits a live link (365 billion live) |
|---|---|---|
| 7 | 3.5 × 10¹² | about 1 in 10 |
| 8 | 2.2 × 10¹⁴ | about 1 in 600 |
| 10 | 8.4 × 10¹⁷ | about 1 in 2.3 million |
| 12 | 3.2 × 10²¹ | about 1 in 8.8 billion |
| 22 (about 131 bits) | 2.7 × 10³⁹ | effectively never |
Your turn: the interviewer changes one requirement: "Keys should be as short as possible. We'll only create 5 million links a year." What do you build, and what do you tell them about guessing?
Check your answer
Ten years is 50 million links. 62⁵ = 916 million, so 5 characters are enough: random 5-character keys with a conditional insert (the space would be at most 5.5% full, so retries stay rare), or leased counter ranges plus a keyed permutation over 62⁵ with small blocks of about 1,000 ids. At only 13,700 creates a day, 100,000-id blocks discarded on every restart would burn through 62⁵ in about 15 months. Then say the uncomfortable part out loud: at 5.5% density, one random guess in 18 hits a live link. "Shortest possible" and "private" pull in opposite directions. If some links are private, give those long random keys or put them behind a login.
💡 Say it like this: "I'll mint keys from leased counter ranges. Each server takes 100,000 ids at a time, so creates never wait on a central service, and a crash wastes at most the rest of one block. Seven base-62 characters give 3.5 trillion keys, about 96 years at 100 million a day. If keys must not be enumerable, I'll put the id through a keyed permutation; if they must be unguessable, I'll switch to long random keys."
Likely follow-up: "What if the allocator is down?" Each server keeps issuing from its current and spare blocks, at least half an hour at these numbers. Redirects never touch the allocator at all.
Move 2: Answer redirects from memory
Pressure: 347,000 lookups a second at peak, and a few links far hotter than the rest.
The read path
The redirect service checks a cache first and falls back to the database on a miss (cache-aside). Two details decide whether it is correct: the cached value carries the expiry time and the status (so a disabled link answers 410), and "this key does not exist" is cached as a real value.
from datetime import datetime, timedelta, timezone
MISSING = ("missing",) # a real value, so it can't be confused with "no entry"
DEFAULT_TTL = timedelta(hours=24)
MISSING_TTL = timedelta(seconds=30)
def resolve(key, cache, db, now):
"""Return (status, url). cache.get gives None when it has no entry."""
entry = cache.get(key, now)
if entry is None: # cache miss: ask the database
row = db.get(key) # (url, expires_at, disabled) or None
entry = row if row is not None else MISSING
ttl = DEFAULT_TTL if row is not None else MISSING_TTL
if row is not None and row[1] is not None:
ttl = min(ttl, row[1] - now) # never cache past expiry
if ttl > timedelta(0):
cache.set(key, entry, ttl, now)
if entry == MISSING:
return 404, None
url, expires_at, disabled = entry
if disabled or (expires_at is not None and expires_at <= now): # on hits too
return 410, None
return 302, url
class DictCache: # stand-in for Redis
def __init__(self):
self.data = {}
def get(self, key, now):
value, dies = self.data.get(key, (None, now))
return value if dies > now else None
def set(self, key, value, ttl, now):
self.data[key] = (value, now + ttl)
t0 = datetime(2026, 9, 24, 12, 0, tzinfo=timezone.utc)
db = {"1000000": ("https://example.com/launch", t0 + timedelta(minutes=10), False),
"1000001": ("https://phish.example", None, True)} # taken down
cache = DictCache()
print(resolve("1000000", cache, db, t0)) # (302, ...)
print(resolve("1000000", cache, db, t0 + timedelta(minutes=11))) # (410, None)
print(resolve("1000001", cache, db, t0)) # (410, None)
print(resolve("nope123", cache, db, t0)) # (404, None)
Four decisions are packed in there:
- The TTL never outlives the link. A link that expires in 10 minutes is cached for at most 10 minutes, and the expiry is checked on every hit as well. A flat one-hour TTL with no expiry check on hits would keep redirecting an expired link for up to 50 extra minutes.
- Misses are cached as a real marker. When clients hammer the same unknown keys (a mistyped link shared widely, a scraper retrying), each one reaches the database once per 30 seconds instead of on every request. Bots probing random keys defeat this, because they almost never repeat a key; the defence there is rate limiting. The marker must not be something falsy like
"". Code that testsif cached:treats an empty string as a miss and sends every probe straight to the database. - Write through on create. When a link is created, the create service also puts it in the cache. That overwrites any "missing" marker left by a bot that guessed the next key, and the first click does not depend on a database replica having caught up.
- Mappings rarely change, so the TTL can be long. 24 hours is fine because edits, expiry and takedowns handle the cache explicitly (Move 6). The TTL is a safety net, not the invalidation plan.
How big, and how good
Don't size the cache as "20% of all links". After one year that is 7.3 billion entries, about 2.2 TB, and it grows forever. Size it by the working set. If 500 million distinct links get clicked on a given day, at about 300 bytes per entry including Redis's per-key overhead, that is 150 GB, or 300 GB with a replica for each shard: a handful of cache nodes.
The hit rate is a measurement, not a law. You can assume one, but say what happens if you are wrong:
| Hit rate | Database reads at peak |
|---|---|
| 90% | 34,700/s |
| 95% | 17,400/s |
| 99% | 3,500/s |
"I'll assume 95%, size the database tier to survive 90%, and watch the real number" is a strong sentence in an interview.
Redis or Memcached? Both are fine for get, set and per-key expiry. Memcached takes an expiry time on every item it stores, so TTL is not a reason to pick Redis. Redis adds replication and optional persistence, so a failed cache node can be replaced by a warm replica instead of an empty one. That matters here, as the failure drill shows.
The celebrity link
Predict first: your cache hits 95%. A campaign sends 20% of all traffic to one link: about 70,000 clicks a second at peak. Does the hit rate go up or down, and what is actually in danger?
Check your answer
The hit rate goes up, because that link is always cached, and the database barely notices. The danger is the one Redis shard that holds that key. A cluster spreads keys across shards, not the load of a single key, so adding shards does not help. Adding read replicas of that shard spreads the load, but only if clients read from replicas.
The standard move is a tiny in-process cache in each redirect server: a few thousand entries with a 5–10 second TTL. With 50 redirect servers and a 10-second TTL, Redis sees about 5 requests a second for the viral key instead of 70,000. The cost is that an edit or takedown can take up to 10 seconds to reach every server; a takedown can also broadcast an explicit invalidation.
Pair it with single-flight: when a hot key is missing, one request per server fetches it while the others wait for that answer. Without it, a hot key that expires or gets evicted sends thousands of identical database reads at the same moment, a cache stampede. Databases & Storage covers eviction policies and stampede protection in more depth.
Move 3: Split the table by the key
Pressure: 18 TB a year, kept forever. The access pattern is narrow: get by key, insert if absent, and list by owner.
Which database
Two answers are equally defensible, and saying why is worth more than the pick:
- A partitioned key-value store such as DynamoDB or Cassandra. Partitioning by key is built in. DynamoDB supports conditional puts (
attribute_not_exists). Cassandra supportsIF NOT EXISTS, but that runs a consensus round (a "lightweight transaction"), which is much slower than a plain write. - Hash-sharded PostgreSQL or MySQL. Familiar, with a real unique constraint and
INSERT … ON CONFLICT DO NOTHINGin PostgreSQL. You own the routing layer and the resharding. In MySQL, giveshort_keya binary (_bin) collation: the default collation ignores case, and base-62 keys such asaB3andAb3are different links.
💡 Say it like this: "The table is a pure key lookup that grows 18 TB a year. I'd use a managed key-value store partitioned on the short key, so partitioning is someone else's job. Sharded Postgres works too if that is what the team runs well. Nothing in this design needs joins."
Choosing the partition key
| Partition by | What happens with counter-generated keys | Verdict |
|---|---|---|
| Key ranges | New keys are consecutive, so every insert lands in the last range, and so do most reads, because new links are the hot ones | One hot partition |
| First character of the key | Every key from 1000000 to 1ZZZZZZ starts with 1: 56.8 billion keys, a year and a half of traffic, on one partition |
Same hotspot, slower |
| Owner id | A redirect only knows the key, so it would ask every partition; big accounts become hot | Scatter-gather on the hottest path |
| Hash of the key | Keys spread evenly, and a redirect computes exactly one partition | ✓ No range scans, which you never need |
How many partitions? Storage decides. Ten years is 182.5 TB. If you keep at most 4 TB per node so that backups and replica rebuilds stay quick (an assumption), that is about 46 storage nodes (shards), each replicated 3 times. The 17,400 database reads a second at peak then come to under 400 per node. The replicas here are for survival, not throughput.
Don't route with hash(key) % N, because changing N moves almost every key. Create many logical partitions up front (say 1,024), map them onto the nodes you have (a few in year one, about 46 by year ten), and move whole partitions as you grow. Consistent hashing is the general tool; see Advanced Topics & Final Prep.
The brand-new link
Replicas apply the primary's changes asynchronously, typically milliseconds to seconds behind. A redirect that misses the cache and reads a lagging replica answers "not found" for a link that exists. Write-through on create covers the common case. For the rest, on a replica miss, check the primary before answering 404. That fallback has a price. A replica cannot tell "lagging" from "never existed", so every probe for a bogus key also reaches the primary. That is why the "missing" answer gets cached and bots get rate-limited.
Your turn: the interviewer adds: "Users are worldwide. A friend in Tokyo must be able to open a link seconds after it is created in Berlin." What changes?
Check your answer
- Terminate TLS near users. Tokyo to an origin in Frankfurt is about 9,300 km, so light in fiber needs about 93 ms for a round trip, and real paths are slower. A fresh HTTPS connection over TCP and TLS 1.3 takes three round trips before the response arrives: at least 280 ms. Through a nearby CDN edge that already holds a warm connection to the origin, the handshakes are local, and only the request itself makes the long trip.
- Regional read stacks. Put a redirect service, a cache and a read replica in each region, so almost every redirect stays in the user's region. Writes go to one home region, say Frankfurt; creates are 1% of traffic and can afford an extra cross-region round trip.
- Read-after-create across regions. On a regional miss, ask the home region before answering 404, and cache "missing" only after the home region confirms it. The create path must also overwrite any "missing" marker in every region's cache (or regional negative TTLs must be a few seconds): someone who tried a custom alias before registering it leaves a 30-second marker behind, and the new link would 404 in that region.
- If the home region fails, redirects keep working from the other regions' replicas while creation pauses or fails over. Say it plainly: redirects are the product, and creation may degrade.
An equally valid variant lets each region accept writes and lease id blocks from its own ranges, so two regions can never mint the same key, with a multi-region table replicating everything (DynamoDB global tables, for example, reconcile concurrent writes with last-writer-wins by default). The cost moves to edits, which need a conflict rule, and to custom aliases: two regions can accept the same alias at the same moment, each conditional write succeeds against its own Region's copy, and last-writer-wins silently drops one of the two links. Route alias creation to one home region, or use a store that evaluates conditions globally (DynamoDB's multi-Region strong consistency mode, for example).
Move 4: Decide who gets to skip you
Pressure: clicks are counted, and destinations can change, expire or be taken down. Every redirect answered from someone else's cache is a click you don't see and a change that doesn't arrive.
Predict first: you return 301 with no cache headers. One million different people click a link, three times each. Which of those clicks can you miss?
Check your answer
Repeats, and possibly first clicks too. A header-less 301 is heuristically cacheable by any cache. Browsers may reuse it, so up to 2 million repeat clicks can go unseen, and a browser holding the old redirect won't see a changed destination. Shared caches (your CDN, a corporate proxy) may store it as well and answer new visitors, whose first clicks then never reach you. So the popular line "with 301, analytics stop after the first click" is wrong in both directions: new visitors normally still reach you, but a shared cache can hide even first clicks. Explicit headers make it predictable: private keeps shared caches out, and max-age bounds browser reuse.
Status codes, precisely
| Code | Name (RFC 9110) | Cacheable without explicit headers? | Request method on redirect |
|---|---|---|---|
| 301 | Moved Permanently | Yes, heuristically | May change POST to GET |
| 302 | Found | No | May change POST to GET |
| 307 | Temporary Redirect | No | Preserved |
| 308 | Permanent Redirect | Yes, heuristically | Preserved |
Source: RFC 9110, section 15.4. Shortener redirects are GETs, so method preservation does not matter here. 302 is the usual pick because it is not cached unless you say so.
The real lever is Cache-Control
Here is what three real shorteners sent when we checked with curl in September 2026 (this can change):
| Service | Status | Cache-Control |
|---|---|---|
| bit.ly | 301 | private, max-age=90 |
| TinyURL | 301 | no-store, no-cache, must-revalidate |
| t.co | 301 | private,max-age=300 |
All three use 301, and all three still control caching. private stops shared caches such as CDNs from storing the response. max-age bounds how long a browser may reuse it. no-store forbids storing it at all. So the rule is: the status code states the meaning; Cache-Control decides who may skip you, and for how long. Whatever max-age you pick is also the longest an edit, expiry or takedown can go unseen by a browser that already has the redirect, and the window in which repeat clicks go uncounted.
Send explicit headers on 404 and 410 too. Both are heuristically cacheable, and a cached 404 for a key that gets created a second later breaks a brand-new link.
Caching at the CDN edge
Letting the CDN cache redirects is tempting: a hit is answered a few milliseconds from the user. Know what you give up:
- Clicks answered at the edge never reach your servers, so origin-side counting misses them. You would count from CDN logs or from an edge function that emits a click event.
- Edits, expiry and takedowns wait for the edge TTL unless every such change also triggers a CDN purge.
A reasonable default: terminate TLS at the edge for the latency win, pass redirects through (private), and cache at the edge only when analytics come from edge logs and takedowns trigger purges.
💡 Say it like this: "I'll return 302 with
Cache-Control: private, max-age=60. Browsers can reuse a redirect for a minute, so edits and takedowns land within a minute and we only miss same-browser repeats inside that minute. If product needs every click, I'll sendno-store."Likely follow-up: "Wouldn't 301 be faster?" Only for repeat clicks from the same browser, and only for as long as the headers allow. The headers are the decision, not the status code.
Move 5: Count clicks off the hot path
Pressure: 115,741 clicks a second on average, owners want counts by day, country and referrer, and the redirect must not wait for any of it.
redirect service --(click event, fire-and-forget)--> Kafka topic "clicks", partitioned by key
| |
v v
stream aggregator: raw-event sink
counts per key per minute |
| |
v v
ClickHouse: per-minute rollups + raw events (30 days)
^
|
owner dashboard queries
An event such as event_id, key, timestamp, country, referrer, user agent is about 200 bytes. That is 23 MB/s on average, 69 MB/s at peak, and 2 TB of raw events a day, which is why dashboards read rollups and raw events expire after a few weeks.
Four decisions:
- Never
UPDATE links SET clicks = clicks + 1per redirect. A viral link turns one row into a lock queue, every update writes a new row version, and redirects start depending on a writable primary. If you want a counter in the database, the aggregator adds+nonce per key per minute. - Fire and forget, with a bounded buffer. If Kafka slows down, the redirect still returns immediately. If the local buffer fills, you drop events and the counts come out a little low for the incident. Say that trade: a missing click is cheaper than a failed redirect.
- Expect duplicates. Producers retry and consumers restart, so in practice events arrive at least once. Give every event an id and make the aggregation idempotent where exact counts matter, for example for billing. Messaging & Queues covers idempotent consumers.
- Not every click is a person. Chat apps fetch links to build previews, and crawlers follow everything. Classify by user agent and report those separately.
Your turn: product adds "limited links": the first 1,000 clicks get a discount page, after which the link returns 410. What changes on the redirect path?
Check your answer
The count can no longer be asynchronous for those links, because the decision depends on it. Before redirecting a limited link, do an atomic increment (Redis INCR on a per-link counter) and return 410 once the result exceeds 1,000. Send Cache-Control: no-store on limited links, so no browser reuses the discount redirect and every click reaches the counter. Everyone else keeps the fire-and-forget path. The cost is an extra round trip on limited links only. If Redis fails over and loses recent increments, a few extra clicks can get through. If the limit involves money and must be exact, keep the counter in a durable store with a conditional update, and accept the extra latency.
Move 6: Let links die on time
Pressure: a link can exist in the database, in Redis, in each server's in-process cache, and in browser caches. Expiry, edits and takedowns must reach all of them within a bound you can state.
Expiry
- Check
expires_aton every read, including cache hits. The resolver above does this. - Cap every cache at the time left: the Redis TTL, the in-process TTL and the
max-ageyou send. - Return
410for expired,404for never existed. 410 tells clients the link existed and is gone on purpose. - Reclaim space later. Expired rows still take storage, so a job purges them in batches:
import sqlite3
def purge_expired(db, cache, now, batch=1000):
removed = 0
while True: # a loop, not recursion
keys = [k for (k,) in db.execute(
"SELECT short_key FROM links WHERE expires_at <= ? LIMIT ?", (now, batch))]
if not keys:
return removed
db.executemany("DELETE FROM links WHERE short_key = ?", [(k,) for k in keys])
db.commit() # 1. the database first
for k in keys:
cache.pop(k, None) # 2. then the cache
removed += len(keys)
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE links (short_key TEXT PRIMARY KEY, url TEXT, expires_at INTEGER)")
db.executemany("INSERT INTO links VALUES (?, ?, ?)",
[(f"k{i}", "https://example.com", i % 3) for i in range(2500)])
cache = {"k0": "https://example.com"}
print(purge_expired(db, cache, now=1)) # 1667 rows had expires_at <= 1
print(db.execute("SELECT COUNT(*) FROM links").fetchone()[0]) # 833 left
Small batches keep each transaction short, so replication lag and undo or WAL growth stay small while live traffic continues. A loop matters too: a version that calls itself for the next batch hits Python's default recursion limit of 1,000 frames after about a million rows.
A managed store can do the deleting for you. DynamoDB's TTL feature deletes expired items typically within a few days of their expiry time, which is fine for storage but is exactly why the read path must still check expires_at.
Never give an expired key to a new link: someone printed it, and their old poster would now open a stranger's page. Counter keys never repeat, so with them you may delete the row outright; after the purge the key is simply unknown and answers 404. Random and pooled keys need a tombstone. Their only referee is "insert if absent", and a deleted row is absent, so a freshly drawn key could land on a dead, printed one. Purge the URL and metadata but keep the key and a status (a few bytes), and make the random generator and the pool filler treat tombstones as taken. A tombstone also keeps 410 working forever.
Takedowns and edits: which copy first?
Predict first: a phishing link must stop now. It is cached in Redis with a 24-hour TTL and gets 3,000 clicks a second. Do you delete the cache entry first, or update the database first?
Check your answer
Update the database first, then delete the cache entry, then purge the CDN if you cache redirects there. If you delete the cache first, a click can land between the two steps: it misses the cache, reads the still-active row and puts the phishing URL back into the cache for another 24 hours. At 3,000 clicks a second, a gap of even a millisecond is likely to contain a click. Database-first leaves only the moments before the cache delete lands, when clicks still see the old cached value.
A rarer race remains even then: a reader that fetched the old row just before your update can write it into the cache just after your delete. A second delete a few seconds later (a "delayed double delete") closes that window in practice, and the in-process caches need their broadcast invalidation or their 10-second TTL.
Move 7: Don't become a phishing service
Pressure: anyone can make your trusted-looking domain point anywhere. Short links hide destinations, which is exactly what phishers want.
At creation:
- Parse the URL and allow only
httpandhttps. That blocksjavascript:,data:andfile:. Cap the length with a policy you state, for example 2,048 characters. - Rate-limit creates per account and per IP. The algorithms are in Design URL Shortener & Rate Limiter.
- Check the destination against threat lists. Google's Safe Browsing API is for non-commercial use; commercial services use the Web Risk API. Creation is not latency-critical, so a synchronous check is affordable.
- Rescan later. A destination can turn malicious after it was shortened, and threat lists update. Flagged links get a warning page instead of a redirect; confirmed abuse gets a takedown, in the Move 6 order.
Make create requests idempotent. A client whose POST timed out will retry, and without protection it gets two links. Store the response under the Idempotency-Key header for a day and return it on the retry. Networking Basics covers idempotent HTTP APIs.
SSRF is about fetching, not redirecting. A redirect sends the user's browser to the destination; your server never requests it. The risk appears once your service fetches destinations itself, for link previews, page titles or scanning. Then check the resolved IP address against private, loopback and link-local ranges (such as the cloud metadata address 169.254.169.254), and fetch through an egress proxy. Regular expressions over hostnames miss forms like 127.1, [::1] and 2130706433.
Custom aliases
import re
ALIAS = re.compile(r"[A-Za-z0-9_-]{3,30}")
RESERVED = {"api", "admin", "login", "static", "health", "about"}
def alias_problem(alias: str):
if not ALIAS.fullmatch(alias): # fullmatch: the WHOLE string must fit
return "use 3-30 letters, digits, '-' or '_'"
if alias.lower() in RESERVED:
return "reserved"
return None # uniqueness is the database's job
print(alias_problem("spring-sale")) # None
print(alias_problem("admin\n")) # use 3-30 letters, ...
print(alias_problem("../../etc")) # use 3-30 letters, ...
print(bool(re.match(r"^[A-Za-z0-9_-]+$", "admin\n"))) # True: the classic trap
⚠️ In Python, $ also matches just before a final newline, so re.match(r"^…$") accepts "admin\n": an alias with a control character gets past validation and past the reserved-word check. Use fullmatch. Reserved words protect your own routes (api, admin, login); an offensive-words list is a separate moderation concern.
The namespace clash nobody mentions. Aliases and generated keys live in the same table. A user can register the alias 4kQ9zTe today: exactly 7 base-62 characters, so one day the counter reaches it. With a plain insert, that create fails; with an unconditional put, the generated link silently replaces the alias. Two fixes, both fine:
- Disjoint namespaces: generated keys are always exactly 7 base-62 characters, so reject aliases that are exactly 7 letters or digits (or require aliases to contain
-or_). - The primary key referees: every insert is conditional; an alias that is taken gets
409, and a generated insert that hits an alias takes the next id.
Failure drill: what breaks, and what you give up
Interviewers love "and then this node dies". Here are the events worth rehearsing, with the numbers from this design:
| Event | Without the move | Move | What you give up |
|---|---|---|---|
| A Redis node dies | Its share of keys misses until a new node warms up | Replicas with failover: a warm copy takes over | Memory for replicas |
| The whole Redis cluster restarts empty at peak | The database gets 347,000 reads a second: 10× the load it was sized for (90% hits), 20× its normal load | Never restart every node at once; persistence (RDB/AOF) so nodes come back warm; single-flight; warm from recent click events; shed load | Disk and restart time for persistence; a warm-up period |
| One link goes viral | One Redis shard saturates | In-process cache (10 s) + single-flight | Edits and takedowns lag up to 10 s |
| The id allocator is down | Servers partway through a block run dry within minutes | Double-buffered blocks (at least 29 minutes of tolerance); allocator on a synchronously replicated store | Ids wasted on restarts |
| The allocator fails over and forgets a lease | Two servers mint the same keys; an unconditional put overwrites a live link | Conditional inserts as the referee | A retry on conflict |
| A replica lags 2 seconds | New links 404 on a cache miss | Write-through on create; check the primary before a 404 | Probes for bogus keys reach the primary |
| Kafka is slow | Redirects fine; events buffer, then drop | Bounded fire-and-forget buffer | Counts run low during the incident |
| The home region is down | Everything fails in a single-region design | Regional read stacks keep redirects up | Creation pauses until failover |
| A client retries a timed-out create | Two links for one request | Idempotency-Key |
Stored responses for a day |
Predict first: the cache cluster comes back empty at 20:00, the daily peak. What do you do in the first minute, before any cache is warm?
Check your answer
Protect the database, because it is the only copy of the truth. Single-flight collapses identical misses, so a hot link costs one database read per server, not thousands. Admission control sheds the excess: return a fast 503 with Retry-After for a slice of traffic rather than letting every request time out. Prioritize the links clicked in the last hour, from the click stream, when warming. Then fix the cause. Replicas would not have helped, because they restart empty too. Restart cache nodes one at a time, and enable persistence so a restarted node reloads its data instead of starting cold.
The whole design on one page
Put the seven moves together and trace one request down each path. First a redirect, with the key already in Redis:
browser --HTTPS--> CDN edge ------> load balancer --> redirect service
(TLS ends here; | 1. in-process cache (10 s): miss
redirects pass | 2. Redis GET 1000000: url + expires_at
through) | 3. expiry check passes
| 4. click event into the local buffer (no wait)
v
302 Location: <long url> Cache-Control: private, max-age=60
On a Redis miss, step 2 reads the links store (partition = hash(key)) and fills Redis.
The buffer drains to Kafka --> aggregator (per key, per minute) --> ClickHouse --> dashboards.
Server-side, that is a few milliseconds. Now a create:
browser --> CDN edge --> load balancer --> create service
| 1. validate URL, rate-limit, threat-list check
| 2. next id from the leased block
| (allocator: one call per 100,000 ids)
| 3. keyed permutation + base 62 -> 7-character key
| 4. conditional insert into the links store
| 5. SET key in Redis (write-through)
v
201 Created Location: https://sho.rt/<key>
Retries carry the same Idempotency-Key, so a create that timed out returns the same link instead of making a second one.
Final round: no label on the prompt
Real prompts don't say which moves they need. Start from the contract, find the pressure, then choose the move.
Challenge 1: Pastebin
Users paste text up to 1 MB. 5 million pastes a day, 10 reads per paste, pastes expire after 30 days by default and are never edited. Where does the text live, what are the keys, can the CDN cache, and how does expiry work?
Check your answer
- Storage: put the text in object storage and keep a small metadata row (key, object path,
expires_at) in the key-value store. At an assumed 10 KB average, that is 50 GB a day and 1.5 TB at steady state with 30-day expiry. Reads are 50 million a day, about 580 a second: small. - Keys: the same moves. Pastes are often private, which argues for 10+ random characters.
- Caching: the content is immutable, so unlike redirects it is safe to cache at the CDN with
max-agecapped at the time left, plus a purge on takedown. View counts, if wanted, come from CDN logs. - Expiry: object-storage lifecycle rules can delete by age, but they run asynchronously, so still check
expires_aton read.
Challenge 2: Password-reset links
Email a link that resets a password: 2 million a day, valid for 15 minutes, usable once. It looks like a shortener. Is it?
Check your answer
Not really. Every speed trick from this lesson is wrong here, because single use and freshness beat latency.
- Key: a 128-bit random token (22 base-62 characters). Store only a SHA-256 hash of it, so a database leak doesn't leak live tokens. Unlike a password, a 128-bit random token needs no salt or slow hash, and an unsalted hash can still be looked up by equality.
- No caching anywhere:
Cache-Control: no-store, no CDN, no replicas on the read path. - Single use needs an atomic consume:
UPDATE reset_tokens SET used_at = now() WHERE token_hash = … AND used_at IS NULL AND expires_at > now()succeeds for exactly one request. - Consume it on the form's
POST, not on theGET. Corporate mail scanners fetch links in emails, and a token that is burned onGETwould be used up before the user ever clicks.
Challenge 3: QR codes on packaging
A brand prints 50 million unique QR codes, one per package. A scan must work for 10 years, the destination may change, marketing wants scan counts by region within an hour, and a TV ad drives 20,000 scans a second. Some codes also enter a prize draw.
Check your answer
- Keys: 50 million fit in 5 characters, but prize codes must be unguessable, so use random keys long enough to be sparse: 12 characters gives odds of about 1 in 64 trillion per guess with 50 million live codes. QR codes have room for that.
- Redirects:
302with a shortmax-ageorno-store, so destination changes land quickly and scans are counted. TLS at the edge for mobile latency. - Counts by region within an hour: click events with country, aggregated per minute. 20,000 scans a second is well within the read path.
- Ten years: never expire, replicate across regions, and serve from a domain the brand controls, so the codes outlive any vendor.
Cheat sheet: pressure → move → cost
| When the requirements say… | Reach for | You give up |
|---|---|---|
| Unique short keys from many servers | Leased counter ranges + base 62; 7 characters last about 96 years at 100 M/day | Keys reveal order unless permuted |
| Keys must not be enumerable | A keyed permutation of the id | Still guessable while the space is dense |
| Keys must be unguessable | Long random keys (12–22 characters) and a conditional insert | Short links |
| Same URL → same link | A per-owner deduplication index | An extra index and write |
| 100 reads per write | Cache-aside, write-through on create, negative caching | Staleness you must bound |
| One viral link | In-process cache + single-flight | Seconds of staleness |
| Storage grows forever | Partition by hash of the key, many logical partitions | No key range scans; resharding work |
| New links must work at once | Write-through; check the primary before a 404 | Bogus probes reach the primary |
| Count clicks | 302 or explicit Cache-Control; events into a log |
More traffic reaches the origin |
| Global latency | TLS at the edge; regional caches and replicas; cache redirects at the edge only with edge logs and purges | Analytics plumbing, cross-region complexity |
| Expiry and takedowns | Check on every read, TTL capped at time left, database then cache, batch purge, tombstones for random keys | A cleanup job |
| Abuse | Scheme allowlist, rate limits, threat-list scans, rescans, warning pages | Friction for honest users |
Before moving on
Pick one row of the cheat sheet and explain it aloud as you would in an interview, in under two minutes: the requirement, the number that makes it matter, the move, what breaks without it, and what you give up. Then answer the follow-up you'd dread most: "What happens when traffic grows 10×?" If you can do that for three rows from a blank page, you can handle this prompt, and most variations of it.
In a 45-minute slot, this prompt fits the course's four phases, Scope → Sketch → Deep dive → Wrap-up, from Interview Framework & Strategy:
| Minutes | Phase | For this prompt |
|---|---|---|
| 0–3 | Intro and prompt | Listen, and confirm the kind of shortener |
| 3–10 | 1. Scope | The contract, plus the numbers that decide something: writes/s, reads/s, storage per year, key length |
| 10–20 | 2. Sketch | Two endpoints, one table, and one create and one redirect traced end to end |
| 20–35 | 3. Deep dive | One topic, with its failure modes: key generation, caching and redirects, or expiry |
| 35–40 | 4. Wrap-up | Summary, top risks, what changes at 10× |
| 40–45 | Your questions | Your questions for the interviewer |
Check yourself at minutes 10, 20 and 35. If you are behind, compress; don't skip a phase.
Next: Design URL Shortener & Rate Limiter for the rate limiter this design leans on, Twitter/Instagram News Feed for fan-out and celebrity problems at a larger scale, and Mock Interviews & Communication to practise this prompt against a clock.