System design deep dive
Design a URL Shortener (Bitly): deep dive
Designing a URL shortener is the classic warm-up system design question: it looks trivial, but it tests capacity estimation, ID generation, read-heavy caching, and redirect semantics in under 45 minutes. Here is a complete walkthrough with numbers you can defend.
Requirements to pin down first
- Functional: create a short link for a long URL, redirect a short link to the original, optional custom alias and expiry, basic click counts.
- Non-functional: redirects must be fast (p99 under 50 ms at the server) and highly available; short codes must be unique and not guessable in sequence; links last for years.
- Out of scope unless asked: user accounts, detailed analytics dashboards, link editing.
Capacity estimates
| Quantity | Assumption | Result |
|---|---|---|
| New links | 100 million per month | About 40 writes/s average, 200/s peak |
| Redirects | 100:1 read to write ratio | About 4,000 reads/s average, 20,000/s peak |
| Total links | 5 years of retention | About 6 billion records |
| Storage | About 500 bytes per record including indexes | About 3 TB |
| Code space | 7 base62 characters | 62^7, about 3.5 trillion codes |
| Cache | Hottest 10 million links at 500 bytes | About 5 GB, fits on one Redis node with a replica |
Two conclusions follow. Write load is tiny, so one well-provisioned database could handle it; the real work is serving reads fast. And 7 characters gives more than 500 times the codes we need for five years, so we never need to worry about running out.
API
POST /api/v1/links
body: { "longUrl": "https://...", "alias": "spring-sale", "expiresAt": "2027-01-01" }
201 -> { "shortUrl": "https://sho.rt/spring-sale", "code": "spring-sale" }
409 if the alias is taken
GET /{code}
301 or 302 with Location: <longUrl>
404 if unknown, 410 if expiredData model
A single table, links(code PK, long_url, created_at, expires_at, owner_id), covers the core. Because every redirect is a primary-key lookup, a key-value store such as DynamoDB or Cassandra fits naturally, though Postgres with the code as primary key works perfectly well at 3 TB with sharding by code once it outgrows one node. Click counts go to a separate pipeline, never into the hot redirect row.
Generating short codes
The cleanest approach is to give each link a unique 64-bit integer and base62-encode it. To avoid a single counter bottleneck, each app server leases a block of IDs, for example 10,000 at a time, from a small coordinator table or ZooKeeper, and hands them out from memory. To keep codes from being sequential and guessable, run the integer through a reversible permutation (a simple Feistel network or multiply-by-a-large-odd-number modulo 62^7) before encoding.
static final String ALPHABET =
"0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
static String encode(long id) {
if (id == 0) return "0";
StringBuilder sb = new StringBuilder();
while (id > 0) {
sb.append(ALPHABET.charAt((int) (id % 62)));
id /= 62;
}
return sb.reverse().toString(); // e.g. 125 -> "21"
}The alternative, hashing the long URL with MD5 and taking the first 7 characters, needs a collision check on every insert (read, then conditional write) and gives the same code to everyone who shortens the same URL, which breaks per-user analytics. Mention it, then explain why the counter approach is simpler.
Components
| Component | Role |
|---|---|
| CDN / edge | Optionally caches 301 responses for the most popular links |
| Load balancer | Spreads traffic across stateless app servers |
| Write service | Validates URLs, checks aliases, assigns IDs, stores the mapping |
| Redirect service | Looks up code in Redis, falls back to the database, returns the redirect |
| Redis cache | Cache-aside with LRU eviction for hot codes, plus negative caching for unknown codes |
| Database | Durable code to URL mapping, replicated across zones |
| Click pipeline | Redirect service emits click events to Kafka; consumers aggregate counts |
301 or 302?
A 301 Moved Permanently lets browsers and proxies cache the redirect, which cuts load and latency, but repeat clicks then never reach your server, so analytics undercount and you cannot change the destination later. A 302 Found keeps every click flowing through the service. Bitly-style products that sell analytics generally use 301 with short cache headers or 302; the important thing is to state the trade-off and choose deliberately. More on cache layers in caching.
Common mistakes in interviews
- Spending ten minutes on the hash function and never estimating traffic.
- Incrementing a click counter on the link row during each redirect, turning a cheap read into a contended write.
- Forgetting abuse: malicious URLs need scanning against a blocklist, and creation endpoints need rate limits.
- Ignoring custom alias collisions with generated codes; reserve a separate namespace or check both.
- Not handling expired links, which should be lazily rejected on read and cleaned up by a background job.
Interview questions
›How do you guarantee two servers never generate the same code?
Give each server a disjoint range of IDs. A coordinator atomically increments a counter by a block size and returns the block start, so server A might own 1,000,000 to 1,009,999 while B owns the next block. If a server crashes, the unused part of its block is simply skipped, which is harmless given the size of the code space.
›How would you scale redirects to 1 million requests per second?
Push the hottest links to the CDN edge, scale stateless redirect servers horizontally, and shard the Redis cache across a cluster. Because link popularity follows a power law, a cache of a few tens of gigabytes can serve well over 95% of redirects without touching the database.
›Should the same long URL always produce the same short code?
Usually no. Different users want separate links with separate analytics and expiry. If deduplication is required, keep a secondary index from a hash of the long URL to its code, scoped per user.
›How do you collect analytics without slowing redirects?
The redirect handler sends a small event (code, timestamp, country, referrer) to Kafka asynchronously and returns immediately. Stream consumers aggregate counts per code per hour into an analytics store, so the redirect path stays a single cache lookup.
›How would you prevent enumeration of all short links?
Do not expose raw sequential IDs: scramble them with a reversible permutation before base62 encoding, or use random codes with a uniqueness check. Combine this with rate limiting on 404s per client so scanning the code space is impractical.
