System design deep dive
API Gateway: deep dive
An API gateway is the single front door for every client request, handling the cross-cutting work such as authentication, routing, throttling, and observability so each backend service does not reimplement it. The interview skill is knowing what belongs in the gateway and what must stay out.
How a request passes through a gateway, step by step
- The client sends
GET /v2/users/42/orderswith anAuthorization: Bearer <jwt>header toapi.example.com. - The gateway terminates TLS and assigns a request ID that will be propagated to every downstream service.
- It verifies the JWT signature against cached public keys, checks expiry and scopes, and rejects with 401 or 403 without touching any backend.
- It applies rate limiting for the caller, for example 100 requests per minute for the free tier, returning 429 when exceeded.
- A route table maps
/v2/users/*/ordersto the Order service cluster. The gateway picks a healthy instance, applies a timeout, and may retry idempotent GETs once. - On the way back it can transform or compress the response, add CORS headers, and emit access logs and latency metrics per route.
Gateway vs load balancer vs service mesh
| Component | Traffic it handles | Main job |
|---|---|---|
| Load balancer | Any traffic to one service pool | Spread requests across identical instances |
| API gateway | North-south: external clients to services | Auth, routing by API, quotas, API versioning |
| Service mesh | East-west: service to service | mTLS, retries, traffic splitting, telemetry |
| Backend for frontend | One client type (mobile, web) | Aggregate and shape data for that client |
In practice these stack: a cloud load balancer in front of a horizontally scaled gateway fleet, which routes to services that talk to each other through a mesh.
When to use a gateway (and when not to)
A gateway is worth it once you expose several services to outside clients, because it gives one stable public surface while the internals change, and one place to enforce security policy. It is also how public API products manage keys, usage plans, and billing. For a single backend service, an LB plus middleware inside the app is simpler. Be wary of gateways that grow into an orchestration layer full of business rules: every team then needs gateway changes to ship, and you have rebuilt the monolith at the edge.
Responsibilities: belongs in the gateway or not?
- Belongs: TLS termination, token validation, coarse authorization (is this caller allowed to call this API at all), quotas, request size limits, routing and canary splits, API versioning, CORS, access logs.
- Borderline: response caching and simple aggregation. Fine when kept generic; push complex aggregation into a BFF.
- Does not belong: fine-grained permission checks (can user 42 edit document 9), pricing rules, data validation that depends on domain state. Those live in the owning service.
Common mistakes in interviews
- Drawing a single gateway instance; it must be stateless and horizontally scaled behind a load balancer.
- Doing all authorization at the gateway and trusting every internal call, which fails as soon as an internal service is compromised. Services should still verify identity (zero trust).
- Retrying non-idempotent requests like
POST /paymentsat the gateway, which can charge customers twice. - Keeping rate-limit counters in gateway memory, so limits are per instance rather than per user. Use a shared store.
- Forgetting that the gateway adds a hop; it should add only a few milliseconds at p99.
Interview questions
›How does the gateway validate JWTs without calling the auth service on every request?
JWTs are signed, so the gateway only needs the issuer's public keys, which it fetches from a JWKS endpoint and caches. Validation is then a local signature and expiry check. Revocation is handled with short token lifetimes plus refresh tokens, or a small denylist cached at the gateway.
›Why would you use multiple gateways or BFFs?
Different clients need different shapes of data and evolve at different speeds. A mobile BFF can return compact, aggregated payloads to save round trips on slow networks, while a partner API gateway enforces strict versioning and quotas. Splitting also limits the blast radius of a bad gateway deploy.
›How can a gateway support canary releases?
Its routing rules can send a percentage of traffic, say 5%, or a specific set of users to a new version of a service. Metrics per route reveal whether error rates or latency rise, and the split is increased or rolled back without client changes.
›What happens if the gateway goes down?
Every external request fails, so the gateway is deployed as multiple stateless instances across availability zones behind a load balancer, with health checks and autoscaling. Configuration is stored externally and rolled out gradually so a bad route change cannot break every instance at once.
›How does an API gateway handle request aggregation?
For a screen that needs user, orders, and recommendations, the gateway or BFF calls the three services in parallel and merges the results into one response. Each call has its own timeout so a slow recommendation service degrades to an empty section instead of failing the whole page.