How would you design a rate limiter for a public API?
Choose the key, the algorithm, and what the client sees when it is limited.
Points worth saying
- Limit a key: API token, user, or IP. Say which one and why.
- Token bucket allows a burst. Sliding window is easier to explain as a quota.
- Return 429 with Retry-After. Fail open or closed on a store outage, and say which.
A full answer
Decide the key first. For a public API I would limit by API token, with a coarser IP limit in front to absorb anonymous traffic. A user id is wrong for server-to-server keys that are shared.
A token bucket fits APIs that can tolerate a short burst. The bucket refills at the allowed rate and each request takes a token. A sliding window counter is easier to explain as “100 requests per minute” and is cheap in Redis with a timestamped key.
The check belongs at the edge, before expensive work. Store counters in a shared cache so every app node agrees. Use a short TTL so abandoned keys expire. Make the increment and the compare atomic.
When the limit is exceeded, respond with 429 and Retry-After. Document the limit in headers on successful responses too. If the cache is down, I would fail open for read APIs and fail closed for write or auth APIs, and I would say that choice out loud because it is a product decision.
Mark your progress
Create a practice profile to mark this question.