What is the Rate Limiting pattern?
A rate limiter counts requests per key (an API key, user, IP or tenant) and rejects requests over the limit, usually with 429 Too Many Requests and a Retry-After header. Common algorithms are token bucket, which allows short bursts, and sliding window. Limits are usually enforced at the gateway, sometimes again inside services.
What problem does the Rate Limiting pattern solve?
Without limits, one client, whether buggy, abusive or just busy, can use all your capacity and degrade service for everyone. Rejecting excess requests at the edge is cheap and keeps the load you accept within what you can actually handle.
How do you implement the Rate Limiting pattern?
- Choose the key you limit on and the limits per tier, based on measured capacity and fair use.
- Pick an algorithm. Token bucket suits APIs that should tolerate short bursts.
- Enforce it at the gateway with a shared counter store so limits hold across instances.
- Return 429 with Retry-After and rate limit headers so well-behaved clients can slow down.
- Monitor rejections per key, and decide whether to fail open or closed if the limiter's store is down.
When should you use it?
- You run a public or multi-tenant API.
- Some clients can generate far more traffic than others.
- A downstream system has a hard capacity limit that you must protect.
When should you avoid it?
- All traffic comes from trusted internal callers with their own backpressure. Use load shedding or bulkheads instead.
What do you gain?
- Protects capacity and fairness
- Cheap rejection at the edge
- Gives clients a clear signal to slow down
What does it cost?
- Limits that are wrong frustrate legitimate users
- Distributed counting adds latency and a dependency
- Needs tiers and exceptions over time
What are common mistakes with the Rate Limiting pattern?
- Limiting per instance, so the real limit changes with the number of replicas.
- No Retry-After header, so clients retry immediately and make it worse.
- Taking the whole API down when the counter store fails.
Rate Limiting in practice
Gather Chat
Spam bots flooded busy channels during a viral event. Message volume spiked, and latency rose for everyone on the shared gateway.
The gateway now enforces token-bucket limits per user, per channel and per API key, returning 429 with Retry-After. Bots are throttled while normal users are unaffected.
Example decision record
- Context
- Public endpoints are open to abuse and unpredictable spikes.
- Decision
- Enforce token-bucket limits per API key at the gateway, backed by a shared store.
- Consequences
- Excess requests get 429 with Retry-After.
- High-volume customers need their own limit tiers.