A rate limiter is a control layer that sits in front of your API and enforces a maximum number of requests any single client can make within a defined time window. Too many requests in too short a period, and the limiter blocks or delays the extras. It's one of the most straightforward defences in web infrastructure, yet a surprising number of small services skip it until something breaks.
Why rate limiting exists
APIs are exposed endpoints. Anyone with the URL can hammer them. Without a rate limiter, a single misbehaving client, whether a buggy script, a scraper, or a deliberate attacker, can consume all of a server's available capacity. The result is either a crashed service or massive cloud bills. Neither is acceptable in production.
Rate limiting also prevents abuse that isn't overtly malicious. A developer who accidentally writes an infinite loop against your API can take down your service just as effectively as a distributed denial-of-service attack. The limiter catches both cases with the same mechanism.
There's a commercial reason too. Most SaaS APIs sell access in tiers. A rate limiter is what actually enforces those tiers. Without it, a free-plan user can consume the same resources as an enterprise customer.
How a rate limiter works in practice
The core idea is simple: track how many requests a client has made, compare that count against a threshold, and either allow or reject the request. The complexity is in choosing which algorithm to use and where to store the state.
The four main algorithms
The fixed window algorithm is the easiest to implement. It divides time into fixed buckets (say, one-minute intervals) and allows up to N requests per bucket. The downside: a client can send N requests at the very end of one window and N more at the very start of the next, effectively doubling the allowed rate for a brief period.
The sliding window log fixes that by storing a timestamp for every request in a rolling log. When a new request arrives, the limiter discards timestamps older than the window length and counts what's left. Accurate, but memory-hungry at scale.
The sliding window counter is a compromise. It uses two fixed-window counts, current and previous, and applies a weighted formula based on how far through the current window you are. It approximates the sliding log without storing individual timestamps.
The token bucket algorithm gives each client a bucket that fills with tokens at a steady rate. Each request costs one token. If the bucket is empty, the request is rejected. Token buckets allow short bursts, which is often more realistic than hard per-second caps. AWS API Gateway uses a token bucket approach by default.
Where the rate limiter lives
A rate limiter can sit at several points in your stack. Common placements include the API gateway, a reverse proxy like Nginx, or as middleware inside the application itself.
The API gateway is the most popular choice for public-facing APIs. It handles rate limiting before the request reaches any application code, which means a rejected request costs almost nothing in compute. If you're running services behind a reverse proxy, that proxy can handle limiting before traffic fans out to multiple backends.
Application-level limiting is useful when you need request context that isn't available at the gateway layer, such as the logged-in user's subscription plan or their specific resource ownership. It costs more per request but gives you finer control.
State storage: the hard part
A single-server rate limiter can keep its state in memory. Fast, simple, no dependencies. The problem is that most production systems run more than one server. If three instances of your API each keep their own in-memory counters, a client can send three times your intended limit by round-robining across all three.
The standard solution is a shared data store. Redis is the near-universal choice because it's fast enough to add only a fraction of a millisecond to each request, and it supports atomic increment operations that prevent race conditions when multiple servers update the same counter simultaneously.
Distributed rate limiting introduces one real trade-off: a dependency on the cache layer. If Redis goes down and your rate limiter fails closed, all traffic stops. If it fails open, the limit is temporarily unenforced. Which you choose depends on whether uptime or protection is the higher priority for your particular service.
What happens when the limit is hit
A well-behaved rate limiter returns an HTTP 429 status code ("Too Many Requests") with a Retry-After header telling the client how many seconds to wait. It should also include X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset headers on every response, not just the rejected ones. This lets clients back off gracefully before they hit the wall.
Silently dropping requests is poor practice. It leaves clients confused about whether their request was received, and it makes debugging harder for everyone. Return a clear error and a clear wait time.
Rate limiting and security
Rate limiting is not a complete security solution. It slows down a brute-force attack but doesn't stop a determined attacker rotating through thousands of IP addresses. For login endpoints, rate limiting should sit alongside account lockout policies, CAPTCHA challenges, and anomaly detection.
That said, rate limiting is a cheap and effective first layer. A brute force attack that can only attempt 5 passwords per minute instead of 50,000 is still an attack, but it becomes practically useless for cracking anything but the weakest credentials.
Setting the right limits
The biggest mistake teams make is setting limits that are either too tight (breaking legitimate users) or too loose (providing no real protection). Start by measuring your legitimate traffic patterns before choosing thresholds. If your 99th-percentile user makes 120 requests per minute, a 100-request limit will cause constant false positives.
Different endpoints warrant different limits. A read-heavy search endpoint can tolerate far more requests than a write endpoint that triggers a database operation or sends an email. Blanket limits applied across all routes are a starting point, not a final configuration.
Rate limiting is one of those infrastructure decisions that feels optional right up until the moment it isn't. A few lines of middleware configuration or a single API gateway setting is all it takes. The cost of setting it up is low. The cost of skipping it can be catastrophic.