Rate Limiting a FastAPI App with Redis: The Token Bucket, Done Right

Every public API needs a rate limiter. Without one, a buggy client, a scraper, or a runaway AI agent can take the service down for everyone. It is also one of the most common system design questions, which makes it worth understanding properly rather than pasting a library.

This post builds a real one: a token bucket in Redis, enforced as a FastAPI dependency.

Pick an algorithm#

The token bucket is the usual default because it matches how clients behave: quiet for a while, then a burst. You choose two numbers, the refill rate (the sustained limit) and the capacity (the largest burst).

Why it must be atomic#

The naive version reads the token count, subtracts one, and writes it back. With several app servers or workers, two requests can read the same value and both spend the same token. The fix is to do the whole read-modify-write in one step inside Redis, using a Lua script. Redis runs a script without interleaving other commands.

The Lua script#

-- KEYS[1] bucket key
-- ARGV[1] refill rate (tokens/second), ARGV[2] capacity, ARGV[3] cost
local rate     = tonumber(ARGV[1])
local capacity = tonumber(ARGV[2])
local cost     = tonumber(ARGV[3])

-- Use Redis's own clock so app servers with skewed clocks agree.
local t   = redis.call("TIME")
local now = tonumber(t[1]) + tonumber(t[2]) / 1000000

local data   = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(data[1]) or capacity
local ts     = tonumber(data[2]) or now

tokens = math.min(capacity, tokens + (now - ts) * rate)

local allowed = 0
local retry   = 0
if tokens >= cost then
  tokens  = tokens - cost
  allowed = 1
else
  retry = (cost - tokens) / rate
end

redis.call("HSET", KEYS[1], "tokens", tokens, "ts", now)
-- Idle buckets are full again after capacity/rate seconds; drop them after twice that.
redis.call("EXPIRE", KEYS[1], math.ceil(capacity / rate) * 2)

return { allowed, tostring(retry) }

Two small decisions matter. The retry time is returned as a string because Redis converts Lua numbers to integers in replies and would truncate 0.4 to 0. And the key expires on its own, so you never accumulate buckets for clients who left.

The FastAPI dependency#

import math

import redis.asyncio as redis
from fastapi import Depends, FastAPI, HTTPException, Request

r = redis.from_url("redis://localhost:6379", decode_responses=True)
bucket = r.register_script(LUA_SCRIPT)  # the script above as a string


def rate_limit(rate: float = 5, capacity: int = 20, cost: int = 1):
    async def dependency(request: Request) -> None:
        client = request.client.host if request.client else "unknown"
        key = f"rl:{request.url.path}:{client}"

        allowed, retry = await bucket(keys=[key], args=[rate, capacity, cost])
        if not allowed:
            raise HTTPException(
                status_code=429,
                detail="Too many requests",
                headers={"Retry-After": str(math.ceil(float(retry)))},
            )

    return dependency


app = FastAPI()


@app.get("/search", dependencies=[Depends(rate_limit(rate=2, capacity=10))])
async def search(q: str):
    return {"q": q}

A factory function lets each route set its own limits: strict on login, generous on reads. Reusing the same dependency everywhere keeps the behaviour consistent.

Details that bite in production#

Identify the client correctly. Behind a load balancer or reverse proxy, request.client.host is the proxy's address, so every user shares one bucket. Run Uvicorn with trusted proxy settings so the real client IP is resolved, and never read X-Forwarded-For blindly, because clients can forge it.

Limit by user, not just IP. Once a request is authenticated, key the bucket on the user or API key. IPs are shared (offices, mobile carriers) and easy to rotate.

Always return 429 with Retry-After. Well-behaved clients back off exactly as long as you tell them to. Without it they guess, usually by hammering harder.

Decide what happens when Redis is down. You have two honest options. Fail open keeps the API available but unprotected; fail closed protects the backend but turns a Redis outage into an API outage. For most read endpoints fail open and log loudly; for expensive or abusive-prone ones, fail closed.

Charge by cost. The cost argument is there for a reason. A cheap lookup costs 1; an export that scans a table costs 10. The same bucket then enforces fairness by work done, not by request count. This is also how you would budget tokens for an LLM endpoint.

Test the behaviour, not the library#

Write a test that fires capacity + 1 requests instantly and asserts the last one returns 429 with a Retry-After header. Then wait 1 / rate seconds and assert one request succeeds. Those two tests lock in the whole contract.

Summary#

A token bucket in Redis is about forty lines, atomic by construction, and enforces both an average rate and a burst size. Put it in a dependency, key it on the best identity you have, and decide your Redis-failure policy before an outage decides it for you.