grilling

- **Fixed window:** 100 per clock minute. Simple, but allows boundary bursts. - **Rolling window:** No more than 100 in any trailing 60 seconds. Stricter; needs more bookkeeping. - **Token bucket:** Refill at 100/60 tokens per second, up to a chosen burst capacity. Allows controlled bursts and a sustained average rate. **For general API traffic, I’d usually start with a token bucket**, but explicitly choose the burst capacity. A capacity of 100 permits an immediate 100-request burst; it does **not** guarantee at most 100 requests in every rolling minute. ## A stronger baseline 1. **Authenticate the key, then resolve its owner and plan.** Don’t trust client-supplied plan information. 2. **Enforce a shared bucket** using Redis or a dedicated rate-limit service. The refill/check/decrement must be atomic—not separate reads and writes. 3. **Define the quota scope:** per key, per account, or both. If customers can mint keys, account-level enforcement is often necessary. 4. **Return 429 when exhausted**, with a `Retry-After` value reflecting the time until another request could be admitted, rounded up to whole seconds. That is guidance, not a reservation: concurrent requests may consume newly available capacity. 5. **Add separate concurrency or cost controls** for expensive endpoints. One cheap read and one large export may not deserve equal weight. 6. **Use bounded storage and expiration** for idle limiter state. Local in-memory limits can still serve as a coarse emergency guard, but shouldn’t be presented as the authoritative global quota. ## The failure-policy decision you need now **What happens when the shared limiter is unavailable?** - **Fail open:** Preserve API availability, but lose quota enforcement and potentially expose backends. - **Fail closed:** Protect backends, but make the limiter an API availability dependency. A limiter outage is generally a **503**, not evidence that the client exceeded its quota. - **Degraded mode:** Apply conservative local limits temporarily, accepting that they’re approximate. Pick this deliberately, monitor it, and load-test the failure path. ## Questions I’d settle before implementation 1. Is this limit primarily for **backend protection, customer fairness, or a contractual quota**? 2. Do you need a strict rolling-minute cap, or are bursts acceptable? 3. Is the API deployed in one region or several? 4. Should multiple enterprise keys share an account allowance? 5. Do rejected requests, authentication failures, and failed backend requests count? Those answers determine whether a shared token bucket is enough—or whether you need stricter accounting and additional protection layers.