Catalog
concept#Architecture#Reliability#Platform#Security

Rate Limiting

A mechanism to limit request rates to services and APIs, ensuring stability, fairness and protection against overload and abuse.

Rate limiting is an architectural pattern that restricts the rate of requests to services or APIs to prevent overload and abuse.
Established
Medium

Classification

  • Medium
  • Technical
  • Architectural
  • Intermediate

Technical context

API gateways (e.g., Kong, Apigee)CDNs with edge throttlingService mesh / Envoy rate limit service

Principles & goals

Protect system stability from external traffic spikesClear separation of quotas, rate and burst behaviorVisibility and metrics as basis for adjustment
Run
Domain, Team

Use cases & scenarios

Compromises

  • Misconfiguration leads to customer dissatisfaction
  • Bypass methods (e.g., IP rotation) reduce effectiveness
  • Excessive throttling can impact revenue
  • Start with conservative limits and iterate based on metrics
  • Differentiate global, customer and endpoint-specific limits
  • Provide transparent error and retry signals (Retry-After header)

I/O & resources

  • Identifiers (API key, IP, user ID)
  • Load and usage patterns
  • Service capacity models and SLAs
  • Throttle responses (e.g., HTTP 429)
  • Metrics and alerts
  • Adaptive backoff behavior

Description

Rate limiting is an architectural pattern that restricts the rate of requests to services or APIs to prevent overload and abuse. It specifies allowed request frequencies, throttling behavior, and response signals. Common deployments include API gateways, CDNs, and in-service controls to ensure predictable availability and fair resource sharing.

✔

  • Prevents overload and resulting outages
  • Protects against abuse and DoS attempts
  • Enables fair resource sharing between customers

✖

  • Incorrect limits can block legitimate users
  • Additional infrastructure and latency from central limit services
  • Complexity in distributed systems with many identities

  • requests per second

    Measure of incoming request rate per endpoint or key.

  • throttle hits (rejected requests)

    Number of requests rejected due to limits.

  • latency under throttling

    Response time statistics when limiting is active.

API gateway with token bucket

An API gateway implements token-bucket algorithms to limit bursts and stabilize endpoints.

CDN-based throttling

A CDN throttles requests to origin servers to smooth spikes and reduce costs.

Service-internal circuit breaker combined with rate limiting

Combining circuit breakers with per-endpoint rate limiting prevents cascading failures.

1

Analyze traffic patterns and define SLAs

2

Choose a limiting strategy (token bucket, leaky bucket, fixed window)

3

Integrate into gateway/service, set up monitoring and alerting

⚠️ Technical debt & bottlenecks

  • Monolithic limiting logic hinders scalable adjustments
  • No central metric, cumbersome troubleshooting
  • Outdated blacklist mechanisms instead of adaptive rules
central decision pointnetwork latency for distributed checksscaling limits of external rate limiters
  • Setting a too-low default limit for all users
  • Ignoring burst behavior during peak times
  • Relying solely on client-side rate limiting
  • Confusing quotas with real-time rate limiting
  • Insufficient testing under realistic load
  • Ignoring legal constraints in user segmentation
Knowledge of HTTP and API designExperience with observability and monitoringUnderstanding of distributed systems and consistency
Scalability under traffic spikesAvailability and fault toleranceCost control via limited resource usage
  • • Need for precise identifiers (API key, IP, user)
  • • Response time budget must not be exceeded
  • • Legal/compliance constraints on user segmentation