FDEBench Join us live

Q
138 / 159

Platform & multi-tenancy

Design an API rate limiter for a multi-node service. A colleague proposes counting requests per clock minute. Show them why that fails and what you would build.

question bank FDEBench bank

01
Ask

What a strong candidate asks first

Before designing anything. Skipping this is the most common way the round is lost.

  • Are we protecting capacity, enforcing a plan, or stopping abuse? Different strictness.
  • Is a burst above the sustained rate acceptable, or is smoothness the requirement?
  • What's the key — user, API key, tenant, IP?
  • How bad is a false throttle? That sets fail-open versus fail-closed.

02
Rubric

What earns the points

7 criteria, 100 points total. Scored independently, partial credit allowed.

Shows 2x the limit in 2 seconds 20 pts

Demonstrates the fixed-window flaw with numbers — full allowance at the end of one window and again at the start of the next, so twice the limit lands in seconds. Full credit requires the arithmetic, not just "fixed windows are bursty".

If you miss it You asserted fixed windows are bad without showing it. Walk the numbers: 100 at 11:59:59 and 100 at 12:00:00 is 200 requests in two seconds, entirely within the stated limit. The concrete case is what convinces your colleague.
Token bucket for rate and burst 20 pts

Proposes a token bucket and maps its two parameters onto the two business quantities — refill rate is sustained throughput, capacity is tolerated burst.

If you miss it You named an algorithm without connecting it to the requirement. A token bucket's refill rate and capacity are exactly "sustained rate" and "allowed burst" — say that, and the parameter choice becomes a product decision instead of a guess.
Shared counter with safe updates 20 pts

Keeps token count and last-refill timestamp in a central store, updated atomically (script or compare-and-set), and states the failure of per-node memory: the effective limit multiplies by node count.

If you miss it Per-node counters mean your 100/minute limit is really 100 × nodes. Say where the state lives and how it is updated atomically, or the limit is fiction.
Plans for the counter going down 15 pts

Decides explicitly what happens when the central store is slow or down — local approximate limiting, degraded limits, or an argued fail-open/fail-closed choice — rather than leaving the limiter as a hard dependency on every request.

If you miss it You made the limiter a synchronous hard dependency with no failure plan. Decide now whether an unreachable counter store means every request is allowed or every request is refused, and defend it.
Compares other limiting methods 10 pts

Mentions sliding window log (exact, stores every timestamp), sliding window counter (approximate, cheap), or leaky bucket (smooths output), with the tradeoff attached.

If you miss it You presented one algorithm as the only option. Naming the alternatives and their memory-versus-accuracy tradeoff shows you chose rather than defaulted.
Limits by more than IP address 8 pts

Chooses the limiting key deliberately and notes IP alone breaks behind NAT, shared egress, and mobile carriers.

If you miss it You didn't say what the limit is keyed on. IP alone punishes everyone behind one office NAT and misses a distributed caller entirely.
Clear retry info and metrics 7 pts

Returns 429 with retry-after and limit headers, and exposes per-key throttle metrics so throttling is debuggable.

If you miss it A limiter with no observability is a source of unexplained customer reports. Return retry-after and limit headers, and emit per-key throttle counts.

03
Flags

Red flags

Reported separately, not subtracted — this is the "what would have ended the interview" note.

  • Keeps counters in application memory across a fleet.
  • No behaviour defined for when the counter store is unavailable.
  • Limits on IP only.

—