FDEBench Join us live

Q
10 / 159

Platform & multi-tenancy

What are the key steps to design a multi-tenant LLM platform ensuring data isolation for 500 enterprise clients?

question bank FDEBench bank Source

01
Ask

What a strong candidate asks first

Before designing anything. Skipping this is the most common way the round is lost.

  • Are customers fine-tuning on their own data, or only doing retrieval?
  • What's the regulatory posture — do any tenants need physical separation?
  • Is "never" a contractual guarantee, or best-effort?

02
Rubric

What earns the points

6 criteria, 100 points total. Scored independently, partial credit allowed.

Lists every way data can leak 25 pts

Names the distinct places a leak can occur — retrieval index, cache, fine-tuned weights, logs and traces, error messages. Credit scales with how many they surface; three or more is full credit.

If you miss it You secured one path. Cross-tenant leakage has several independent routes — the vector index, the response cache, fine-tuned weights, and logs are each capable of leaking on their own.
Carries customer ID everywhere 15 pts

Carries a tenant identifier through every layer and scopes each query by it, rather than filtering at the application layer only.

If you miss it Your tenant scoping lived in application code. A single query that forgets the filter is a breach — push the constraint down to where it can't be forgotten.
Splits the cache by customer 15 pts

Specifically calls out that a shared cache defeats every other isolation control, and partitions it.

If you miss it You isolated storage and retrieval but left the cache shared. A cache hit crosses every boundary you just built.
Keeps fine-tunes per customer 15 pts

Handles per-tenant model customization without co-mingling training data — separate adapters, separate storage, or an argued decision not to offer per-tenant tuning.

If you miss it You didn't address per-tenant fine-tuning. Weights trained on one customer's data are that customer's data — they can't be served to anyone else.
Checks answers before they ship 15 pts

Validates on the way out, not only on the way in — checks the response is scoped to the requesting tenant before it ships.

If you miss it All your controls were on the request path. Add a check on the response path — verify what you're about to return belongs to the tenant who asked.
Can prove isolation to auditors 15 pts

Logs access per tenant in a way that can demonstrate isolation to an auditor or after an incident.

If you miss it You can't prove isolation held. Enterprise buyers need an audit trail that shows which tenant accessed what, not just an assurance.

03
Flags

Red flags

Reported separately, not subtracted — this is the "what would have ended the interview" note.

  • Relies on prompt instructions to enforce isolation.
  • Treats this as an access-control problem only, ignoring the model layer.

—