RAG interview questions: what FDE loops ask and how answers are scored

Most RAG questions in a forward deployed engineer loop are not "what is RAG". They describe a system that leaks, goes stale or answers wrongly with confidence, and a customer who needs it fixed. The interviewer is listening for the order you reason in.

This guide draws on the FDEBench bank, where 19 of the 159 questions sit under Retrieval & RAG, and on the scoring rubrics published with them. For where the design round sits in the whole loop, start with the round-by-round guide.

What do FDE interviews ask about RAG?

They ask you to diagnose a broken system, design under a hard constraint, or defend a choice. Definitions still appear as warm-ups, but they don't decide the round.

• Diagnose: "Your RAG chatbot gives fluent but wrong answers. Diagnose it." Or the leak: "A multi-tenant RAG assistant is returning documents from other customers’ accounts. Diagnose the design flaw and tell me how you would fix it."
• Constraint: "How do you handle stale context in a RAG pipeline where the knowledge base updates every 5 minutes?" Or scale: an embedding pipeline that has to stay accurate from 1M to 1B items.
• Defend a choice: "Why is semantic search alone not enough for production RAG?" and "When is agentic RAG worth the complexity over plain RAG?"
• Search and ranking: a search engine that "cites sources and ranks answers by confidence rather than just relevance".

One candidate's report from an Eightfold.ai round, posted on Reddit and unverified, asks you to pitch an end-to-end RAG service in 15 minutes, with evals and a rollback story. That is the whole topic in one prompt.

What should you ask before designing a RAG system?

Ask four things before you draw a box: what the queries look like, how fresh answers must be, who may see which documents, and what a wrong answer costs.

• Query mix. What share are single-fact lookups, and what share compare or chain across documents? That one number decides whether an agentic loop is worth building.
• Freshness. "Updates every 5 minutes" is the write rate, not the requirement. The rubric for bank-41 gives its first criterion, 20 of 100 points, to asking how stale an answer is allowed to be.
• Permissions. Is the leak across tenants or inside one? Does tenant identity come from the session or the request? The asks-first list for bank-128 adds one more: has data already been exposed? Then it is an incident first.
• Cost of a wrong answer. Is "I don't know" acceptable? A bank and a recipe site answer differently.

Which design should you choose: plain, hybrid or agentic RAG?

Choose by the query mix, not by ambition. Plain RAG embeds the question, fetches the top-k chunks and generates. It is fast and fine for single-topic lookups. Agentic RAG, in Meilisearch's description, plans, retrieves and validates instead of retrieving once. DigitalOcean's comparison draws the same line.

Each layer fixes one failure and adds latency. Name both halves.

The strong move is a gate in front. A cheap classifier sends simple queries down the fast path and only complex ones into the loop. Then give the loop exits: an iteration cap, a sufficiency threshold, and a stop when a new pass mostly returns what the last one did. The same loop discipline comes back in the agent design round.

Why doesn't a bigger context window fix retrieval?

Because the failure is noise, not space. More chunks in a larger window means more stale and off-topic text competing with the evidence. A Google Developers Blog post on context-aware multi-agent frameworks puts it plainly: "A large context window will help context-related problems but won't address all context-related problems."

Name the three ways long context goes wrong, often called context rot. Old facts persist and poison later answers. Verbose chunks and tool output distract from the instructions. A newer instruction conflicts with an older one that was never cancelled.

The fix is retrieval discipline. Score each chunk for relevance to this turn, give each content type a token budget, and reject what doesn't clear the bar. Retrieve-and-append is not discipline.

What numbers should you estimate out loud?

Size the index and the latency before you pick a database. Both numbers change the design.

Index memory (illustrative). One billion vectors at 768 dimensions and 4 bytes each is about 3 TB of raw float32, before index overhead. Quantizing to 8-bit integers brings it to roughly 770 GB. Either number drives the design: you now owe a sharding plan, a compression choice and a recall budget for approximate search.

Loop latency (illustrative). Say one retrieve-and-rerank pass takes 300 ms and the sufficiency check is a 700 ms model call. Three passes spend 3 seconds before generation starts. State the number, then say which queries are allowed to pay it.

What should you measure in a RAG system?

Measure retrieval and generation separately, or you can't tell which one broke. A fluent wrong answer can come from either.

• Recall@k on a labeled set of real questions. For approximate search, check against an exact-search sample, continuously. The bank-56 rubric gives this 20 points.
• Precision and re-rank lift. Did the right chunk reach the top three? The recall versus precision question asks which you favor, and when.
• Groundedness. The share of claims supported by a retrieved chunk, with citations checked claim by claim.
• Freshness. The age of retrieved chunks and the indexing lag, with an alert past the agreed tolerance.
• Isolation. Cross-tenant hits, logged with tenant and document ids. The target is zero, tested with a tenant that is trying to break in.
• Latency per stage, and loop passes per query.

The evaluation guide covers golden sets and model judges in more depth.

Which mistakes lose points in a RAG round?

Most of these come straight from the red flags on FDEBench's RAG rubrics. Each one is a question the candidate didn't ask.

• Fixing a leak with a system prompt. Authorization lives in code the model can't influence, with tenant identity from the session and the filter applied before retrieval.
• Re-embedding the whole corpus on a timer instead of from a change feed.
• Forgetting deletes. A retracted document still in the index gets cited.
• Brute-force similarity at a billion items, or never mentioning recall.
• An agentic loop on every query, with no exit.
• "Use a bigger context window" as the fix.

Every chunk you retrieve is also input tokens, which is why the cost and latency round asks you to do that math. RAG is a skill employers name, too: on the FDEBench board you can browse Applied AI / ML roles or search for LLM roles. To build agents end to end before your loop, the free course Claude Code for Forward Deployed Engineering is a good place to start.