FDEBench Join us live

Q
9 / 159

Multimodal & voice

How do you design an on-device LLM that works fully offline on an iPhone, considering model size, memory, and privacy?

question bank FDEBench bank Source

01
Ask

What a strong candidate asks first

Before designing anything. Skipping this is the most common way the round is lost.

  • What's the memory ceiling on the target device, and what else is running?
  • What tasks must work offline versus degrade gracefully?
  • Which devices are we supporting — how old is the oldest?

02
Rubric

What earns the points

6 criteria, 100 points total. Scored independently, partial credit allowed.

Puts numbers on memory and speed 20 pts

Puts real numbers on the constraint — available RAM, model size after compression, tokens/sec on-device. Order-of-magnitude is fine; the point is treating the constraint numerically.

If you miss it You described the constraints qualitatively. On-device is a numbers problem — say how much memory you have, how big the model is after compression, and what throughput that gives you.
Says what compression costs 20 pts

Names compression techniques AND what each costs in quality. Listing quantization/distillation/pruning with no quality discussion earns half credit.

If you miss it You listed compression techniques without saying what they cost. Every one of them trades quality for size — the design decision is how much you're willing to lose.
Narrows what the model must do 20 pts

Narrows what the on-device model is asked to do, so a small model can do it well, rather than shrinking a general assistant.

If you miss it You tried to fit a general assistant on the device. The winning move is usually the opposite — narrow the task until a small model is genuinely good at it.
Decides what may leave the phone 15 pts

Defines what happens when connectivity returns — what escalates to the server and what never leaves the device — or argues for strict on-device only.

If you miss it You didn't define the on-device/server boundary. Fully offline is a requirement for some tasks and a limitation for others — say which are which.
Accounts for battery and heat 15 pts

Raises sustained inference cost: battery drain, thermal throttling, or the need to bound how long the model runs.

If you miss it You didn't mention battery or heat. Sustained on-device inference throttles the chip and drains the battery — it's a real constraint, not a footnote.
Plans how the model gets updated 10 pts

Says how the model gets updated on-device given its size, and what that costs users in bandwidth and storage.

If you miss it You didn't say how the model gets updated. Shipping a multi-gigabyte model to every device is a distribution problem in its own right.

03
Flags

Red flags

Reported separately, not subtracted — this is the "what would have ended the interview" note.

  • Proposes a frontier-scale model on a phone.
  • Says "quantize it" with no quality tradeoff discussion.

—