Here is a junior developer's design for an inference batching system. Review it and explain what you would change.

No reference answer yet

This question is in the bank but has no rubric written for it. It can be read and practised, but not scored.