System design interview question
"Design a conversational AI service. Users send messages, the model responds token by token as it generates, conversations have history, and inference runs on GPUs that are expensive and scarce. A hundred million users, and a single request can occupy a GPU for thirty seconds."
serving an expensive, stateful, streaming workload on a scarce resource. It inverts most web-serving intuitions: requests take seconds rather than milliseconds, the bottleneck is GPU memory rather than CPU, responses stream rather than complete, and the cost per request is measured in cents rather than fractions of a cent. The distinguishing content is batching, KV-cache management, and queueing under scarcity.
On Practick another engineer interviews you live for 60 minutes on a question from this bank. They have the full interviewer guide beside the call: the answers to your clarifying questions, the numbers to hand out, a ladder of follow-ups and a scoring rubric. You get the model solution once the session settles. It is free: interview someone for an hour and you have earned your own.
Book a free mock interviewAll 132 system design interview questions
System design mock interviews · DSA interview questions · Free mock interview practice · Browse interviewers · Sign in