Question bank / Data, System Design, Advanced
How to answer: Design an experimentation platform that supports guardrail metrics and delayed outcomes.
Whether you understand that experiments fail in the design, not only in the maths.
What the interviewer is testing.
Assignment, metrics, bias, governance. Whether you understand that experiments fail in the design, not only in the maths.
An outline for this question.
- Start with what the platform must support: assignment, metrics, analysis and a record of decisions.
- Assign users randomly and consistently, with the same user always in the same group.
- Separate the metrics: one primary, a few guardrails that must not get worse, and some that arrive late.
- Handle delayed outcomes by waiting before reading them, and guard against peeking, bias and overlapping experiments.
The mistake to avoid.
Stopping at the statistics. Without guardrails and a record of decisions, teams ship harm and cannot explain why.
The general method: requirements, design, trade-offs.
The same shape works for most system design questions, whatever the topic.
- Pin down what the system must do, how many users, and what matters most (speed, cost, correctness).
- Sketch the main parts and how data moves between them, starting simple.
- Pick one or two parts to go deep on, and say what you gave up for each choice.
- Finish with how it fails and how you would know: monitoring, retries, limits.
Practice it, out loud.
Reading an outline is not the same as saying it. Answer this question in a mock interview, in your own words, then compare it with the outline above.
More to practice.
Design a notification service for email, SMS, and push at large scale.
Software Engineering, System Design
Design the product and success metrics for a collaborative document approval workflow.
Product, System Design
Explain the difference between correlation and causation to a non-technical stakeholder.
Data, Role Knowledge
Write SQL to calculate seven-day retention by signup cohort.
Data, Coding
This is a Replysis practice question. It is not a leaked or official question from any company, and the outline is a guide to structure, not a script. Use your own experience and facts.