Question bank / Data, Role Knowledge, Advanced
How to answer: How would you evaluate an on-device model when privacy prevents centralized raw-data collection?
Whether you can evaluate without collecting the raw data you would normally rely on.
What the interviewer is testing.
Privacy-preserving evaluation. Whether you can evaluate without collecting the raw data you would normally rely on.
An outline for this question.
- Start from the constraint: raw user data stays on the device.
- Evaluate on the device and send back only aggregate results, with noise added to protect individuals.
- Use a small, consented, or synthetic test set for the cases you cannot see in aggregate.
- Watch for gaps in coverage, since you cannot inspect individual failures, and say how you would catch them.
The mistake to avoid.
Asking to collect the data anyway. The constraint is the point of the question.
The general method: frame, evidence, recommendation.
The same shape works for most role knowledge questions, whatever the topic.
- Frame the problem: what is being decided and who is affected.
- Say what evidence you would gather first, and why that order.
- Give a clear recommendation and the risk you would watch.
Practice it, out loud.
Reading an outline is not the same as saying it. Answer this question in a mock interview, in your own words, then compare it with the outline above.
More to practice.
How do you investigate a production latency regression when no deployment is obvious?
Software Engineering, Role Knowledge
How would you prioritize a roadmap when sales, support, and engineering disagree?
Product, Role Knowledge
Explain the difference between correlation and causation to a non-technical stakeholder.
Data, Role Knowledge
Write SQL to calculate seven-day retention by signup cohort.
Data, Coding
This is a Replysis practice question. It is not a leaked or official question from any company, and the outline is a guide to structure, not a script. Use your own experience and facts.