Advanced Deep Learning Interview Questions #3 - The Leaderboard Overfitting Trap

Winning offline metrics hides the fact that production systems penalize inference latency, orchestration overhead, and failure surface area.

You’re in a Senior ML Engineer interview at Meta and the interviewer asks:

“Your team just ensembled 12 different deep learning models to squeeze out an extra 2% accuracy and secure the top spot on our internal leaderboard. Why is directly deploying this ‘winning’ submission a terrible idea for our live system, and what technique do you use instead?”

Mos…