LLM System Design Interview #3 - The Scaling Law Playbook

Why elite AI engineers never guess model size - and how to use small-scale experiments to de-risk six-figure training runs.

You’re in an AI Engineer interview at Anthropic and the interviewer asks:

“We have a fixed compute budget - 32 H100s for two weeks. To get the best possible model, should we train a larger model on less data, or a smaller model on more data? How do you begin to answer this?”

Most candidates freeze. They guess.

“Uh, I’d probably train a smaller model on mor…