You’re in a ML Engineer interview at OpenAI and the interviewer asks:
“Your team is hitting OOM errors. An intern engineer proposes casting the entire model and optimizer state to bfloat16 to cut memory usage by 50%. Why is this a ticking time bomb that will cause training to go out of control, and what components must stay in FP32?”
Most candidates say:
“…
