Generative Vision Interview Questions #20 - The Reward Hacking Trap

Why chasing a 30% jump in alignment scores silently turns your image model into an adversarial example generator, and why the KL penalty is the only leash that keeps optimization honest.

You’re in a Machine Learning Engineer interview at OpenAI and the interviewer asks:

“You fine-tuned your image model against a reward model. Your alignment scores jumped 30%. But humans say the outputs got worse. What happened and how do you stop it?”

Don’t say: “The reward model must be undertrained, I’d collect more preference data.”

You’re treating a sy…