Advanced Reinforcement Learning Interview Questions #1 - The Stationarity Trap

Assuming a fixed data distribution in RL quietly breaks learning because the agent’s behavior shifts state visitation faster than your optimizer can track.

You’re in a Machine Learning Engineer interview at Anthropic, and the interviewer drops this on you:

“In Supervised Learning, we assume data is IID (Independent and Identically Distributed). Why does applying this assumption to a Reinforcement Learning agent, like a coding assistant, cause catastrophic failure?”

Most of candidates say: “It fails because R…