Advanced Reinforcement Learning Interview Questions #7 - The Dynamics Invariance Trap

Relabeling trajectories across environments with different contact mechanics poisons value estimates by assuming physics that never existed.

You’re in a Senior AI Engineer interview at Boston Dynamics. The interviewer sets a quiet trap:

“We have 10k hours of data from a robot walking on concrete. We want to use Hindsight Relabeling (HER) to bootstrap a new policy for walking on sand. Is this a good idea?”

90% of candidates walk right into the trap.

They say: “Absolutely. Data is expensive. Walk…