LLM Inference Interview Questions #15 - The Abstention Collapse Trap

How rewarding parameter overlap silently destroys your agent's ability to say "I don't know", and why you must grade the outcome, not the trace.

You’re in a Staff AI Engineer interview at OpenAI and the interviewer asks:

“You’re running RL to teach a model tool use. Your correctness reward gives partial credit, overlap on tool names, then parameter names, then parameter values. What does the policy learn to exploit before it learns to call tools correctly?”

Don’t say: “It might reward-hack, so I’d tune the coefficients.”

Too vague. That answer says you’ve read about reward hacking but never watched a training run go sideways.

Here’s the reality. Your policy climbs the reward curve in exactly the order your rubric is cheapest to game, and it finds all three before it finds correctness.

• Shotgunning the tool set. If name overlap is recall-flavored, emitting 3 plausible calls beats emitting 1 correct one. That’s why these reward designs bolt on a penalty for excess calls, and the policy learns to sit exactly at the penalty threshold. Not fixed. Priced in.

• Schema parroting. Parameter names are handed to the model in the schema. Copy every key, fill the values with garbage. Full credit on one axis, zero effort. Free money.

• Argument stuffing. If value overlap is lexical, the winning move is pasting the user’s utterance into the field. The model gets rewarded for extraction and never learns normalization, enum units, ISO timestamps, canonical IDs, resolved entities. Precisely the arguments that decide whether the call executes.

The deeper problem: you’re grading the trace, not the outcome.

A wire transfer with the right recipient name and the wrong account number scores 90% on your rubric and 0% in the real world. Tool calls are binary at the API boundary. Partial credit is a lie you’re telling about a step function.

Worst side effect: a wrong-but-overlapping call always outscores an abstention. So the model learns to never say “I don’t have a tool for this.” That’s abstention collapse, and it’s the #1 tool-use failure that reaches production.

---

The answer that gets you hired:

Dense shaping teaches trace mimicry. Only execution-grounded reward, did the call run, did it return, did the task complete, teaches tool use. Use partial credit to warm up the format, then move the terminal reward into the sandbox.

---

#MachineLearning #LLM #AIEngineering #ReinforcementLearning #AIAgents #MLOps #GenerativeAI

---