LLM Agents Interview Questions #1 - The Privacy Scaling Trap

Increasing capacity without increasing data diversity shifts optimization toward memorization modes that validation loss won’t detect but adversarial prompting will expose.

You're in a Senior AI Engineer interview at NVIDIA. The interviewer sets a trap:

"Your team just upgraded an internal LLM from a 7B to a 70B parameter model using the exact same training dataset and 100k step schedule. You expect a reasoning bump, but SecOps flags a 400% spike in PII ( Personally Identifiable Information ) extraction via simple prompting. Why does scaling up independently degrade privacy, and how do you fix it without rolling back?"

90% of candidates walk right into it.

Most candidates say: "The larger model overfit because we didn't increase the dataset size. We need to apply early stopping, increase dropout, or just run regex data scrubbing on the corpus."

---

But they aren't optimizing for standard overfitting. They are fighting the physics of model capacity.