Machine Learning System Design Interview #9 - The Catastrophic Forgetting Trap
Why fine-tuning on new data without old samples silently erases your model - and how EWC saves the day.
You’re in a AI Research Engineer interview at OpenAI and the lead researcher drops this scenario.
“We deleted our original training dataset for GDPR compliance. We need to teach the live model a new class of data today. How do you do it?”
Most candidates say...
“Easy. I’ll load the latest model checkpoint and fine-tune it on the new data. Maybe I’ll lower the learning rate to be safe.”
Wrong. They just lobotomized your model.
By fine-tuning exclusively on new data without the old data present, they have triggered 𝐂𝐚𝐭𝐚𝐬𝐭𝐫𝐨𝐩𝐡𝐢𝐜 𝐅𝐨𝐫𝐠𝐞𝐭𝐭𝐢𝐧𝐠.
The model optimizes for the new task by overwriting the weights that were critical for the old tasks. You didn’t “add” a feature, you traded one skill for another.
------
𝐓𝐡𝐞 𝐑𝐞𝐚𝐥𝐢𝐭𝐲: This isn’t a training problem; it is a 𝐏𝐥𝐚𝐬𝐭𝐢𝐜𝐢𝐭𝐲-𝐒𝐭𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐃𝐢𝐥𝐞𝐦𝐦𝐚.
