Machine Learning System Design Interview #19 - The Database-as-Queue Trap
How relying on a persistence layer for real-time inference creates silent bottlenecks, and how event brokers fix them.
Youโre in a Machine Learning System Design interview at Google. The interviewer sets a trap:
โWe have 3 upstream microservices generating features. They write to a central ๐๐ฐ๐ด๐ต๐จ๐ณ๐ฆ๐ด ๐๐. Your ML Service queries that DB to get the input vector for inference. How do we scale this to 50k requests per second?โ
90% of candidates walk right into the trap. They start optimizing the SQL.
They immediately focuses on the database performance. They suggest:
- Adding aggressive Read Replicas to handle the load.
- Implementing complex caching layers (Redis) to offload the DB.
- Sharding the Postgres instance based on UserID.
It feels right. ๐๐ง ๐ต๐ฉ๐ฆ ๐ฅ๐ข๐ต๐ข๐ฃ๐ข๐ด๐ฆ ๐ช๐ด ๐ด๐ญ๐ฐ๐ธ, ๐ง๐ช๐น ๐ต๐ฉ๐ฆ ๐ฅ๐ข๐ต๐ข๐ฃ๐ข๐ด๐ฆ, ๐ณ๐ช๐จ๐ฉ๐ต?
But they arenโt solving a ๐๐ฎ๐๐ซ๐ฒ ๐๐ฉ๐ญ๐ข๐ฆ๐ข๐ณ๐๐ญ๐ข๐จ๐ง problem. They are solving a ๐๐จ๐ฎ๐ฉ๐ฅ๐ข๐ง๐ problem.
