RAG Interview Questions #19 - The Concatenation Trap

Why stacking query rewrite strategies silently averages away user intent, and how conditional routing boosts precision without the massive inference tax.

You’re in a Senior ML Engineer interview at Google and the interviewer asks:

“Your conversational RAG has five query rewrite strategies. Why not just run all five and concatenate the outputs?”

Don’t say: “More context is always better, so combining them improves recall.” Wrong approach. That answer tells the interviewer you’ve never watched a rewrite pipeline degrade in production.

Here’s what’s actually happening.

Each rewrite strategy pulls your query vector in a different direction in embedding space. Stack them and you don’t get five signals, you get one blurred centroid that matches nothing well.

The reality: concatenation is not ensembling. It’s averaging away your intent.

Three concrete failure modes:

• Topic dilution. History-summary injects context from turn 3. The user just switched topics at turn 7. Now your retriever is scoring documents against a conversation the user has already abandoned. This is query drift, and it’s silent. Your latency looks fine. Your recall@k looks fine. Your answers are subtly wrong.