You’re in a Senior AI Engineer interview at Google and the interviewer asks:
“Your RAG pipeline scores 90% on factual lookups but collapses on ‘summarize this 300-page report.’ Why, and don’t tell me it’s chunk size?”
Don’t say: “I’d increase the chunk size” or “I’d use a bigger context window.” Wrong axis entirely. You just told them you think this is a resolution problem.
The reality: the theme isn’t in any chunk. So retrieval can’t retrieve it.
Fixed contiguous chunks give you a flat index. Retrieval can only return text that already exists. But a global theme is an emergent property distributed across the whole corpus, it lives nowhere. Top-k over a flat index hands the LLM 5 local facts and asks it to induce a global claim from a biased sample.
You’re asking for a satellite photo and getting handed 20 close-ups of pavement.
The fix isn’t retrieving abstraction. It’s building it.
Recursive clustering + summarization (RAPTOR-style):
