LLM Inference Interview Questions #7 - The Summarization Paradox

How your cost-saving context condenser is secretly fighting your prompt cache, and why accepting an expensive hard reset is cheaper than constantly re-editing history.

You’re in a Senior AI Infrastructure Engineer interview at Anthropic, and the interviewer asks:

“You enabled prompt caching on a 100-step agent trajectory expecting a 5–10x cost drop. In production you’re seeing barely 1.3x, and most per-step content is cache-missing. What’s actually happening, and why does the order of your context decide whether cachin…