RAG Interview Questions #9 - The Equal Split Trap

Why arbitrary chunking tears correlated dimensions apart and quietly ruins vector search, and how rotating your embeddings saves your latency-recall tradeoff.

You’re in a Senior ML Engineer interview at Pinterest and the interviewer asks:

“You’re building a Product Quantization index. You split each vector into M subvectors and run k-means on each. A teammate says the split point doesn’t matter because the information is spread uniformly. Do you ship it?”

Don’t say: “Sure, embeddings are dense, so any equal split works.”

Here’s the reality: that assumption is the fastest way to silently tank your recall.

Real embeddings are not uniform. Transformer outputs are anisotropic, variance is unevenly distributed, and dimensions are correlated. So when you slice into contiguous chunks:

• Some subspaces inherit almost all the signal. Their 256-centroid codebook can’t capture it → high distortion.

• Other subspaces are nearly flat. Their codebook is wasted on noise → dead capacity.

• Correlated dimensions get torn across subspaces, so independent k-means can never reconstruct the structure.

Think of slicing a cake where all the frosting sits on one side. Cut into equal pieces by position and you get some all-frosting, some all-sponge. Nobody’s happy.

The senior move: stop assuming uniformity, engineer it.

• Apply OPQ (Optimized Product Quantization): learn a rotation matrix that redistributes variance evenly across subspaces before splitting. Now every codebook does equal work, and the split point genuinely stops mattering.

And how do you actually pick M? You don’t derive it. You sweep it.

• M is a memory-vs-recall dial, not a constant. With k=256, code size = M bytes/vector.

• Bigger M → finer granularity, better recall, more memory + slower distance math.

• Constrain M so D is divisible by it and the byte layout stays SIMD-aligned (multiples of 4/8),r that alignment is where your latency actually lives.

• Then plot recall@k vs. latency vs. M on your data. The original PQ paper swept it for a reason.

---

The answer that gets you hired: “Uniform information is an assumption I’d never trust on real embeddings, I’d rotate with OPQ to balance variance across subspaces, then sweep M empirically against recall@k and SIMD-aligned latency, not pick it by feel.”

That’s the line that separates someone who read the PQ paper from someone who’s shipped it.

---

#MachineLearning #VectorSearch #RAG #MLEngineering #AI #SystemDesign #Embeddings

---