LLM System Design Interview #5 - The FLOPs Compression Hack

Byte tokenizers look simple - until they 16x your attention cost. The real reason every frontier model still relies on subword compression.

You’re in a AI Engineer interview at OpenAI and the interviewer asks:

“Why don’t we just use a simple byte-based tokenizer? It has a fixed 256-token vocabulary, it’s simple, and it never has an ‘unknown’ token. Why are we still using a complex BPE tokenizer?”

Most candidates say: “Because BPE groups related characters into subwords that have more semanti…