LLM Inference Interview Questions #10 - The JSON Serialization Trap

Why stuffing multi-line code into JSON strings silently pushes your model off-manifold, and how separating the metadata envelope from the raw payload instantly stops escape-character hallucinations.

You’re in a Senior AI Engineer interview at Anthropic and the interviewer asks:

“Your coding agent keeps failing on multi-line Python, escaped quotes, mangled indentation. A teammate says just use a stronger model. What’s the real fix?”

Most candidates say: “Add a JSON repair step and retry the call.”

Wrong instinct. You just built a band-aid around a design flaw.

Here’s what’s actually happening.

This isn’t a reasoning failure. It’s a serialization failure. The model knows the code. It fails at the transport layer.

JSON demands every " becomes \". Every newline becomes \n. Every backslash doubles. Your model isn’t writing Python anymore, it’s writing an encoded version of Python, one escape token at a time.

You’re asking a chef to pass you a soufflé through a letterbox.

Three things break:

a) Out-of-distribution encoding. Pretraining saw billions of lines of raw Python. It saw far less \"\"\"Docstring.\"\"\"\n return x. You moved the payload off-manifold.

b) Silent corruption. A dropped backslash doesn’t throw. The JSON parses. The patch applies. The indentation is wrong, and CI finds out 40 minutes later.

c) Escape tax. Escaped code inflates token count, and every escape is another chance to break.

The fix, in priority order: