The JSON Tax on Token Streaming
For four years, the generative AI boom relied on the clumsiest possible data interchange format: JSON strings wrapped in Server-Sent Events (SSE). Every single token emitted by an inference cluster was stringified, wrapped in quotes, escaped with backslashes, flushed over HTTP/1.1 chunks, and parsed on the client side with JSON.parse.
When you stream a conversational paragraph, JSON overhead is merely wasteful. But when you orchestrate modern multi-agent systems—streaming AST trees, diff buffers, canvas vectors, and tool calls—JSON becomes an outright performance bottleneck.
The V8 Garbage Collector Trap
Streaming 100,000 JSON chunks into a browser UI creates hundreds of thousands of ephemeral intermediate string allocations. On mobile devices, this triggers aggressive V8 garbage collection sweeps, dropping UI frame rates from 120fps down to 40fps.
The Benchmark: JSON vs Binary Frame Protocols
To measure the true serialization penalty, I benchmarked streaming a 20,000-node Abstract Syntax Tree across four serialization standards on identical hardware:
| Serialization Format | Wire Payload Size | Client Parse Time | Memory Allocations | Zero-Copy Decoding |
|---|---|---|---|---|
| Standard UTF-8 JSON | 4.82 MB | 84.2 ms | 148,000 objects | No (Full Parse) |