Cloudflare showed how it squeezes frontier-size open models onto fewer GPUs
An engineering writeup with real production numbers: FP8 KV cache quantization doubled Kimi K2.6's token capacity, and weight compression cut GLM 5.2 from 705GB to 421GB with under 0.8 points of accuracy loss.
Cloudflare shrank a 705GB open model to 421GB, served it 55 percent faster, and published every number.
Cloudflare published an engineering writeup on August 3, 2026 showing how Workers AI serves two frontier-size open models, Moonshot's Kimi K2.6 and Z.ai's GLM 5.2, on fewer GPUs than a naive deployment would need. The techniques are standard, FP8 quantization and weight compression, but the numbers are unusually public for production infrastructure.
The KV cache numbers
For Kimi K2.6, Cloudflare quantized the KV cache from BF16 to FP8. The same hardware now holds 1.37 million tokens of context instead of 686,000, roughly double, and peak throughput reached 2,192 tokens per second at 64 concurrent requests, 41 percent above the BF16 peak. Cloudflare puts the result at 30 percent lower cost per token. The KV cache is usually what actually limits an inference server, because every concurrent conversation holds its context in GPU memory, so doubling capacity there means half the machines for the same traffic.
The weight compression numbers
For GLM 5.2, compression shrank the model from 705GB to 421GB, a 40 percent cut, which drops per-GPU memory from 88GB to 52GB across an 8-way tensor-parallel deployment. Decode speed went up 55 percent at single concurrency and 16 to 27 percent under load, because moving fewer bytes per token is also faster. Measured accuracy loss: under 0.8 points across their benchmarks. Serving runs on SGLang, the open source inference framework, on H200s, with Blackwell validation underway. The numbers are Cloudflare's own, and the accuracy figure summarizes their benchmark suite, not every workload; quantization hits some tasks harder than the average suggests.
Why a build studio cares
Our builds run on Cloudflare, and Workers AI is the shortest path from a static export to a model call without running our own inference. This post is the supply side of that product: the cheaper Cloudflare can serve open models, the longer the free and cheap tiers stay generous, and the more viable open weights become as the default choice instead of the budget one. It is also a rare look at real production serving economics with actual numbers attached.
Next step: read Cloudflare's full writeup. If you are choosing between open and closed models for a build and want the cost math done properly, write to us at hello@gattyworks.com.