The flexibility premium: what the right to consume at any time costs

With usage billing, part of the price does not pay for compute. It pays for the capacity the provider keeps free to absorb spikes, and it is spread across every token billed.

A provider that bills by usage sells two things in the same price: the compute you consume, and the right to consume it whenever you want, without notice. The second part has a cost, and that cost is spread across every token billed.

The cost comes from the reserve. For one customer to triple their consumption on a Tuesday morning, GPUs have to be free at that moment. An idle GPU costs the same as a GPU that is computing: the machine is rented or amortized by the hour, and the rent does not drop when nobody is using it. The provider fronts that capacity and recovers it in the rate.

The arithmetic is direct: a server's cost per token is inversely proportional to its occupancy rate. An 8-GPU B200 server generates up to 7,800 tokens per second, or roughly 20 billion tokens in a month if it runs without stopping. Its operating costs then work out to about $1 per million generated tokens, once the capital is repaid. The same server at 40 % average occupancy produces 8 billion tokens, and the same costs work out to $2.50 per million. The $1.50 difference pays for no additional compute. It pays for the hours the machine spent waiting.

Customer spikes do not cancel each other out. A provider serving thousands of accounts gets real smoothing, but only partial: agentic workloads look alike and cluster in the same office hours, in the same time zones. The reserve stays necessary, and average occupancy stays well below 100 %.

The premium shows up in published prices. Z.ai, the model's maker, bills GLM 5.2 at US$4.40 per million generated tokens; Alibaba Cloud bills the same model at US$3.04*. Both serve it in FP8: the weights, the precision and the compute are the same. The 45% gap comes from the service level: capacity held available, latency held under peak, higher rate limits.

Comparing against a closed model mixes two things together. Anthropic bills Sonnet at US$15 per million generated tokens*, roughly CA$20.55. That price carries the same flexibility premium, but also the cost of training the model and the margin of a product with no exact substitute. The whole gap cannot be attributed to the reserve.

The premium is a good buy when load is irregular: a few thousand calls a day, unpredictable spikes, long stretches with nothing. You are buying elasticity and you are using it. It gets expensive when load is predictable. A fleet of AI agents running every day, at the same hours, at the same pace, pays for every token as if tomorrow could be ten times bigger.

The ratio to watch is simple: your peak throughput divided by your average throughput, measured over two or three weeks. At 2 or 3, the reserve you are funding almost never serves you, and reserved throughput costs less. At 20, elasticity is worth its price. There are two ways to measure that ratio: your provider's console, or a gateway placed in front of your calls.

One caveat on method: providers' average occupancy rates are not public, and the figures above are orders of magnitude. What you can verify on your side is your own ratio between peak and average.

* September 2026 prices, at regular price with no promotion, as posted on OpenRouter. We compare providers that serve GLM 5.2 at the same precision, FP8: an FP4 version processes about twice as many tokens per GPU, at reduced precision.