Throughput or tokens: two ways to buy inference

Paying per token buys a quantity. Paying for throughput buys a capacity. They don't suit the same workloads.

Per-token billing is ideal for irregular load: a few thousand calls a day, unpredictable spikes, long quiet stretches. You pay only for what you consume, and elasticity is total.

It turns hostile the moment load becomes continuous. A fleet of AI agents running eight hours a day, five days a week consumes predictably — and still pays a rate designed to absorb the risk of unpredictability. You are funding someone else's elasticity.

Reserved throughput inverts the logic. You reserve a number of tokens per second, permanently available, and volume stops mattering. An autonomous AI coding agent runs at roughly 50 tokens/second in continuous generation: that's the unit that matters for sizing, far more than the monthly total.

The crossover is simple to compute. Take your current monthly spend, divide by the rate per million generated tokens, and reduce that to an average throughput across the month. If that average is close to the peak throughput you actually need, usage-based billing is costing you dearly for nothing. If it's far below, you have probably already throttled your AI agents.

That last point is the most important, and the hardest to see from the inside: a spend that looks reasonable is often the sign of a constrained workload, not an optimized one.