Day to day, the difference comes from the harness more than the model

The harness is everything you build around the model to put it to work. You write it once, you own it, and it is not re-billed per use.

The harness is everything around the model: the tools it can reach, what it keeps in memory from one turn to the next, what it does when a step fails, the rules it has to follow, and how a task is broken into steps. The model produces text. The harness decides what the model sees, what it can act on, and what happens next.

On a single-step task, the model explains almost all of the result. On a twenty-step task, it explains only part of it. The rest sits outside the model: whether the AI agent could reach the files, whether it read the error output, whether it retried, whether it stopped after three failures instead of burning through more calls.

Software engineering leaderboards show this in the way they report results. On SWE-bench Verified, a submission is never a model on its own: it is a pair, a model and the harness driving it. The same model shows up in several places on the board, with different scores, depending on what was built around it.

The harness also decides the bill, because it decides what gets re-read on every call. An AI agent that sends its whole history back on every turn pays for that re-reading in full, every turn. In agentic loops, re-reading commonly exceeds 130 input tokens for every generated token. A harness that summarizes, drops what is no longer useful, or leans on the prefix cache brings that number down without changing a line of the model.

The difference between the two costs matters more than their size. The harness is a fixed cost: written once, kept in your repository, improved every time someone fixes a failure case. The model is a variable cost: paid on every call, every day, indefinitely.

What follows is simple: switching models does not throw the harness away. Tools, internal rules, task decomposition and failure recovery do not depend on the provider. Pointing a client like Claude Code at a different endpoint takes three environment variables. What is expensive to build stays; what is billed per token gets replaced.

That changes the order of decisions when an AI agent fails. The first reaction is often to reach for a bigger model, which raises the bill immediately and permanently. The question that pays off more is elsewhere: did the AI agent have the right tool, the right context, and a way to recover from the error.

To settle it, isolate the two. Keep the harness identical and swap only the model, on the same tasks, under the same rules. That is the only way to learn which of the two is holding you back. Otherwise every trial changes two things at once and proves nothing.