- Databricks’ internal benchmark found the agentic coding “harness” — the tool wrapping a model — affects total cost as much as the model choice itself.
- On MegaOne’s Engine Score, Claude Code and Lovable lead the code-assistant category at 9 out of 10, with Cursor, GitHub Copilot, Devin, and Replit at 8.
- The open-source harness Pi was among the best at managing context, delivering low cost without sacrificing quality.
- The decision that minimizes cost per merged pull request is model-and-harness together, not either one alone.
The most repeated mistake in agentic coding is treating the model as the whole decision. In 2026, the evidence says it is closer to half. When Databricks benchmarked AI coding across the real tasks its 3,000 engineers perform, it found something that surprised even its own team: the harness — the agentic tool that wraps a model and manages its context and instructions, like Claude Code, Codex, or an open-source alternative — affected total cost as much as the choice of model. “Model choice is only one piece of the puzzle,” the company concluded.
What the harness actually does
A harness is the layer between the raw model and the codebase. It decides how much context to feed the model on each call, which files to retrieve, how to structure the instructions, and when to stop. Those decisions determine how many tokens a task consumes — and because agentic coding runs many model calls per task, small differences in context management compound into large differences in the bill. Databricks found the open-source harness Pi to be among the best at managing the context around each prompt, and therefore one of the lowest-cost choices without sacrificing quality. The lesson is that a well-designed harness on a cheaper model can beat a naive harness on an expensive one.
The capability side: MegaOne’s Engine Scores
On the tools themselves, MegaOne’s Engine Score — an independent 1-to-10 rating — puts Claude Code and Lovable at the top of the code-assistant category at 9 out of 10. A tier below, at 8, sit Cursor, GitHub Copilot, Devin, and Replit; at 7 are Aider, Windsurf, Qodo, and Amazon CodeWhisperer. These scores rate the tool as a whole — the harness plus its default model plus its workflow — which is exactly the unit that determines real-world productivity. They are the capability axis; model price is the cost axis; and the right choice lives at the intersection, not on either axis alone.
The cost math that actually matters
The metric to optimize is not price per token or even model capability in isolation — it is cost per merged pull request. That number is a product of three things: how capable the model is (fewer retries), how efficient the harness is (fewer wasted tokens per step), and how much the model charges per token. A frontier model like Claude Opus 5 at $50 per million output tokens inside an efficient harness can cost less per shipped feature than a cheaper model inside a wasteful one that loops and re-reads context. Conversely, a cheap open-weight model in a well-tuned harness can undercut both.
How to actually choose
For a solo developer, the decision is usually a high-scoring integrated tool — Claude Code or Cursor — on a strong default model, because the productivity gain dwarfs the token cost at individual scale. For an organization running thousands of engineers, the Databricks approach is the right one: benchmark model-and-harness combinations on your own real tasks, measure cost per completed task rather than per token, and be willing to pair an open-weight model with an efficient open-source harness where it wins. MegaOne’s Engine Scores rank the harnesses and the comparisons break down the tradeoffs. The one approach guaranteed to overpay is picking a model first and treating the harness as an afterthought.
