Claude Code, Codex CLI, Pi, and OpenCode each win on different models. Runta's eval found harness choice swinging cost up to 4.4x on the same model.
Vamsi Tallapudi
Manager, Architect Technology at Cognizant
There's no single "best" coding agent harness in 2026. The honest answer is that it depends entirely on which model you're pointing it at, and the gap between a good and a bad pairing is bigger than most people assume. Runta's FrontierHarness Eval v1.1, which ran 22 harnesses against 5 open models across 62 terminal tasks (about 27,280 runs total), found cost-per-pass swinging up to 4.4x just from changing the harness while holding the model fixed. Codex CLI was the cheapest option paired with GLM-5.3 and then fell to 16th place with Kimi-K3, using the exact same model weights.
That's the thing nobody tells you when they post a "Claude Code vs Codex vs Pi vs OpenCode" ranking: those rankings only hold for the model and task set they tested. Swap the model and the leaderboard reshuffles. Composio put it bluntly in one of its benchmark threads: "before you switch models to cut agent costs or fix reliability, test another harness." I'd go further. Test both together, because they're not independent variables.
I spent the week going through Composio's harness benchmarks, the Runta eval, and my own notes from running all four tools against the current model lineup (Claude Opus 5.5, Claude Fable 5.2, GPT-6.1 Sol, Kimi K3.1 Pro). Here's what actually holds up.
All four do roughly the same job: give a model a terminal, a file system, and a loop that keeps calling it until the task's done. The differences are in what they bolt on around that loop.
AGENTS.md instead of CLAUDE.md (though Claude Code now falls back to AGENTS.md too, since v2.1.277). Codex's usage is folding into the same token pool as regular ChatGPT chat starting December 2026, so heavy Codex sessions will eat into your ChatGPT allowance in a way they didn't before.If your model is Claude, Opus 5.5 or Fable 5.2, Claude Code is the obvious pick and I'm not going to pretend otherwise. (If you're deciding between those two Claude models in the first place, see Opus 5 vs Fable 5.) The subagent and skills system is built around how Claude actually behaves, and nothing else matches its permission model for letting an agent loose on a real codebase without babysitting every command.
The catch is cost. Independent testing (and Composio's own numbers) consistently put Claude Code at the expensive end: one Kimi K3 comparison found it costing roughly 4x what Hermes Agent cost for the same tasks, and a separate 30-task benchmark clocked it around $0.195 per successful task against Pi's $0.028 on the same model. Claude Code doesn't optimize for token efficiency, it optimizes for not making you think about token efficiency, and you pay for that difference.
Codex CLI's sandbox is the real selling point. It runs shell commands in a contained environment by default, which matters more than people give it credit for the first time an agent runs rm on a path it misread. Paired with GPT-6.1 Sol (its default, at $2/$10 per million tokens), it's competent and predictable.
Pair it with anything else and the story changes fast. In Composio's Kimi K3 test, Codex ranked dead last on success rate despite sitting mid-pack on speed and cost. In the Runta eval, it tied for the top pass rate (61.9%) across the whole test but was 16th out of 22 specifically on Kimi-K3. Codex CLI is tuned for OpenAI's own models, which, to be fair, is exactly what you'd expect from the vendor's own harness. Just don't assume it travels.
Pi has no sandbox, a tiny system prompt, and basically no built-in guardrails, and it keeps showing up at or near the top of independent benchmarks anyway. In a Composio test running DeepSeek V4 Pro across five harnesses (Claude Code, OpenCode, Hermes, Pi, and DeepSeek's own harness), Pi solved the most tasks. In a 30-task set on DeepSeek V4 Flash, a community fork called Oh My Pi topped the field at 17/30, ahead of Claude Code and Codex (both 16/30) and OpenCode (14/30).
My take: the minimal-prompt theory holds up better than I expected going in. Frontier and near-frontier models are already trained to behave like coding agents, so a thin harness that gets out of the way lets the model's own judgment do more of the work, instead of fighting a 20,000-token system prompt telling it how to be an agent. It's not free of downsides. No sandbox means you're trusting the model's judgment on destructive commands, and the four-tool minimalism means anything fancier (say, structured diff review) is a plugin you have to go find or write yourself.
OpenCode's pitch, any model through one open harness, is still the right pitch. The execution had a rough 2026. Beyond the October Console breach, Composio's benchmarks have it trailing the pack more often than not: last place in the DeepSeek V4 Flash comparison, and generally a step behind Claude Code, Codex, and Pi variants in head-to-head testing. If you're running it, keep it local-only (never link provider keys through Console), and don't expect it to out-benchmark the competition yet. It might get there. It's not there now.
| Model you're running | Best-fit harness | Why |
|---|---|---|
| Claude Opus 5.5 / Fable 5.2 | Claude Code | Native subagents, skills, and permissions built around Claude's behavior |
| GPT-6.1 Sol / GPT-6 Astra | Codex CLI | Default sandbox, same-vendor tuning, shares your ChatGPT usage pool |
| Kimi K3.1 Pro / open-weight models | Pi or the model's native harness | Repeatedly tops independent benchmarks on non-Anthropic models |
| Mixed fleet, self-hosted, or cost-first | OpenCode (cautiously) or Pi | Broadest provider support, but check the benchmarks for your specific model first |
Mostly the model. The Runta eval attributed 93% of cost variation to model choice and the rest to harness. But "the rest" is still a 4.4x swing and an 11-position shuffle in the median harness ranking depending on which model you pick, which is not a rounding error if you're running agents at any real volume. The practical rule: every time you change models, re-test your harness before you assume the old pairing still works. It usually doesn't transfer as cleanly as you'd hope.
New AI tools, automation workflows, and course drops — straight to your inbox. Join 2,400+ builders.
lot-map is a free Claude Code plugin that draws a live map of tool calls, subagents, hooks and context usage beside your chat. Here's how to install and use it.
Gemini 4 Neon launches at $3/$15 per million tokens with a 1M context window, slotting between Hydrogen and Carbon. Here's what the pricing and benchmarks actually mean.
MAI Code 2 claims 96% of Claude Opus 5.5's coding performance at 5% of the price in GitHub Copilot. We checked the benchmarks, the promo pricing, and real-world reports.
Comments