Coderefercoderefer
CoursesWebinarsBlogAbout
Coderefercoderefer

Discover. Learn. Automate. Grow.

Learn

  • Courses
  • Webinars
  • Blog
  • Search

Company

  • About
  • Contact
  • FAQ

Legal

  • Privacy Policy
  • Terms of Service
  • Refund Policy
© 2026 Coderefer. All rights reserved.
HomeBlogClaude Code vs Codex CLI vs Pi vs OpenCode: Best Harness by Model in 2026
ai-toolsNovember 19, 20267 min read

Claude Code vs Codex CLI vs Pi vs OpenCode: Best Harness by Model in 2026

Claude Code, Codex CLI, Pi, and OpenCode each win on different models. Runta's eval found harness choice swinging cost up to 4.4x on the same model.

V

Vamsi Tallapudi

Manager, Architect Technology at Cognizant

Claude Code vs Codex CLI vs Pi vs OpenCode: Best Harness by Model in 2026

There's no single "best" coding agent harness in 2026. The honest answer is that it depends entirely on which model you're pointing it at, and the gap between a good and a bad pairing is bigger than most people assume. Runta's FrontierHarness Eval v1.1, which ran 22 harnesses against 5 open models across 62 terminal tasks (about 27,280 runs total), found cost-per-pass swinging up to 4.4x just from changing the harness while holding the model fixed. Codex CLI was the cheapest option paired with GLM-5.3 and then fell to 16th place with Kimi-K3, using the exact same model weights.

That's the thing nobody tells you when they post a "Claude Code vs Codex vs Pi vs OpenCode" ranking: those rankings only hold for the model and task set they tested. Swap the model and the leaderboard reshuffles. Composio put it bluntly in one of its benchmark threads: "before you switch models to cut agent costs or fix reliability, test another harness." I'd go further. Test both together, because they're not independent variables.

I spent the week going through Composio's harness benchmarks, the Runta eval, and my own notes from running all four tools against the current model lineup (Claude Opus 5.5, Claude Fable 5.2, GPT-6.1 Sol, Kimi K3.1 Pro). Here's what actually holds up.

What's different between these four harnesses?

All four do roughly the same job: give a model a terminal, a file system, and a loop that keeps calling it until the task's done. The differences are in what they bolt on around that loop.

  • Claude Code is Anthropic's own CLI. Big system prompt (14,000 to 24,000 tokens depending on loaded tools), 10+ built-in tools, markdown-defined subagents with their own context windows, a skills system, hooks, and the deepest permission controls of the four. It's also gotten noticeably cheaper to run lately: Claude Code's background subagents now write their status-line updates with Haiku 5.5 instead of re-reading the whole conversation, which Anthropic says cut that specific cost by about 99%.
  • Codex CLI is OpenAI's, Apache 2.0 licensed, and ships with an OS-level sandbox on by default, genuinely the safest of the four if you're pointing it at a repo you don't fully trust. It reads AGENTS.md instead of CLAUDE.md (though Claude Code now falls back to AGENTS.md too, since v2.1.277). Codex's usage is folding into the same token pool as regular ChatGPT chat starting December 2026, so heavy Codex sessions will eat into your ChatGPT allowance in a way they didn't before.
  • Pi is the opposite philosophy: four tools (read, write, edit, bash), a system prompt under 1,000 tokens, and everything else as an opt-in TypeScript extension. Built by Mario Zechner, now maintained by Earendil Inc (co-founded by Flask creator Armin Ronacher). MIT licensed, connects to 15+ model providers, no built-in sandbox. JetBrains built a whole GUI, ThinkRail, just to wrap it.
  • OpenCode is the bring-your-own-everything option: any provider, 1000+ models, LSP diagnostics wired in for better refactor feedback. Good idea, rough year. OpenCode Console got breached in October, exposing stored API keys for 1.4% of users, on top of a January CVE for an unauthenticated local server and a separate dispute with Anthropic over spoofing Claude Code's client identity.

Claude Code: still the default if you're already paying Anthropic

If your model is Claude, Opus 5.5 or Fable 5.2, Claude Code is the obvious pick and I'm not going to pretend otherwise. (If you're deciding between those two Claude models in the first place, see Opus 5 vs Fable 5.) The subagent and skills system is built around how Claude actually behaves, and nothing else matches its permission model for letting an agent loose on a real codebase without babysitting every command.

The catch is cost. Independent testing (and Composio's own numbers) consistently put Claude Code at the expensive end: one Kimi K3 comparison found it costing roughly 4x what Hermes Agent cost for the same tasks, and a separate 30-task benchmark clocked it around $0.195 per successful task against Pi's $0.028 on the same model. Claude Code doesn't optimize for token efficiency, it optimizes for not making you think about token efficiency, and you pay for that difference.

Codex CLI: safest sandbox, worst at everything that isn't OpenAI's model

Codex CLI's sandbox is the real selling point. It runs shell commands in a contained environment by default, which matters more than people give it credit for the first time an agent runs rm on a path it misread. Paired with GPT-6.1 Sol (its default, at $2/$10 per million tokens), it's competent and predictable.

Pair it with anything else and the story changes fast. In Composio's Kimi K3 test, Codex ranked dead last on success rate despite sitting mid-pack on speed and cost. In the Runta eval, it tied for the top pass rate (61.9%) across the whole test but was 16th out of 22 specifically on Kimi-K3. Codex CLI is tuned for OpenAI's own models, which, to be fair, is exactly what you'd expect from the vendor's own harness. Just don't assume it travels.

Pi: the one that keeps winning benchmarks it shouldn't be able to

Pi has no sandbox, a tiny system prompt, and basically no built-in guardrails, and it keeps showing up at or near the top of independent benchmarks anyway. In a Composio test running DeepSeek V4 Pro across five harnesses (Claude Code, OpenCode, Hermes, Pi, and DeepSeek's own harness), Pi solved the most tasks. In a 30-task set on DeepSeek V4 Flash, a community fork called Oh My Pi topped the field at 17/30, ahead of Claude Code and Codex (both 16/30) and OpenCode (14/30).

My take: the minimal-prompt theory holds up better than I expected going in. Frontier and near-frontier models are already trained to behave like coding agents, so a thin harness that gets out of the way lets the model's own judgment do more of the work, instead of fighting a 20,000-token system prompt telling it how to be an agent. It's not free of downsides. No sandbox means you're trusting the model's judgment on destructive commands, and the four-tool minimalism means anything fancier (say, structured diff review) is a plugin you have to go find or write yourself.

OpenCode: good idea, bad year

OpenCode's pitch, any model through one open harness, is still the right pitch. The execution had a rough 2026. Beyond the October Console breach, Composio's benchmarks have it trailing the pack more often than not: last place in the DeepSeek V4 Flash comparison, and generally a step behind Claude Code, Codex, and Pi variants in head-to-head testing. If you're running it, keep it local-only (never link provider keys through Console), and don't expect it to out-benchmark the competition yet. It might get there. It's not there now.

Which harness actually fits which model, in 2026

Model you're runningBest-fit harnessWhy
Claude Opus 5.5 / Fable 5.2Claude CodeNative subagents, skills, and permissions built around Claude's behavior
GPT-6.1 Sol / GPT-6 AstraCodex CLIDefault sandbox, same-vendor tuning, shares your ChatGPT usage pool
Kimi K3.1 Pro / open-weight modelsPi or the model's native harnessRepeatedly tops independent benchmarks on non-Anthropic models
Mixed fleet, self-hosted, or cost-firstOpenCode (cautiously) or PiBroadest provider support, but check the benchmarks for your specific model first

So is the model or the harness the bigger lever?

Mostly the model. The Runta eval attributed 93% of cost variation to model choice and the rest to harness. But "the rest" is still a 4.4x swing and an 11-position shuffle in the median harness ranking depending on which model you pick, which is not a rounding error if you're running agents at any real volume. The practical rule: every time you change models, re-test your harness before you assume the old pairing still works. It usually doesn't transfer as cleanly as you'd hope.

Watch

Claude Code vs Codex vs OpenCode and four more: which coding agent to pick in 2026
ai-tools claude-code codex 2026

Comments

    Get AI tricks that save you hours every week

    New AI tools, automation workflows, and course drops — straight to your inbox. Join 2,400+ builders.

    Read Next

    How to Use lot-map in Claude Code: A Live Map of Tool Calls, Subagents, Hooks and Context Usage
    ai-tools

    How to Use lot-map in Claude Code: A Live Map of Tool Calls, Subagents, Hooks and Context Usage

    lot-map is a free Claude Code plugin that draws a live map of tool calls, subagents, hooks and context usage beside your chat. Here's how to install and use it.

    November 19, 20268 min read
    Gemini 4 Neon Is Here: $3/$15 Pricing, 1M Context, and Where It Fits Between Hydrogen and Carbon
    ai-tools

    Gemini 4 Neon Is Here: $3/$15 Pricing, 1M Context, and Where It Fits Between Hydrogen and Carbon

    Gemini 4 Neon launches at $3/$15 per million tokens with a 1M context window, slotting between Hydrogen and Carbon. Here's what the pricing and benchmarks actually mean.

    November 19, 20269 min read
    MAI Code 2 vs Claude Opus 5.5 in GitHub Copilot: Worth Switching?
    ai-tools

    MAI Code 2 vs Claude Opus 5.5 in GitHub Copilot: Worth Switching?

    MAI Code 2 claims 96% of Claude Opus 5.5's coding performance at 5% of the price in GitHub Copilot. We checked the benchmarks, the promo pricing, and real-world reports.

    November 18, 20267 min read
    View all posts →