Most "Codex vs Claude Code" comparisons quote vendor benchmarks, or one third-party token figure that gets copied from page to page. Few of them publish the tasks, the prompts or the commit, so their numbers cannot be checked. We did the boring version instead: four real tasks on one pinned open-source repository, each run twice per CLI, headless, with the exact commands, prompts and numbers below, and the raw data to download. Measured on 2026-10-06 with Claude Code 2.1.291 (claude-opus-5-5) and Codex CLI 0.160.0 (gpt-6.1-sol), default models, default effort.

TL;DR

• Quality: a tie. All 16 runs passed their objective checks, and all four reviews caught the 3 planted bugs with no false alarm.
• Cost: Codex wins. $0.074 per passed task against $0.181 for Claude Code at API list prices, 2.4x cheaper, and cheaper on every single task.
• Speed: close. Claude Code was faster on the bug fix and the feature, the refactor was a tie, and Codex won the review because Claude Code chose to stress-test a hanging regex.

Codex vs Claude Code at a glance

Prices are from the official pages on 2026-10-06: claude.com/pricing, ChatGPT plans for Codex and the OpenAI API pricing.

How we tested (and how you can rerun it)

Everything needed to check or rerun the benchmark is in the raw data archive (180 KB): the four prompts, the hidden acceptance tests, the run and check scripts, the full JSON transcript of each of the 16 runs and the computed metrics.

The repository and the four tasks

The repository is python-humanize/humanize (MIT): 1,725 lines of Python in src/, about 860 tests that run in about 7 seconds, pinned at commit 785e5dcc0d0308ad0dff3f6cc0faa7085ad0375b. Every run got a fresh git worktree of its starting commit and its own virtualenv, so no run could see another one's work.

The review commit adds a parse_size() function, the inverse of naturalsize(), plus 12 passing tests that avoid every bug. To recreate it, append this to src/humanize/filesize.py (with import re at the top) and export it from humanize/__init__.py:

_SIZE_PATTERN = re.compile(r"^\s*(-?(?:\d+)+(?:\.\d+)?)\s*([A-Za-z]*)\s*$")


def parse_size(value: str) -> int:
    """Parse a size written by `naturalsize` back into a number of bytes."""
    match = _SIZE_PATTERN.match(value)
    if match is None:
        msg = f"invalid size: {value!r}"
        raise ValueError(msg)
    number, unit = float(match.group(1)), match.group(2)

    if unit in ("", "B", "Byte", "Bytes"):
        return int(number)
    if unit in suffixes["decimal"]:
        return int(number * 1000 ** (suffixes["decimal"].index(unit) + 1))
    if unit in suffixes["binary"]:
        return int(number * 1024 ** suffixes["binary"].index(unit))
    msg = f"unknown unit: {unit!r}"
    raise ValueError(msg)

The three planted bugs:

1. Logic: the binary branch forgets the + 1, so parse_size("1.0 KiB") returns 1.
2. Edge case: int() truncates the float product, so parse_size("8.2 MB") returns 8199999.
3. Security: the nested quantifier (?:\d+)+ backtracks exponentially. parse_size("1" * 24 + "!") takes 2.6 seconds on our machine and each extra digit doubles it: a denial of service on untrusted input.

The exact commands and prompts

Setup of one run:

git clone https://github.com/python-humanize/humanize && cd humanize
git worktree add --detach ../run-1 785e5dcc0d0308ad0dff3f6cc0faa7085ad0375b
cd ../run-1
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e '.[tests]' pytest

The two commands, same prompt text for both, default model and effort, permissionless:

# Claude Code 2.1.291
env -i HOME="$HOME" PATH="$PATH" IS_SANDBOX=1 \
  claude -p "$PROMPT" --output-format stream-json --verbose \
    --dangerously-skip-permissions \
    --setting-sources project,local --strict-mcp-config --no-session-persistence < /dev/null

# Codex CLI 0.160.0
env -i HOME="$HOME" PATH="$PATH" \
  codex exec --json --dangerously-bypass-approvals-and-sandbox \
    --ignore-user-config --skip-git-repo-check -C "$WORKTREE" "$PROMPT" < /dev/null

A few flags deserve a word:

• env -i, --setting-sources project,local, --strict-mcp-config and --ignore-user-config keep our personal settings, plugins, MCP servers and model overrides out of the runs, so both CLIs run with their defaults.
• stream-json gives the full transcript; its last event is the same result object as --output-format json (usage, total_cost_usd, duration_ms, turns).
• Our VM runs as root, and Claude Code refuses --dangerously-skip-permissions as root unless IS_SANDBOX=1 says the machine is disposable. That refusal is a sensible default.
• These are the same headless invocations a workflow runner types for an unattended step (claude -p, codex exec, permission prompts off), so the numbers describe CI and scheduled use, not interactive chat.

Every prompt ended with the same line: "The project virtualenv is ready in .venv: run the tests with .venv/bin/python -m pytest -q. Work only inside this repository. Do not commit." The four prompts, in full:

[Bug fix]
Fix this bug, reported against this repository (humanize):

`intword()` decides whether rounding pushed a value up into the next magnitude with:

    if not largest_ordinal and rounded_value * power == powers[ordinal + 1]:

For values above ~10**22, `rounded_value * power` is evaluated in floating point and no longer
equals the exact `powers[ordinal + 1]`, so the carry is skipped and the number is rendered against
the lower magnitude:

    >>> import humanize
    >>> humanize.intword(10**24 - 1)
    '1000.0 sextillion'      # expected '1.0 septillion'
    >>> humanize.intword(10**27 - 1)
    '1000.0 septillion'      # expected '1.0 octillion'

The same happens at 10**30 and 10**33.

Add a regression test.

[Feature]
Add two keyword-only parameters to `humanize.natural_list` (src/humanize/lists.py):

- `conjunction: str = "and"`: the word placed before the last item.
  `natural_list(["a", "b", "c"], conjunction="or")` returns `"a, b or c"`.
- `max_items: int | None = None`: when set and the iterable has more than `max_items` items, keep
  the first `max_items` items and replace the rest with "N more", joined with the conjunction.
  `natural_list(["a", "b", "c", "d"], max_items=2)` returns `"a, b and 2 more"`;
  `natural_list(["a", "b", "c", "d"], max_items=1)` returns `"a and 3 more"`. When the iterable has
  `max_items` items or fewer, the output is the same as without the parameter.
- `max_items` lower than 1 raises `ValueError`.
- The default behavior must not change. Update the docstring (with examples) and add tests.

[Refactor]
Refactor, with no behavior change: move `scientific()` and `metric()` out of
src/humanize/number.py into a new module src/humanize/notation.py, together with the helpers and
constants only they use.

- `humanize.scientific`, `humanize.metric`, `humanize.number.scientific` and
  `humanize.number.metric` must keep working.
- Helpers still needed by number.py must not be duplicated.
- The new module must appear in the API documentation (docs/ and mkdocs.yml) like the other modules.
- Do not modify the existing tests; the whole test suite must pass.

[Code review]
You are reviewing a pull request. The change under review is the last commit of this repository
(see `git show HEAD`).

Review it for correctness, edge cases, security and performance. Report each problem as a numbered
list: file and line, severity (high, medium or low), what is wrong, and a concrete input that
demonstrates it. Only list real problems. Do not modify any file.

The review prompt does not end with the test line, since the reviewer must not change anything.

What we measured, and what "cost" means here

• Pass or fail on the checks above, run by a script after each run, never by the agent itself.
• Wall time from launch to exit. The 16 runs went in two waves of 8 parallel runs on a 4 vCPU VM, which can add a few seconds to test-heavy runs on both sides.
• Tokens as reported by each CLI: Claude Code's result event (uncached input, cache writes, cache reads, output) and Codex's turn.completed events (input, cached input, output; reasoning tokens are reported as part of output, 16 to 225 per run).
• Lines changed, from git diff --shortstat before the hidden tests were copied in.
• API-equivalent cost, computed from the official list prices on 2026-10-06. Claude Opus 5.5: $4 per million input tokens, $0.20 cache reads, $20 output, cache writes at $5 (5-minute) or $8 (1-hour, twice the input price). Claude Code wrote its cache with the 1-hour lifetime in every run, and our figure matched the CLI's own total_cost_usd to the cent. GPT-6.1 Sol: $2 per million input tokens, $0.10 cached input, $10 output (cache writes are billed at $2.50, but Codex reported none).

"API-equivalent" matters. With an API key you pay exactly these amounts. On a Claude Pro or ChatGPT Plus subscription you pay a flat fee, and the same tokens are consumed from your usage limits instead.

Results

Bug fix from a real issue

All four runs changed the same line and added a regression test. Claude Code compared against powers[ordinal + 1] / power both times; Codex did the same once and used the integer division // once, which is exactly the upstream fix. Codex explored more before editing (listing files, looking for an AGENTS.md, running the targeted test, then the suite, then git diff --check), which shows in its tool calls and input tokens. It still cost half as much.

Feature from a spec

The four implementations are nearly interchangeable: same keyword-only signature, same validation, same slicing for "N more". A precise spec leaves little room for a model to differ. Claude Code finished about 18 seconds sooner (median) by batching its reads and test runs into fewer tool calls.

Refactor without behavior change

This task hides a trap: the helper _format_not_finite is used by both modules, so a naive move creates a circular import. The runs found three different ways out:

• Claude Code (both runs): notation.py imports the helper from number.py, and number.py re-exports metric and scientific through a module-level __getattr__. It works, but the re-exported names are invisible to autocompletion and untyped for static checkers.
• Codex, run 1: number.py imports from notation.py at the top, and notation.py imports the helper inside the function bodies.
• Codex, run 2: moves the helper into a new private module, _number.py, imported by both. The cleanest structure of the four, at the price of one file nobody asked for.

All four pass every check. Claude Code paid 2.9x more here, mostly for twice the output and 20k tokens of cache writes per run.

Code review with planted bugs

We expected this to be where one agent pulled ahead. It wasn't: every review found the ReDoS, the missing + 1 and the float truncation, each with a reproducing input, and none reported a bug that wasn't one.

The difference is in style. Codex returned exactly three findings, each probed with a short timeout. Claude Code verified every claim by running code, counted the damage (590 wrong results out of 99,999 values between 0.01 kB and 999.99 kB), and added valid remarks we had not planted: GNU-style suffixes like 2.9K that naturalsize() produces but parse_size() rejects, an OverflowError on very long inputs where the docstring promises ValueError, and localized output that cannot be parsed back. In its second run it also started a background ReDoS probe with 30,000 digits, waited, then killed it, which is why that run took 160 seconds. It also rated the binary bug "high" both times, where Codex said "high" once and "medium" once.

Our daily pull request review template runs this kind of review every morning; the numbers above say a Codex step does the planted-bug part at a third of the price.

Totals: tokens, time and cost per passed task

Where the money goes is the interesting part:

• Claude Code: 56% of its cost is cache writes ($0.80), 32% is output ($0.46) and 12% cache reads. It sends almost nothing uncached (90 tokens over eight runs) but writes 8k to 22k tokens per run into a 1-hour cache at $8 per million. For one-shot headless runs, that cache rarely pays for itself.
• Codex: 65% of its cost is uncached input ($0.38), 19% output and 17% cached input. It reads more and calls more tools, but every one of those tokens is cheaper, and it writes half the output.

Pricing and usage limits in practice

Official plans on 2026-10-06:

Sources: claude.com/pricing, Claude Code with Pro or Max, Codex pricing.

Which is cheaper? Per task, Codex, by 1.8x to 3.3x on our four tasks. Projected at API prices, 100 tasks like these cost about $18 with Claude Code and $7.40 with Codex. An unattended job that runs 20 such tasks every weekday (about 440 a month) would cost about $80 against $33. That is the budget question for anything scheduled, like a weekly dependency update or a nightly triage.

Which hits limits first on a $20 plan? We can't say from our data: none of the 16 runs hit a limit, and neither vendor converts its limits into tokens. What we can say is that Claude Code consumed about twice the API-equivalent value per task. If both vendors size their $20 plans to a similar dollar value of compute, which is not published, Claude Code would reach its limit sooner.

Which one should you use?

When Claude Code is the better pick

• Reviews where you want a second opinion on the design, not only the bugs. It found 2 to 3 extra valid issues per review and backed each with a run.
• Short, well-specified tasks with a human waiting. It was 7 to 18 seconds faster (median) on the bug fix and the feature, by batching tool calls.
• Teams already on Claude Max, where the per-token price does not apply and limits are the only cost.

When Codex is the better pick

• Anything unattended and billed per token: CI, scheduled jobs, batch reviews. Same pass rate, 2.4x cheaper overall.
• Focused reviews where you want the bugs and nothing else: three findings, no false alarm, under $0.05.
• Refactors, cautiously: its two runs produced the most conventional module structure, but n=2 is not a trend.

Is there a "best AI coding agent" right now?

Not on this evidence. On four everyday tasks, the two leading CLIs produced code of the same quality, sometimes line for line. What separated them was price and verbosity, and both move with each model release. The honest answer is to measure on your own repository, with your own tasks, every time a model changes.

Using Codex and Claude Code together

"Most people use both" is in every comparison, usually followed by a handoff log that you paste from one terminal into the other. The cleaner split follows the results: route each task type to the agent that measured best for it, and let a workflow do the handoff.

In SideHub, a workflow is a list of steps run by an agent on your own machine, and each step can set its own CLI with provider (claude, codex, gemini or copilot). Here is a sketch that implements with Claude Code and reviews with Codex:

format: sidehub.workflow/v1
name: Implement, then cross-review
defaultProvider: claude
parameters:
  - { name: issue, label: Issue number, type: string, required: true }
steps:
  - name: Implement the issue
    prompt: |
      Read GitHub issue #{{inputs.issue}} with `gh issue view`, implement it on a new branch
      and run the test suite. Do not push.
  - name: Review the diff
    provider: codex
    inputsFrom: ["1"]
    prompt: |
      Review the uncommitted diff of this repository for correctness, edge cases, security and
      performance. List each problem with file, line, severity and a reproducing input.

Each step runs as its own run, with its CLI, duration, exit code and the token usage read from the CLI's transcript, and you can follow it from anywhere, phone included. That means you can rerun this comparison on your own codebase and see the cost of each step, instead of trusting a blog post, this one included. The template gallery has ready-made workflows to start from, such as the nightly issue triage, a typical scheduled job where the cheaper agent per run adds up.

Limitations of this benchmark

• Small sample. Two runs per task per CLI. The review run that took 160 seconds shows how much a single run can move a median.
• One repository, one language, small tasks. A tidy Python library with fast tests, and tasks that take 30 seconds to 3 minutes. Long, ambiguous tasks in a large codebase may separate the agents more.
• Possible contamination. The bug fix comes from a public pull request committed on 2026-09-16. One Codex run reproduced the upstream fix character for character, which may be competence or memory.
• Our planted bugs. We wrote the review diff, so its bugs may be easier to spot than real ones.
• Default settings only. Default models and effort, no project instruction files. Higher effort settings, other models or a tuned CLAUDE.md / AGENTS.md would change the numbers.
• Parallel runs. Eight runs at a time on 4 vCPUs can inflate wall times by a few seconds on both sides.
• Dated prices. API prices and plan limits are those of 2026-10-06, and Claude Code's choice of a 1-hour cache drives most of its cost. A change on either side would move the ratio.

The verdict

On four everyday tasks, Codex (gpt-6.1-sol) and Claude Code (claude-opus-5-5) were equally good: 16 passes out of 16, every planted bug found, no false alarm. Codex was 1.8x to 3.3x cheaper per task at API prices; Claude Code was a bit faster on small coding tasks and wrote richer reviews. Use Codex where cost per run adds up, Claude Code where you want depth or a quick answer, and rerun these four tasks on your own repository: the commands and prompts are all above.