Question 1

What is the Artificial Analysis Coding Agent Index?

Accepted Answer

A composite score for coding agents (Claude Code, Codex, Devin Fusion CLI and others) running real software-engineering tasks. It is the equal-weight average of pass@1 on three suites: DeepSWE v1.1 (113 tasks), Terminal-Bench 4.0 (66 tasks) and SWE-Atlas-QnA (124 tasks), with three attempts per task.

Question 2

Why does cost per task vary so much between agents?

Accepted Answer

Cost per task is measured API spend — actual token consumption multiplied by the host model's token prices. Agents that use long-context reasoning or take more steps (such as Claude Code on Fable 5.1) can cost 50x more per task than efficient combinations like Codex on DeepSeek V4, even when their index scores are close.

Question 3

Does the same model score the same in every agent?

Accepted Answer

No. The harness matters: the same host model (for example Fable 5.1 or GPT-6 Astra) achieves different index scores, costs and runtimes in Claude Code, Codex or Devin Fusion CLI. Each row on this page is one agent × model configuration, not a model-only score.

Question 4

How often is this leaderboard updated?

Accepted Answer

Artificial Analysis re-runs evaluations as agents and models ship new versions; aiplans.dev syncs the public snapshot nightly and shows the snapshot date at the top of the table.

#	Agent / model	Index	DeepSWE	Terminal-Bench	SWE-Atlas	Cost / task	Time / task
1	Claude Code claude-fable-5.1	62.2%Best	64.3%	57.6%	64.8%	$12.39	34.8m
2	Devin Fusion CLI claude-fable-5.1	61.7%	63.1%	56.1%	65.9%	$7.90	35.8m
3	Codex gpt-6-astra	61.6%	67.6%	55.6%	61.8%	$7.47	29.4m
4	Claude Code claude-opus-5	59.7%	62.5%	54.5%	62.1%	$10.79	41.9m
5	Devin Fusion CLI gpt-6-astra	58.9%	67.3%	50.0%	59.4%	$4.54	24.7m
6	Codex gpt-5.6-sol	54.6%	72.3%	37.4%	54.0%	$6.58	20.6m
7	Muse Code Muse Spark 1.3 (max)	54.3%	71.7%	31.8%	59.4%	$3.98	18.4m
8	Opencode glm-5.3	53.6%	61.4%	39.9%	59.4%	$4.24	48.1m
9	Kimi Code CLI kimi-k3	51.9%	68.4%	21.2%	66.1%	$5.05	1h
10	Grok Build grok-4.6	47.0%	64.9%	17.7%	58.3%	$3.57	19.5m
11	Claude Code qwen3.8-max	43.3%	51.0%	16.7%	62.1%	$3.49	1.1h
12	Codex deepseek-v4-pro	43.1%	57.2%	10.1%	61.8%	$0.24	40.3m
13	Antigravity SDK v0.1.12 gemini-3.8-flash	41.9%	65.8%	14.6%	45.2%	$2.47	11.7m
14	Codex deepseek-v4-flash	38.7%	54.3%	10.6%	51.3%	$0.08	18.3m

Coding Agent Leaderboard

How the index is built