Question 1

What is the Artificial Analysis Coding Agent Index?

Accepted Answer

A composite score for coding agents (Claude Code, Codex, Devin Fusion CLI and others) running real software-engineering tasks. It is the equal-weight average of pass@1 on three suites: DeepSWE v1.1 (113 tasks), Terminal-Bench 4.0 (66 tasks) and SWE-Atlas-QnA (124 tasks), with three attempts per task.

Question 2

Why does cost per task vary so much between agents?

Accepted Answer

Cost per task is measured API spend — actual token consumption multiplied by the host model's token prices. Agents that use long-context reasoning or take more steps (such as Claude Code on Fable 5.1) can cost 50x more per task than efficient combinations like Codex on DeepSeek V4, even when their index scores are close.

Question 3

Does the same model score the same in every agent?

Accepted Answer

No. The harness matters: the same host model (for example Fable 5.1 or GPT-6 Astra) achieves different index scores, costs and runtimes in Claude Code, Codex or Devin Fusion CLI. Each row on this page is one agent × model configuration, not a model-only score.

Question 4

How often is this leaderboard updated?

Accepted Answer

Artificial Analysis re-runs evaluations as agents and models ship new versions; aiplans.dev syncs the public snapshot nightly and shows the snapshot date at the top of the table.

#	Agent / model	Index	DeepSWE	Terminal-Bench	SWE-Atlas	Cost / task	Time / task
1	Claude Code claude-sonnet-5.5	68.4%Best	72.0%	66.2%	66.9%	$14.19	1.5h
2	Devin Fusion CLI claude-opus-5.5	68.2%	72.6%	62.6%	69.4%	$9.02	55m
3	Muse Code Muse Wonder Full (max)	68.0%	77.0%	59.1%	68.0%	—	28.8m
4	Claude Code claude-opus-5.5	66.0%	68.4%	63.1%	66.4%	$13.04	1.1h
5	Antigravity CLI Gemini 4 Argon	63.8%	78.8%	56.1%	56.5%	$5.84	34.5m
6	Muse Code Muse Wonder	63.7%	75.8%	48.0%	67.2%	$8.04	22.4m
7	Claude Code claude-fable-5.1	62.2%	64.3%	57.6%	64.8%	$12.39	34.8m
8	Claude Code claude-opus-5.5	61.7%	—	63.1%	65.9%	$11.42	1h
9	Codex gpt-6-astra	61.6%	67.6%	55.6%	61.8%	$7.50	29.4m
10	Devin Fusion CLI claude-opus-5.5	61.5%	—	62.6%	68.3%	$8.00	51.1m
11	Codex gpt-6.1-sol	60.1%	69.6%	53.0%	57.8%	$1.55	24.4m
12	GitHub Copilot Copilot Fusion (max)	59.9%	—	71.2%	57.0%	$5.59	36.1m
13	Claude Code claude-sonnet-5.5	59.3%	—	66.2%	64.2%	$13.30	1.3h
14	Muse Code Muse Wonder Full (Max)	58.9%	—	59.1%	68.8%	—	34.9m
15	Claude Code claude-fable-5.1	57.6%	—	57.6%	66.4%	$11.53	35.3m
16	Grok Build grok-4.7	56.3%	72.6%	33.3%	62.9%	$8.82	39.2m
17	Codex gpt-6-astra	54.7%	—	55.6%	59.1%	$7.04	24.8m
18	Codex gpt-6.1-sol	54.1%	—	53.0%	59.9%	$1.50	23.7m
19	Opencode glm-5.3	53.6%	61.4%	39.9%	59.4%	$4.37	48.1m
20	Antigravity CLI Gemini 4 Argon	53.2%	—	56.1%	55.1%	$11.73	34.7m
21	Muse Code Muse Wonder	52.0%	—	48.0%	61.6%	$8.05	24.5m
22	Kimi Code CLI kimi-k3	51.9%	68.4%	21.2%	66.1%	$5.05	1h
23	Opencode glm-5.3	48.9%	—	39.9%	58.6%	$5.23	56.2m
24	Mistral Vibe Mistral Large 4	48.3%	—	41.4%	58.9%	$1.26	48.4m
25	Codex gpt-6.1-terra	48.2%	—	46.0%	53.8%	$0.69	16.4m
26	Grok Build grok-4.7	46.6%	—	33.3%	62.9%	$9.48	35.8m
27	DeepSeek Harness DeepSeek V4.1 Flash (1017)	46.3%	—	30.8%	61.6%	$0.30	21.4m
28	Kimi Code CLI kimi-k3	41.4%	—	21.2%	59.9%	$5.17	1h
29	Muse Code muse-spark-open	41.2%	60.2%	5.6%	57.8%	$3.33	25.2m
30	Claude Code Claude Haiku 5.5	36.8%	46.0%	30.8%	33.6%	$2.80	1.5h
31	Mirror beam-1.0-501b-a23b	27.7%	39.8%	3.5%	39.8%	—	42.4m
32	Mirror beam-1.0-501b-a23b	20.5%	—	3.5%	37.6%	$0.32	38.7m

Coding Agent Leaderboard

How the index is built