AI Coding Agent Index & Leaderboard
Real-time accuracy benchmarks, SWE-bench Verified scores, Terminal-Bench 4.0 metrics, speed, and per-task costs for top autonomous coding systems.
Claude Code - Claude 3.7 Sonnet (Hybrid/Thinking)
Coding Agent Accuracy & Speed Index
Click any column header to re-sort rankings in real-time.
# Rank | Agent / Architecture | Agent Index 🎯 | SWE-bench | Terminal-Bench | Cost / Task | Time / Task | Tokens / Task | Action |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Code - Claude 3.7 Sonnet (Hybrid/Thinking)Frontier Leader AnthropicClaude 3.7 Sonnet (Thinking) | 85% | 72.5% | 84% | $3.80 | 11.4m | 4.2M | Explore |
| 2 | Cursor CLI & Agent - Claude 3.7 SonnetBest DX & Speed CursorClaude 3.7 Sonnet + Composer | 82% | 70.1% | 80% | $3.10 | 8.2m | 3.8M | Explore |
| 3 | Codex CLI - OpenAI o3-mini (High)Deep Reasoning OpenAIo3-mini (high reasoning) | 81% | 71.2% | 78% | $3.40 | 7.5m | 4.5M | Explore |
| #4 | Antigravity CLI - Gemini 2.5 Pro / Flash2M Context / Value GoogleGemini 2.5 Pro & Flash (DeepMind) | 78% | 68.5% | 76% | $0.85 | 5.9m | 3.1M | Explore |
| #5 | Windsurf Cascade - Claude 3.7 SonnetFlow Architecture AnthropicCascade 3.7 + Flow | 75% | 66% | 74% | $2.60 | 9.2m | 3.4M | Source |
| #6 | Aider - DeepSeek R1 & Claude 3.7Lowest Cost / Git Native DeepSeekDeepSeek R1 + Sonnet Hybrid | 73% | 64.2% | 72% | $0.38 | 13.5m | 5.8M | Explore |
| #7 | Cline / Roo Code - Claude 3.7 SonnetOpen Source IDE Agent AnthropicClaude 3.7 Sonnet (VS Code) | 71% | 63.1% | 70% | $2.90 | 10.8m | 3.6M | Source |
| #8 | Devin 2.5 Autonomous LoopFully Autonomous OpenAIDevin Multimodal Engine | 68% | 60.5% | 67% | $11.50 | 22.0m | 12.5M | Source |
| #9 | Cursor CLI - Composer 2.5 FastSpeed Champion CursorComposer 2.5 Fast | 67% | 59.2% | 65% | $0.50 | 4.9m | 1.8M | Explore |
| #10 | OpenCode - Qwen 2.5 Coder 32BOpen Weights Best AlibabaQwen 2.5 Coder 32B Instruct | 65% | 58% | 64% | $0.28 | 9.8m | 2.8M | Source |
| #11 | Claude Code - GLM-5.2 / Kimi K2.6Bilingual / Code Zhipu AIGLM-5.2 & Kimi Hybrid | 62% | 55% | 61% | $1.20 | 18.5m | 4.5M | Explore |
| #12 | Gemini CLI - Gemini 2.0 FlashUltra Cheap GoogleGemini 2.0 Flash Agent | 60% | 53.4% | 59% | $0.20 | 5.2m | 1.6M | Explore |
| #13 | DeepSeek V3 Agent ProCost Leader DeepSeekDeepSeek V3 (671B MoE) | 58% | 51.8% | 57% | $0.18 | 12.0m | 3.8M | Explore |
Human blind preference rating aggregating >100,000 randomized A/B matchups on LMSYS. Measures general conversational IQ and instruction following.
Autonomous resolution of real GitHub issues across 500+ production repositories, requiring bug localization, patch synthesis, and passing unit tests.
Google-Proof Graduate-Level QA written by PhD experts in biology, physics, and chemistry. Tests deep multi-disciplinary reasoning.
American Invitational Mathematics Examination problems assessing test-time compute, mathematical proof search, and combinatorial problem solving.
Frontier Trends in Agentic Software Engineering
From Autocomplete to Autonomous CLI Loops: The paradigm of software development has shifted from inline tab completions to terminal-native agents like Claude Code and Cursor CLI that execute tasks, run test suites, and resolve merge conflicts autonomously.
Test-Time Compute & Verification: Leading coding agents trade inference compute for accuracy. By generating candidate patches, executing local unit tests, and verifying diffs, pass rates exceed 70%+ on SWE-bench Verified.
Hybrid Reasoning Architectures: Next-gen agents combine low-latency fast models for file navigation with deep reasoning models (e.g., Claude 3.7 Sonnet Thinking, o3-mini) for complex architectural refactoring.
Theoretical Mechanics of the Coding Agent Index
The Coding Agent Index is an empirical evaluation of autonomous software engineering systems under complex execution environments. In classical computer science, code generation was treated as single-turn token prediction. Modern agentic systems treat software engineering as an iterative graph search and verification problem. The index aggregates performance across three core dimensions: DeepSWE v1.1, Terminal-Bench 4.0, and SWE-bench Verified. DeepSWE presents agents with real-world issues pulled from production repositories, requiring them to localize bugs, edit files, and verify changes. Terminal-Bench measures capability in interactive shell environments—handling long-lived terminal sessions, parsing stdout/stderr, and recovering from failed builds. Performance is measured using pass@1 accuracy over multiple stochastic runs to prove baseline reliability in non-deterministic environments.
- Repository-Scale Bug Localization: Digesting structural schemas, tracing imports, and pinpointing exact bug locations without context exhaustion.
- Interactive Terminal Feedback Loops: Tracing stdout/stderr from compilers, linters, and unit tests, and dynamically synthesizing fixes.
- Deterministic Output Verification: Running unit tests locally within sandboxed containers to verify functional correctness before generating git commits.
Harness Architecture & Sandboxed Execution
In autonomous agent architectures, the 'harness' represents the critical execution boundary between the LLM planner and the operating system. The harness intercepts model tool calls, translates them into safe system calls (bash execution, AST-based file edits, workspace search), and streams the environment response back into context. Modern harnesses utilize ephemeral Docker containers, strict resource throttling, and stateful memory tracking (file diffs, active processes, git states) allowing agents like Claude Code, Cursor, and Codex to backtrack when encountering regressions.
- Stateful Memory Tracking: Maintaining deltas of file changes, active terminal processes, and environment variables across asynchronous planning turns.
- Sandboxed Execution Isolation: Running arbitrary user-space binaries and tests in secure containers to prevent runtime side-effects.
- Self-Correction & Exception Handling: Parsing stack traces, compile errors, and runtime crashes to guide the next iteration loop.
Frequently Asked Questions
What is the difference between SWE-bench Verified and Terminal-Bench 4.0?
SWE-bench Verified evaluates an agent's ability to resolve real GitHub issues and pass unit test suites on complex Python repositories. Terminal-Bench 4.0 tests agentic interaction within real shell environments—such as configuring servers, compiling multi-language binaries, navigating git trees, and executing DevOps workflows.
Why do autonomous coding agents differ in cost per task?
Cost per task is determined by the number of reasoning steps, model token pricing, and context growth during multi-turn exploration. Agents optimized for speed and context caching (like Cursor and Antigravity) reduce costs to $0.80-$3.10, whereas deep multi-candidate verification runs can reach $4-$12 per complex bug fix.