Numbers with a source and a date.
Quality comes from the official SWE-bench leaderboard JSON. Price and context come from the public OpenRouter models API. Speed and Artificial Analysis scores are unknown in this snapshot, for the reasons below. Every figure has a source and a fetch timestamp.
data/bench.json. Prior snapshot kept at data/bench-2026-10-23T0631Z.json. SWE-bench: official JSON, fetched 2026-11-22T12:28:32-05:00. OpenRouter: models API, fetched 2026-11-22T12:28:32-05:00. Official instance counts are from the SWE-bench homepage cards the same day. For Artificial Analysis, use their site.Quality — SWE-bench
Verified · 500 official instances (count from swebench.com cards) · 180 rows in the official leaderboard JSON fetched 2026-11-22T12:28:32-05:00. Source: SWE-bench official leaderboard JSON. “Resolved” is the official resolved-% score. Cost is whatever the official board included (USD in their file); unknown if they left it null. “Checked” is the official field. Each row keeps its own date from SWE-bench.
| # | Agent | Model | Resolved % | Date | Cost | Open model | Checked | Attempts | Site |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Sonar Foundation Agent Sonar | Claude 4.5 Opus Anthropic | 79.2 | 2025-12-05 | unknown | no | no | Attempts - 1 | source |
| 2 | live-SWE-agent UIUC | Claude 4.5 Opus Anthropic | 79.2 | 2025-12-15 | unknown | no | no | Attempts - 1 | source |
| 3 | TRAE ByteDance | Doubao-Seed-Code ByteDance | 78.8 | 2025-09-28 | unknown | no | no | Attempts - 2+ | source 1 · source 2 |
| 4 | live-SWE-agent UIUC | Gemini 3 Pro Preview Google DeepMind | 77.4 | 2025-11-20 | unknown | no | no | Attempts - 1 | source |
| 5 | EPAM AI/Run Developer Agent EPAM Systems, Inc. | Claude 4 Sonnet Anthropic | 76.8 | 2025-08-04 | unknown | no | no | Attempts - 2+ | source |
| 6 | Atlassian Rovo Dev Atlassian | Multiple Anthropic | 76.8 | 2025-09-02 | unknown | no | no | Attempts - 2 | source |
| 7 | mini-SWE-agent SWE-agent | Claude 4.5 Opus Anthropic | 76.8 | 2026-02-17 | 376.95 | no | no | Attempts - 1 | source |
| 8 | ACoder ACoder | Multiple Anthropic | 76.4 | 2025-08-19 | unknown | no | no | Attempts - 2+ | source |
| 9 | mini-SWE-agent SWE-agent | Gemini 3 Flash Google DeepMind | 75.8 | 2026-02-17 | 177.98 | no | no | Attempts - 1 | source |
| 10 | mini-SWE-agent SWE-agent | MiniMax M2.5 Minimax | 75.8 | 2026-02-17 | 36.64 | yes | no | Attempts - 1 | source |
| 11 | Warp Warp | Multiple OpenAI | 75.6 | 2025-09-01 | unknown | no | no | Attempts - 2+ | source |
| 12 | mini-SWE-agent SWE-agent | Claude 4.6 Opus Anthropic | 75.6 | 2026-02-17 | 275.76 | no | no | Attempts - 1 | source |
| 13 | TRAE TRAE | Multiple Anthropic | 75.2 | 2025-06-12 | unknown | no | no | Attempts - 2+ | source |
| 14 | Harness AI Harness | Claude Sonnet 4 Anthropic | 74.8 | 2025-07-31 | unknown | no | no | Attempts - 1 | source |
| 15 | Sonar Foundation Agent Sonar | Claude 4.5 Sonnet Anthropic | 74.8 | 2025-11-03 | unknown | no | no | Attempts - 1 | source |
| 16 | Lingxi-v1.5 Lingxi | Claude 4 Sonnet Anthropic | 74.6 | 2025-07-20 | unknown | no | no | Attempts - 1 | source |
| 17 | JoyCode JoyCode | Multiple Anthropic | 74.6 | 2025-09-15 | unknown | no | no | Attempts - 1 | source |
| 18 | Refact.ai Agent Refact.ai | Multiple Anthropic | 74.4 | 2025-06-03 | unknown | no | no | Attempts - 1 | source |
| 19 | Prometheus-v1.2.1 EuniAI | GPT-5 OpenAI | 74.4 | 2025-10-15 | unknown | no | no | Attempts - 2+ | source |
| 20 | mini-SWE-agent SWE-agent | Claude 4.5 Opus Anthropic | 74.4 | 2025-11-24 | 360.62 | no | yes | Attempts - 1 | source |
| 21 | mini-SWE-agent SWE-agent | Gemini 3 Pro Preview Google DeepMind | 74.2 | 2025-11-18 | 229.98 | no | yes | Attempts - 1 | source |
| 22 | Salesforce AI Research SAGE Salesforce AI Research | Multiple Anthropic | 73.8 | 2025-11-03 | unknown | no | no | Attempts - 2+ | source |
| 23 | Tools | Claude 4 Opus Anthropic | 73.2 | 2025-05-22 | unknown | no | no | Attempts - 1 | source |
| 24 | Salesforce AI Research SAGE Salesforce AI Research | Multiple Anthropic | 73.0 | 2025-10-21 | unknown | no | no | Attempts - 2+ | source |
| 25 | mini-SWE-agent SWE-agent | GLM 5 Z-AI | 72.8 | 2026-02-17 | 267.19 | yes | no | Attempts - 1 | source |
| 26 | mini-SWE-agent SWE-agent | GPT 5.2 OpenAI | 72.8 | 2026-02-17 | 236.78 | no | no | Attempts - 1 | source |
| 27 | mini-SWE-agent SWE-agent | GPT 5.2 Codex OpenAI | 72.8 | 2026-02-19 | 224.71 | no | no | Attempts - 1 | source |
| 28 | Tools | Claude 4 Sonnet Anthropic | 72.4 | 2025-05-22 | unknown | no | no | Attempts - 1 | source |
| 29 | OpenHands OpenHands | GPT-5 OpenAI | 71.8 | 2025-08-07 | unknown | no | yes | Attempts - 1 | source |
| 30 | mini-SWE-agent SWE-agent | GPT 5.2 OpenAI | 71.8 | 2025-12-11 | 260.13 | no | no | Attempts - 1 | source |
| 31 | mini-SWE-agent SWE-agent | Claude 4.5 Sonnet Anthropic | 71.4 | 2026-02-17 | 328.95 | no | no | Attempts - 1 | source |
| 32 | Bloop Bloop AI | Claude 4 Sonnet Anthropic | 71.2 | 2025-07-10 | unknown | no | no | Attempts - 1 | source |
| 33 | Qodo Command Qodo | Multiple Anthropic | 71.2 | 2025-07-15 | unknown | no | no | Attempts - 1 | source |
| 34 | Prometheus-v1.2 EuniAI | GPT-5 OpenAI | 71.2 | 2025-09-29 | unknown | no | no | Attempts - 1 | source |
| 35 | Lingxi v1.5 Huawei | Kimi K2 Moonshot AI | 71.2 | 2025-10-14 | unknown | yes | no | Attempts - 1 | source |
| 36 | Warp Warp | Multiple Anthropic | 71.0 | 2025-06-23 | unknown | no | no | Attempts - 1 | source |
| 37 | Moatless Tools Moatless AI | Claude 4 Sonnet Anthropic | 70.8 | 2025-06-11 | unknown | no | yes | Attempts - 1 | source |
| 38 | mini-SWE-agent SWE-agent | Kimi K2.5 Moonshot AI | 70.8 | 2026-02-17 | 73.28 | yes | no | Attempts - 1 | source |
| 39 | TRAE | Undisclosed | 70.6 | 2025-05-19 | unknown | no | no | Attempts - 2+ | source |
| 40 | mini-SWE-agent SWE-agent | Claude 4.5 Sonnet Anthropic | 70.6 | 2025-09-29 | 279.17 | no | yes | Attempts - 1 | source |
| 41 | Refact.ai Agent Refact.ai | Multiple Anthropic | 70.4 | 2025-05-15 | unknown | no | no | Attempts - 1 | source |
| 42 | OpenHands OpenHands | Claude 4 Sonnet Anthropic | 70.4 | 2025-05-24 | unknown | no | yes | Attempts - 1 | source |
| 43 | Augment Agent v1 | Undisclosed | 70.4 | 2025-06-10 | unknown | no | no | Attempts - 1 | source |
| 44 | devlo devlo | Multiple Anthropic | 70.2 | 2025-05-19 | unknown | no | no | Attempts - 1 | source |
| 45 | Zencoder | Multiple Anthropic | 70.0 | 2025-04-30 | unknown | no | no | Attempts - 2+ | source |
| 46 | mini-SWE-agent SWE-agent | DeepSeek V3.2 DeepSeek | 70.0 | 2026-02-17 | 223.92 | yes | no | Attempts - 1 | source |
| 47 | OpenHands Qwen | Qwen3-Coder-480B-A35B-Instruct Qwen | 69.6 | 2025-08-05 | unknown | yes | no | Attempts - 1 | source |
| 48 | mini-SWE-agent SWE-agent | Gemini 3 Pro Google DeepMind | 69.6 | 2026-02-26 | 480.01 | no | no | Attempts - 1 | source |
| 49 | mini-SWE-agent SWE-agent | GPT 5.2 OpenAI | 69.0 | 2025-12-11 | 134.83 | no | no | Attempts - 1 | source |
| 50 | Nemotron-CORTEXA NVIDIA | Multiple OpenAI | 68.2 | 2025-05-16 | unknown | no | no | Attempts - 2+ | source |
| 51 | Undisclosed Z.ai | GLM-4.6 Z.ai | 68.2 | 2025-09-30 | unknown | yes | no | Attempts - 1 | source |
| 52 | mini-SWE-agent SWE-agent | Claude 4 Opus Anthropic | 67.6 | 2025-08-02 | 565.64 | no | yes | Attempts - 1 | source |
| 53 | SWE-agent SWE-agent | Claude 4 Sonnet Anthropic | 66.6 | 2025-05-22 | unknown | no | yes | Attempts - 1 | source |
| 54 | mini-SWE-agent SWE-agent | Claude 4.5 Haiku Anthropic | 66.6 | 2026-02-17 | 165.46 | no | no | Attempts - 1 | source |
| 55 | Aime-coder v1 ByteDance DevInfra | Claude 3.7 Sonnet Anthropic | 66.4 | 2025-05-14 | unknown | no | no | Attempts - 2+ | source |
| 56 | mini-SWE-agent SWE-agent | GPT 5.1 OpenAI | 66.0 | 2025-11-20 | 153.08 | no | no | Attempts - 1 | source |
| 57 | mini-SWE-agent SWE-agent | GPT 5.1 Codex OpenAI | 66.0 | 2025-11-24 | 294.44 | no | no | Attempts - 1 | source |
| 58 | OpenHands OpenHands | Undisclosed | 65.8 | 2025-04-15 | unknown | no | yes | Attempts - 2+ | source |
| 59 | Augment Agent v0 | Undisclosed | 65.4 | 2025-03-16 | unknown | no | no | Attempts - 2+ | source |
| 60 | Amazon Q Developer Agent AWS | Undisclosed | 65.4 | 2025-04-05 | unknown | no | no | Attempts - 2+ | source |
| 61 | OpenHands OpenHands | Kimi K2 Moonshot AI | 65.4 | 2025-07-16 | unknown | yes | yes | Attempts - 1 | source |
| 62 | mini-SWE-agent SWE-agent | GPT 5 OpenAI | 65.0 | 2025-08-07 | 140.19 | no | yes | Attempts - 1 | source |
| 63 | mini-SWE-agent SWE-agent | Claude 4 Sonnet Anthropic | 64.93 | 2025-07-26 | 185.73 | no | yes | Attempts - 1 | source |
| 64 | W&B Programmer O1 crosscheck5 | o1-preview OpenAI | 64.6 | 2025-01-17 | unknown | no | no | Attempts - 2+ | source |
| 65 | PatchPilot-v1.1 | o4-mini OpenAI | 64.6 | 2025-05-03 | unknown | no | no | Attempts - 2+ | unknown |
| 66 | Undisclosed Z.ai | GLM-4.5 Z.ai | 64.2 | 2025-07-28 | unknown | yes | no | Attempts - 1 | source |
| 67 | AgentScope Alibaba | Claude 3.5 Sonnet Anthropic | 63.4 | 2025-02-06 | unknown | no | no | Attempts - 2+ | source |
| 68 | mini-SWE-agent SWE-agent | Kimi K2 Thinking Moonshot AI | 63.4 | 2025-12-10 | 219.15 | yes | yes | Attempts - 1 | source |
| 69 | Tools Anthropic | Claude 3.7 Sonnet Anthropic | 63.2 | 2025-02-24 | unknown | no | no | Attempts - 2+ | source |
| 70 | Blackbox AI Agent | Undisclosed | 62.8 | 2025-01-10 | unknown | no | no | Attempts - 2+ | source |
| 71 | EPAM AI/Run Developer Agent | Claude 3.5 Sonnet Anthropic | 62.8 | 2025-02-28 | unknown | no | no | Attempts - 2+ | source |
| 72 | SWE-agent SWE-agent | Claude 3.7 Sonnet w/ Review Heavy Anthropic | 62.4 | 2025-02-25 | unknown | no | yes | Attempts - 2+ | source |
| 73 | CodeStory Midwit Agent | swe-search Anthropic | 62.2 | 2024-12-21 | unknown | no | no | Attempts - 2+ | source |
| 74 | mini-SWE-agent SWE-agent | MiniMax M2 Minimax | 61.0 | 2025-11-24 | 214.13 | yes | yes | Attempts - 1 | source |
| 75 | OpenHands OpenHands | 4x Scaled | 60.8 | 2025-02-03 | unknown | no | yes | Attempts - 2+ | source |
| 76 | EntroPO + R2E 42-b3yond-6ug | Qwen3-Coder-30B-A3B-Instruct Qwen | 60.4 | 2025-09-01 | unknown | yes | no | Attempts - 2+ | source |
| 77 | Learn-by-interact Google | Claude 3.5 Sonnet Anthropic | 60.2 | 2025-01-10 | unknown | no | no | Attempts - 2+ | source |
| 78 | mini-SWE-agent SWE-agent | DeepSeek V3.2 Reasoner deepseek | 60.0 | 2025-12-01 | 14.04 | yes | yes | Attempts - 1 | unknown |
| 79 | mini-SWE-agent SWE-agent | GPT 5 mini OpenAI | 59.8 | 2025-08-07 | 17.74 | no | yes | Attempts - 1 | source |
| 80 | DeepSWE-Preview Agentica | TTS(Bo16) OpenAI | 58.8 | 2025-06-29 | unknown | yes | no | Attempts - 2+ | source |
| 81 | mini-SWE-agent SWE-agent | o3 OpenAI | 58.4 | 2025-07-26 | 166.83 | no | yes | Attempts - 1 | source |
| 82 | devlo | Undisclosed | 58.2 | 2024-12-13 | unknown | no | no | Attempts - 2+ | source |
| 83 | Nemotron-CORTEXA NVIDIA | Multiple Anthropic | 58.2 | 2025-04-10 | unknown | no | yes | Attempts - 2+ | source |
| 84 | Emergent E1 | Multiple Anthropic | 57.2 | 2024-12-23 | unknown | no | no | Attempts - 2+ | source |
| 85 | Gru | Undisclosed | 57.0 | 2024-12-08 | unknown | no | no | Attempts - 2+ | source |
| 86 | Artemis Agent v2 Turintech | Claude Sonnet 4 Anthropic | 57.0 | 2025-09-24 | unknown | no | no | Attempts - 1 | source |
| 87 | SWE-Rizzo | Undisclosed | 56.6 | 2025-04-05 | unknown | no | no | Attempts - 2+ | source |
| 88 | mini-SWE-agent SWE-agent | Devstral Small (2512) mistral | 56.4 | 2025-12-09 | unknown | yes | yes | Attempts - 1 | unknown |
| 89 | mini-SWE-agent SWE-agent | GPT 5 mini OpenAI | 56.2 | 2026-02-17 | 23.6 | no | no | Attempts - 1 | source |
| 90 | EPAM AI/Run Developer Agent | Claude 3.5 Sonnet Anthropic | 55.4 | 2024-12-12 | unknown | no | no | Attempts - 2+ | source |
| 91 | mini-SWE-agent SWE-agent | Qwen3-Coder 480B/A35B Instruct Qwen | 55.4 | 2025-08-02 | 123.96 | yes | yes | Attempts - 1 | source |
| 92 | mini-SWE-agent SWE-agent | GLM 4.6 Z.ai | 55.4 | 2025-12-01 | 48.31 | yes | yes | Attempts - 1 | unknown |
| 93 | Amazon Q Developer Agent AWS | Undisclosed | 55.0 | 2024-12-02 | unknown | no | no | Attempts - 2+ | source |
| 94 | devlo | Claude 3.5 Sonnet Anthropic | 54.2 | 2024-11-08 | unknown | no | no | Attempts - 2+ | source |
| 95 | mini-SWE-agent SWE-agent | GLM 4.5 Z.ai | 54.2 | 2025-08-22 | 148.6 | yes | yes | Attempts - 1 | source |
| 96 | mini-SWE-agent SWE-agent | Devstral (2512) mistral | 53.8 | 2025-12-09 | unknown | yes | yes | Attempts - 1 | unknown |
| 97 | FrogBoss-32B-2510 Microsoft | Frogboss 32B 2510 | 53.6 | 2025-11-10 | unknown | no | no | Attempts - 1 | source |
| 98 | mini-SWE-agent SWE-agent | Gemini 2.5 Pro Google DeepMind | 53.6 | 2025-07-26 | 144.19 | no | yes | Attempts - 1 | source |
| 99 | CodeSweep - SWE-agent CodeSweep Inc. | Kimi K2 Instruct Moonshot AI | 53.4 | 2025-08-04 | unknown | yes | no | Attempts - 1 | source |
| 100 | Bracket.sh | Undisclosed | 53.2 | 2025-01-20 | unknown | no | no | Attempts - 2+ | source |
| 101 | OpenHands OpenHands | CodeAct v2.1 (claude-3-5-sonnet-20241022) Anthropic | 53.0 | 2024-10-29 | unknown | no | yes | Attempts - 2+ | source |
| 102 | mini-SWE-agent SWE-agent | Claude 3.7 Sonnet Anthropic | 52.8 | 2025-07-20 | 177.31 | no | yes | Attempts - 1 | source |
| 103 | Google Jules Google | Gemini 2.0 Flash (v20241212-experimental) Google DeepMind | 52.2 | 2024-12-12 | unknown | no | no | Attempts - 2+ | source |
| 104 | EntroPO + R2E 42-b3yond-6ug | Qwen3-Coder-30B-A3B-Instruct Qwen | 52.2 | 2025-09-01 | unknown | yes | no | Attempts - 1 | source |
| 105 | Engine Labs Engine Labs | Claude 3.5 Sonnet Anthropic | 51.8 | 2024-11-25 | unknown | no | no | Attempts - 1 | source |
| 106 | AutoCodeRover-v2.1 | Claude 3.5-Sonnet-20241022 Anthropic | 51.6 | 2025-01-22 | unknown | no | no | Attempts - 2+ | source |
| 107 | OpenHands Qwen | Qwen3-Coder-30B-A3B-Instruct Qwen | 51.6 | 2025-08-05 | unknown | yes | no | Attempts - 1 | source |
| 108 | Agentless-1.5 Agentless | Claude 3.5 Sonnet Anthropic | 50.8 | 2024-12-02 | unknown | no | no | Attempts - 2+ | source |
| 109 | Solver | Undisclosed | 50.0 | 2024-10-28 | unknown | no | no | Attempts - 2+ | source |
| 110 | Bytedance MarsCode Agent Bytedance | Undisclosed | 50.0 | 2024-11-25 | unknown | no | no | Attempts - 2+ | source |
| 111 | nFactorial | Multiple Anthropic | 49.2 | 2024-11-05 | unknown | no | no | Attempts - 2+ | source |
| 112 | Tools Anthropic | Claude 3.5 Sonnet Anthropic | 49.0 | 2024-10-22 | unknown | no | no | Attempts - 2+ | source |
| 113 | Composio SWE-Kit | Multiple Anthropic | 48.6 | 2024-10-25 | unknown | no | yes | Attempts - 2+ | source |
| 114 | AppMap Navie v2 | Multiple Anthropic | 47.2 | 2024-11-06 | unknown | no | yes | Attempts - 2+ | source |
| 115 | Skywork-SWE-32B Skywork AI | TTS(Bo8) Qwen | 47.0 | 2025-06-16 | unknown | yes | yes | Attempts - 2+ | source |
| 116 | OpenHands OpenHands | DevStral Small 2505 Mistral | 46.8 | 2025-05-20 | unknown | yes | yes | Attempts - 1 | source |
| 117 | Emergent E1 | Multiple Anthropic | 46.6 | 2024-10-23 | unknown | no | no | Attempts - 2+ | source |
| 118 | AutoCodeRover-v2.0 | Claude 3.5-Sonnet-20241022 Anthropic | 46.2 | 2024-11-08 | unknown | no | no | Attempts - 2+ | source |
| 119 | PatchPilot | Co-PatcheR | 46.0 | 2025-05-28 | unknown | yes | no | Attempts - 2+ | source |
| 120 | Solver | Undisclosed | 45.4 | 2024-09-24 | unknown | no | no | Attempts - 2+ | source |
| 121 | Gru | Undisclosed | 45.2 | 2024-08-24 | unknown | no | no | Attempts - 2+ | source |
| 122 | FrogMini-14B-2510 Microsoft | Frogmini 14B 2510 | 45.0 | 2025-11-10 | unknown | no | no | Attempts - 1 | source |
| 123 | mini-SWE-agent SWE-agent | o4-mini OpenAI | 45.0 | 2025-07-26 | 104.99 | no | yes | Attempts - 1 | source |
| 124 | CodeShellAgent | Gemini 2.0 Flash (Experimental) Google DeepMind | 44.2 | 2025-01-18 | unknown | no | no | Attempts - 2+ | source |
| 125 | mini-SWE-agent SWE-agent | Kimi K2 Instruct Moonshot AI | 43.8 | 2025-08-07 | 265.8 | yes | yes | Attempts - 1 | source |
| 126 | Solver | Undisclosed | 43.6 | 2024-09-20 | unknown | no | no | Attempts - 2+ | source |
| 127 | Agentless Lite Agentless | O3 Mini OpenAI | 42.4 | 2025-02-14 | unknown | no | yes | Attempts - 2+ | source |
| 128 | Amazon Nova Premier 1.0 Amazon Nova | Amazon.nova Premier v1:0 | 42.4 | 2025-05-27 | unknown | no | no | Attempts - 2+ | source |
| 129 | R2E-Gym Agentica | DeepSWE-Preview | 42.2 | 2025-06-29 | unknown | yes | no | Attempts - 1 | source |
| 130 | SWE-Exp SWE-Exp | DeepSeek V3 0324 DeepSeek | 42.0 | 2025-08-06 | unknown | yes | no | Attempts - 2 | source |
| 131 | nFactorial | Multiple Anthropic | 41.6 | 2024-10-30 | unknown | no | no | Attempts - 2+ | source |
| 132 | ugaiforge | Undisclosed | 41.6 | 2025-01-12 | unknown | no | no | Attempts - 2+ | unknown |
| 133 | Agentless Mini Agentless | Llama3-SWE-RL-70B Meta | 41.2 | 2025-02-26 | unknown | no | no | Attempts - 2+ | source |
| 134 | Honeycomb | Undisclosed | 40.6 | 2024-08-20 | unknown | no | no | Attempts - 2+ | source |
| 135 | Composio SWEkit | Claude 3.5 Sonnet Anthropic | 40.6 | 2024-10-16 | unknown | no | no | Attempts - 2+ | source |
| 136 | Tools Anthropic | Claude 3.5 Haiku Anthropic | 40.6 | 2024-10-22 | unknown | no | no | Attempts - 2+ | source |
| 137 | Nebius AI | Multiple Meta | 40.6 | 2024-11-13 | unknown | no | no | Attempts - 2+ | source |
| 138 | SWE-agent SWE-agent | SWE-agent-LM-32B Qwen | 40.2 | 2025-05-11 | unknown | yes | yes | Attempts - 1 | source |
| 139 | EPAM AI/Run Developer Agent | Claude 3.5 Sonnet Anthropic | 39.6 | 2024-10-29 | unknown | no | no | Attempts - 2+ | source |
| 140 | mini-SWE-agent SWE-agent | GPT 4.1 OpenAI | 39.58 | 2025-07-26 | 73.16 | no | yes | Attempts - 1 | source |
| 141 | Amazon Q Developer Agent AWS | Undisclosed | 38.8 | 2024-07-21 | unknown | no | no | Attempts - 2+ | source |
| 142 | Agentless-1.5 Agentless | GPT-4o OpenAI | 38.8 | 2024-10-28 | unknown | no | no | Attempts - 1 | source |
| 143 | AutoCodeRover | GPT-4o OpenAI | 38.4 | 2024-06-28 | unknown | no | no | Attempts - 2+ | source |
| 144 | Skywork-SWE-32B Skywork AI | Qwen2.5 Coder 32B Instruct Qwen | 38.0 | 2025-06-16 | unknown | yes | yes | Attempts - 1 | source |
| 145 | SWE-agent SWE-agent | DevStral Small 2507 Mistral | 38.0 | 2025-07-25 | unknown | yes | yes | Attempts - 1 | source |
| 146 | Factory Code Droid | Undisclosed | 37.0 | 2024-06-17 | unknown | no | no | Attempts - 2+ | source |
| 147 | mini-SWE-agent SWE-agent | GPT 5 nano OpenAI | 34.8 | 2025-08-07 | 19.04 | no | yes | Attempts - 1 | source |
| 148 | SWE-agent SWE-agent | Claude 3.5 Sonnet Anthropic | 33.6 | 2024-06-20 | unknown | no | yes | Attempts - 1 | source |
| 149 | SWE-Fixer | Qwen2.5 (7B + 72B) Qwen | 32.8 | 2025-03-06 | unknown | yes | yes | Attempts - 1 | source |
| 150 | MASAI | GPT-4o OpenAI | 32.6 | 2024-06-12 | unknown | no | no | Attempts - 2+ | source |
| 151 | Artemis Agent v1 | Undisclosed | 32.0 | 2024-11-20 | unknown | no | no | Attempts - 2+ | source |
| 152 | nFactorial | Undisclosed | 31.6 | 2024-10-07 | unknown | no | no | Attempts - 2+ | source |
| 153 | SWE-Fixer | Qwen2.5 (7B + 72B) Qwen | 30.2 | 2024-11-28 | unknown | no | no | Attempts - 2+ | source |
| 154 | Lingma Agent Alibaba | Lingma SWE-GPT 72b (v0925) OpenAI | 28.8 | 2024-10-02 | unknown | no | no | Attempts - 2+ | source 1 · source 2 |
| 155 | mini-SWE-agent SWE-agent | Gemini 2.5 Flash Google DeepMind | 28.73 | 2025-07-26 | 66.94 | no | yes | Attempts - 1 | source |
| 156 | EPAM AI/Run Developer Agent | GPT-4o OpenAI | 27.0 | 2024-10-16 | unknown | no | no | Attempts - 2+ | source |
| 157 | AppMap Navie | GPT-4o OpenAI | 26.2 | 2024-06-15 | unknown | no | yes | Attempts - 2+ | source |
| 158 | mini-SWE-agent SWE-agent | gpt-oss-120b OpenAI | 26.0 | 2025-08-07 | 28.56 | yes | yes | Attempts - 1 | source |
| 159 | nFactorial | Undisclosed | 25.8 | 2024-10-01 | unknown | no | no | Attempts - 2+ | source |
| 160 | Amazon Q Developer Agent AWS | Undisclosed | 25.6 | 2024-05-09 | unknown | no | no | Attempts - 2+ | source |
| 161 | Lingma Agent Alibaba | Lingma SWE-GPT 72b (v0918) OpenAI | 25.0 | 2024-09-18 | unknown | no | no | Attempts - 2+ | source |
| 162 | EPAM AI/Run Developer Agent | GPT-4o OpenAI | 24.0 | 2024-08-20 | unknown | no | no | Attempts - 2+ | source |
| 163 | mini-SWE-agent SWE-agent | GPT 4.1 mini OpenAI | 23.94 | 2025-07-20 | 217.58 | no | yes | Attempts - 1 | source |
| 164 | SWE-agent SWE-agent | GPT-4o OpenAI | 23.2 | 2024-07-28 | unknown | no | yes | Attempts - 1 | source |
| 165 | MCTS-Refine-7B | MCTS Refine 7B | 23.2 | 2025-06-27 | unknown | yes | no | Attempts - 1 | source |
| 166 | SWE-agent SWE-agent | GPT-4 (1106) OpenAI | 22.4 | 2024-04-02 | unknown | no | yes | Attempts - 1 | source |
| 167 | mini-SWE-agent SWE-agent | GPT 4o OpenAI | 21.62 | 2025-07-20 | 765.57 | no | yes | Attempts - 1 | source |
| 168 | mini-SWE-agent SWE-agent | Llama 4 Maverick Instruct Meta | 21.04 | 2025-07-20 | 156.85 | no | yes | Attempts - 1 | source |
| 169 | Lingma Agent Alibaba | Lingma SWE-GPT 7b (v0925) OpenAI | 18.2 | 2024-10-02 | unknown | no | no | Attempts - 2+ | source 1 · source 2 |
| 170 | SWE-agent SWE-agent | Claude 3 Opus Anthropic | 15.8 | 2024-04-02 | unknown | no | yes | Attempts - 1 | source |
| 171 | mini-SWE-agent SWE-agent | Gemini 2.0 Flash Google DeepMind | 13.52 | 2025-07-26 | 0 | no | yes | Attempts - 1 | source |
| 172 | Lingma Agent Alibaba | Lingma SWE-GPT 7b (v0918) OpenAI | 10.2 | 2024-09-18 | unknown | no | no | Attempts - 2+ | source |
| 173 | mini-SWE-agent SWE-agent | Llama 4 Scout Instruct Meta | 9.06 | 2025-07-20 | 57.82 | no | yes | Attempts - 1 | source |
| 174 | mini-SWE-agent SWE-agent | Qwen2.5-Coder 32B Instruct Qwen | 9.0 | 2025-08-03 | 34.06 | yes | yes | Attempts - 1 | source |
| 175 | RAG baseline | Claude 3 Opus Anthropic | 7.0 | 2024-04-02 | unknown | no | yes | Attempts - 2+ | source |
| 176 | RAG baseline | Claude 2 Anthropic | 4.4 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
| 177 | RAG baseline | GPT-4 (1106) OpenAI | 2.8 | 2024-04-02 | unknown | no | yes | Attempts - 2+ | unknown |
| 178 | RAG baseline SWE-agent | SWE-Llama 7B Meta | 1.4 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
| 179 | RAG baseline SWE-agent | SWE-Llama 13B Meta | 1.2 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
| 180 | RAG baseline SWE-agent | GPT-3.5 OpenAI | 0.4 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
Lite · 300 official instances (count from swebench.com cards) · 84 rows in the official leaderboard JSON fetched 2026-11-22T12:28:32-05:00. Source: SWE-bench official leaderboard JSON. “Resolved” is the official resolved-% score. Cost is whatever the official board included (USD in their file); unknown if they left it null. “Checked” is the official field. Each row keeps its own date from SWE-bench.
| # | Agent | Model | Resolved % | Date | Cost | Open model | Checked | Attempts | Site |
|---|---|---|---|---|---|---|---|---|---|
| 1 | ExpeRepair-v1.0 | Claude 4 Sonnet Anthropic | 60.33 | 2025-06-25 | unknown | no | no | Attempts - 2+ | source |
| 2 | Refact.ai Agent Refact.ai | Multiple Anthropic | 60.0 | 2025-04-25 | unknown | no | no | Attempts - 1 | source |
| 3 | KGCompass GLEAM-Lab | Claude 4 Sonnet Anthropic | 58.33 | 2025-09-06 | unknown | no | no | Attempts - 2+ | source |
| 4 | SWE-agent SWE-agent | Claude 4 Sonnet Anthropic | 56.67 | 2025-05-26 | unknown | no | yes | Attempts - 1 | source |
| 5 | Isoform | Undisclosed | 55.0 | 2025-01-14 | unknown | no | no | Attempts - 2+ | source |
| 6 | SemAgent_Multi-v1.0 Columbia | Multiple Anthropic | 51.67 | 2025-06-25 | unknown | no | no | Attempts - 2+ | source |
| 7 | Isea | Claude 3.5 Sonnet Anthropic | 51.33 | 2025-09-11 | unknown | no | no | Attempts - 2+ | unknown |
| 8 | EntroPO + R2E 42-b3yond-6ug | Qwen3-Coder-30B-A3B-Instruct Qwen | 49.67 | 2025-09-01 | unknown | yes | no | Attempts - 2+ | source |
| 9 | Blackbox AI Agent | Undisclosed | 49.0 | 2024-12-20 | unknown | no | no | unknown | source |
| 10 | Codev co-dev.ai | Multiple Anthropic | 49.0 | 2025-05-28 | unknown | no | no | Attempts - 1 | source |
| 11 | Gru | Undisclosed | 48.67 | 2024-12-08 | unknown | no | no | unknown | source |
| 12 | Globant Code Fixer Agent | Undisclosed | 48.33 | 2024-11-27 | unknown | no | no | unknown | source |
| 13 | ExpeRepair-v1.0 | Multiple Anthropic | 48.33 | 2025-06-13 | unknown | no | no | Attempts - 2+ | source |
| 14 | SWE-agent SWE-agent | Claude 3.7 Sonnet Anthropic | 48.0 | 2025-02-26 | unknown | no | yes | Attempts - 1 | source |
| 15 | devlo | Claude 3.5 Sonnet Anthropic | 47.33 | 2024-11-22 | unknown | no | no | unknown | source |
| 16 | DARS Agent | Multiple Anthropic | 47.0 | 2025-02-05 | unknown | no | no | unknown | source |
| 17 | KGCompass GLEAM-Lab | Claude 3.5 Sonnet Anthropic | 46.0 | 2025-06-19 | unknown | no | no | Attempts - 2+ | source |
| 18 | EntroPO + R2E 42-b3yond-6ug | Qwen3-Coder-30B-A3B-Instruct Qwen | 45.0 | 2025-09-01 | unknown | yes | no | Attempts - 1 | source |
| 19 | Kodu-v1 | Claude 3.5 Sonnet Anthropic | 44.67 | 2024-12-07 | unknown | no | no | unknown | source |
| 20 | CodeFuse-CGM Ant Group | Undisclosed | 44.0 | 2025-03-10 | unknown | yes | yes | Attempts - 2+ | source |
| 21 | CodeStory Aide | Mixed Models | 43.0 | 2024-07-02 | unknown | no | no | unknown | source |
| 22 | Lingxi | Claude 3.5 Sonnet Anthropic | 42.67 | 2025-05-09 | unknown | no | no | unknown | source |
| 23 | OpenHands OpenHands | CodeAct v2.1 (claude-3-5-sonnet-20241022) Anthropic | 41.67 | 2024-10-25 | unknown | no | yes | unknown | source |
| 24 | Codart AI Codart AI | Multiple Anthropic | 41.67 | 2025-05-15 | unknown | no | no | Attempts - 1 | source |
| 25 | PatchKitty-0.9 | Claude 3.5 Sonnet Anthropic | 41.33 | 2024-12-20 | unknown | no | no | unknown | unknown |
| 26 | Composio SWE-Kit | Multiple Anthropic | 41.0 | 2024-10-30 | unknown | no | no | unknown | source |
| 27 | OrcaLoca + Agentless-1.5 Agentless | Claude 3.5 Sonnet Anthropic | 41.0 | 2025-01-13 | unknown | no | no | unknown | source |
| 28 | Agentless-1.5 Agentless | Claude 3.5 Sonnet Anthropic | 40.67 | 2024-12-02 | unknown | no | no | unknown | source |
| 29 | OpenCSG Starship Agentic Coder | GPT-4 (0806) OpenAI | 39.67 | 2025-01-13 | unknown | no | no | unknown | source |
| 30 | Bytedance MarsCode Agent Bytedance | Undisclosed | 39.33 | 2024-09-12 | unknown | no | no | unknown | source |
| 31 | Moatless Tools Moatless Tools | Claude 3.5 Sonnet Anthropic | 39.0 | 2025-01-14 | unknown | no | yes | unknown | source |
| 32 | Honeycomb | Undisclosed | 38.33 | 2024-08-20 | unknown | no | no | unknown | source |
| 33 | Moatless Tools Moatless Tools | Claude 3.5 Sonnet Anthropic | 38.33 | 2024-11-17 | unknown | no | yes | unknown | source |
| 34 | AbanteAI MentatBot | GPT-4o OpenAI | 38.0 | 2024-06-27 | unknown | no | no | unknown | source |
| 35 | Patched.Codes Patchwork | Claude 3.5 Sonnet Anthropic | 37.0 | 2025-01-04 | unknown | no | no | unknown | source |
| 36 | KGCompass GLEAM-Lab | DeepSeek V3 DeepSeek | 36.67 | 2025-06-09 | unknown | yes | no | Attempts - 2+ | source |
| 37 | AppMap Navie v2 | Multiple Anthropic | 36.0 | 2024-11-13 | unknown | no | yes | unknown | source |
| 38 | Gru | Undisclosed | 35.67 | 2024-08-11 | unknown | no | no | unknown | source |
| 39 | CodeFuse-AAIS | Undisclosed | 35.67 | 2025-01-04 | unknown | no | no | unknown | source |
| 40 | Isoform | Undisclosed | 35.0 | 2024-08-29 | unknown | no | no | unknown | source |
| 41 | Bytedance MarsCode Agent Bytedance | GPT-4o OpenAI | 34.0 | 2024-07-23 | unknown | no | no | unknown | source |
| 42 | SuperCoder2.0 | Undisclosed | 34.0 | 2024-08-06 | unknown | no | no | unknown | source |
| 43 | Alibaba Lingma Agent Alibaba | Undisclosed | 33.0 | 2024-06-22 | unknown | no | no | unknown | source |
| 44 | Agentless Lite Agentless | O3 Mini OpenAI | 32.33 | 2025-02-14 | unknown | no | yes | unknown | source |
| 45 | Agentless-1.5 Agentless | GPT-4o OpenAI | 32.0 | 2024-10-28 | unknown | no | no | unknown | source |
| 46 | Factory Code Droid | Undisclosed | 31.33 | 2024-06-17 | unknown | no | no | unknown | source |
| 47 | CodeShellTester | GPT-4o OpenAI | 31.33 | 2024-11-11 | unknown | no | no | unknown | source |
| 48 | AutoCodeRover | GPT-4o OpenAI | 30.67 | 2024-06-21 | unknown | no | no | unknown | source |
| 49 | Moatless Tools Moatless Tools | Deepseek V3 DeepSeek | 30.67 | 2025-01-11 | unknown | yes | yes | unknown | source |
| 50 | Aegis | o3-mini_1.0 OpenAI | 30.33 | 2025-02-07 | unknown | no | no | unknown | source |
| 51 | AIGCode Infant-Coder | Undisclosed | 30.0 | 2024-09-08 | unknown | no | no | unknown | source |
| 52 | Kortix AI | claude-3-5-sonnet-20241022 Anthropic | 30.0 | 2024-12-03 | unknown | no | yes | unknown | source |
| 53 | Amazon Q Developer Agent AWS | Undisclosed | 29.67 | 2024-07-21 | unknown | no | no | unknown | source |
| 54 | Agentless + RepoGraph Agentless | GPT-4o OpenAI | 29.67 | 2024-08-08 | unknown | no | no | unknown | source |
| 55 | CodeR | GPT-4 (1106) OpenAI | 28.33 | 2024-06-04 | unknown | no | no | unknown | source |
| 56 | reproducedRG | GPT-4o OpenAI | 28.0 | 2024-11-17 | unknown | no | no | unknown | unknown |
| 57 | SIMA | GPT-4o OpenAI | 27.67 | 2024-07-06 | unknown | no | no | unknown | source |
| 58 | MASAI | GPT-4o OpenAI | 27.33 | 2024-06-12 | unknown | no | no | unknown | source |
| 59 | Agentless Agentless | GPT-4o OpenAI | 27.33 | 2024-06-30 | unknown | no | no | unknown | source |
| 60 | IBM Research Agent-101 IBM | Undisclosed | 26.67 | 2024-06-12 | unknown | no | no | unknown | source |
| 61 | Moatless Tools Moatless Tools | Claude 3.5 Sonnet Anthropic | 26.67 | 2024-06-23 | unknown | no | yes | unknown | source |
| 62 | OpenHands OpenHands | CodeAct v1.8 Anthropic | 26.67 | 2024-07-25 | unknown | no | yes | unknown | source |
| 63 | Aider | GPT-4o & Claude 3 Opus Anthropic | 26.33 | 2024-05-23 | unknown | no | no | unknown | source |
| 64 | HyperAgent | GPT-4o OpenAI | 25.33 | 2024-09-25 | unknown | no | no | unknown | source |
| 65 | Moatless Tools Moatless Tools | GPT-4o OpenAI | 24.67 | 2024-06-17 | unknown | no | yes | unknown | source |
| 66 | SWE-Fixer | Qwen2.5 (7B + 72B) Qwen | 24.67 | 2025-03-06 | unknown | yes | yes | unknown | source |
| 67 | OpenCSG StarShip CodeGenAgent | GPT-4 (0613) OpenAI | 23.67 | 2024-05-24 | unknown | no | no | unknown | source |
| 68 | IBM AI Agent SWE-1.0 IBM | Multiple | 23.67 | 2024-10-16 | unknown | no | no | unknown | source |
| 69 | SWE-Fixer | Qwen2.5 (7B + 72B) Qwen | 23.33 | 2024-11-28 | unknown | yes | no | unknown | source |
| 70 | SWE-agent SWE-agent | Claude 3.5 Sonnet Anthropic | 23.0 | 2024-06-20 | unknown | no | yes | unknown | source |
| 71 | AppMap Navie | GPT-4o OpenAI | 21.67 | 2024-06-15 | unknown | no | yes | unknown | source |
| 72 | Bytedance AutoSE Bytedance | Multiple OpenAI | 21.67 | 2024-08-28 | unknown | no | no | unknown | unknown |
| 73 | Amazon Q Developer Agent AWS | Undisclosed | 20.33 | 2024-05-09 | unknown | no | no | unknown | source |
| 74 | AutoCodeRover | GPT-4 (0125) OpenAI | 19.0 | 2024-05-30 | unknown | no | no | unknown | source |
| 75 | SWE-agent SWE-agent | GPT-4o OpenAI | 18.33 | 2024-07-28 | unknown | no | yes | unknown | source |
| 76 | SWE-agent SWE-agent | GPT-4 (1106) OpenAI | 18.0 | 2024-04-02 | unknown | no | yes | unknown | source |
| 77 | MCTS-Refine-7B | MCTS Refine 7B | 16.33 | 2025-06-27 | unknown | yes | no | Attempts - 1 | source |
| 78 | SWE-agent SWE-agent | Claude 3 Opus Anthropic | 11.67 | 2024-04-02 | unknown | no | yes | unknown | source |
| 79 | RAG baseline | Claude 3 Opus Anthropic | 4.33 | 2024-04-02 | unknown | no | yes | unknown | source |
| 80 | RAG baseline | Claude 2 Anthropic | 3.0 | 2023-10-10 | unknown | no | yes | unknown | source |
| 81 | RAG baseline | GPT-4 (1106) OpenAI | 2.67 | 2024-04-02 | unknown | no | yes | unknown | source |
| 82 | RAG baseline SWE-agent | SWE-Llama 7B Meta | 1.33 | 2023-10-10 | unknown | no | yes | unknown | source |
| 83 | RAG baseline SWE-agent | SWE-Llama 13B Meta | 1.0 | 2023-10-10 | unknown | no | yes | unknown | source |
| 84 | RAG baseline SWE-agent | GPT-3.5 OpenAI | 0.33 | 2023-10-10 | unknown | no | yes | unknown | source |
Test · 2294 official instances (count from swebench.com cards) · 24 rows in the official leaderboard JSON fetched 2026-11-22T12:28:32-05:00. Source: SWE-bench official leaderboard JSON. “Resolved” is the official resolved-% score. Cost is whatever the official board included (USD in their file); unknown if they left it null. “Checked” is the official field. Each row keeps its own date from SWE-bench.
| # | Agent | Model | Resolved % | Date | Cost | Open model | Checked | Attempts | Site |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Sonar Foundation Agent Sonar | Claude 4.5 Opus Anthropic | 52.62 | 2025-12-19 | unknown | no | no | Attempts - 1 | source |
| 2 | Salesforce AI Research SAGE Salesforce AI Research | Multiple Anthropic | 44.25 | 2025-10-27 | unknown | no | no | Attempts - 2+ | source |
| 3 | Atlassian Rovo Dev Atlassian | Undisclosed | 41.98 | 2025-06-05 | unknown | no | no | Attempts - 1 | source |
| 4 | Amazon Q Developer Agent AWS | Undisclosed | 37.1 | 2025-05-22 | unknown | no | no | Attempts - 2+ | source |
| 5 | SWE-agent 1.0 SWE-agent | Claude 3.7 Sonnet Anthropic | 33.83 | 2025-02-27 | unknown | no | yes | Attempts - 1 | source |
| 6 | Amazon Q Developer Agent AWS | Undisclosed | 29.99 | 2025-01-31 | unknown | no | no | Attempts - 1 | source |
| 7 | OpenHands OpenHands | CodeAct v2.1 (claude-3-5-sonnet-20241022) Anthropic | 29.38 | 2024-11-03 | unknown | no | yes | Attempts - 1 | source |
| 8 | AutoCodeRover-v2.0 | Claude 3.5-Sonnet-20241022 Anthropic | 24.89 | 2024-11-21 | unknown | no | no | Attempts - 1 | source |
| 9 | Honeycomb | Undisclosed | 22.06 | 2024-08-20 | unknown | no | no | Attempts - 1 | source |
| 10 | Amazon Q Developer Agent AWS | Undisclosed | 19.75 | 2024-07-21 | unknown | no | no | Attempts - 1 | source |
| 11 | Factory Code Droid | Undisclosed | 19.27 | 2024-06-17 | unknown | no | no | Attempts - 1 | source |
| 12 | AutoCodeRover | GPT-4o OpenAI | 18.83 | 2024-06-28 | unknown | no | no | Attempts - 1 | source |
| 13 | SWE-agent SWE-agent | Claude 3.5 Sonnet Anthropic | 18.13 | 2024-06-20 | unknown | no | yes | Attempts - 1 | unknown |
| 14 | AppMap Navie | GPT-4o OpenAI | 14.6 | 2024-06-15 | unknown | yes | yes | Attempts - 1 | source |
| 15 | Amazon Q Developer Agent AWS | Undisclosed | 13.82 | 2024-05-09 | unknown | no | no | Attempts - 1 | source |
| 16 | SWE-agent SWE-agent | GPT-4 (1106) OpenAI | 12.47 | 2024-04-02 | unknown | no | yes | Attempts - 1 | source |
| 17 | SWE-agent SWE-agent | GPT-4o OpenAI | 11.99 | 2024-07-28 | unknown | no | yes | Attempts - 1 | source |
| 18 | SWE-agent SWE-agent | Claude 3 Opus Anthropic | 10.51 | 2024-04-02 | unknown | no | yes | Attempts - 1 | unknown |
| 19 | RAG baseline | Claude 3 Opus Anthropic | 3.79 | 2024-04-02 | unknown | no | yes | Attempts - 1 | source |
| 20 | RAG baseline | Claude 2 Anthropic | 1.96 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
| 21 | RAG baseline | GPT-4 (1106) OpenAI | 1.31 | 2024-04-02 | unknown | no | yes | Attempts - 1 | unknown |
| 22 | RAG baseline SWE-agent | SWE-Llama 13B Meta | 0.7 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
| 23 | RAG baseline SWE-agent | SWE-Llama 7B Meta | 0.7 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
| 24 | RAG baseline SWE-agent | GPT-3.5 OpenAI | 0.17 | 2023-10-10 | unknown | no | yes | Attempts - 1 | unknown |
Multilingual · 300 official instances (count from swebench.com cards) · 13 rows in the official leaderboard JSON fetched 2026-11-22T12:28:32-05:00. Source: SWE-bench official leaderboard JSON. “Resolved” is the official resolved-% score. Cost is whatever the official board included (USD in their file); unknown if they left it null. “Checked” is the official field. Each row keeps its own date from SWE-bench.
| # | Agent | Model | Resolved % | Date | Cost | Open model | Checked | Attempts | Site |
|---|---|---|---|---|---|---|---|---|---|
| 1 | mini-SWE-agent SWE-agent | Gemini 3 Flash Google DeepMind | 72.7 | 2026-02-13 | 105.55 | no | yes | Attempts - 1 | source |
| 2 | mini-SWE-agent SWE-agent | Claude 4.6 Opus Anthropic | 72.0 | 2026-02-13 | 198.89 | no | yes | Attempts - 1 | source |
| 3 | mini-SWE-agent SWE-agent | Claude 4.5 Opus Anthropic | 70.7 | 2026-02-13 | 250.07 | no | yes | Attempts - 1 | source |
| 4 | mini-SWE-agent SWE-agent | GLM 5 Z.ai | 69.7 | 2026-02-13 | 192.74 | yes | yes | Attempts - 1 | source |
| 5 | mini-SWE-agent SWE-agent | Gemini 3 Pro Google DeepMind | 68.7 | 2026-02-13 | 306.33 | no | yes | Attempts - 1 | source |
| 6 | mini-SWE-agent SWE-agent | MiniMax M2.5 Minimax | 68.3 | 2026-02-16 | 30.06 | yes | yes | Attempts - 1 | source |
| 7 | mini-SWE-agent SWE-agent | Kimi K2.5 Moonshot AI | 67.3 | 2026-02-13 | 207.85 | yes | yes | Attempts - 1 | source |
| 8 | mini-SWE-agent SWE-agent | Claude 4.5 Sonnet Anthropic | 67.0 | 2026-02-13 | 200.86 | no | yes | Attempts - 1 | source |
| 9 | mini-SWE-agent SWE-agent | GPT 5.2 OpenAI | 66.7 | 2026-02-13 | 160.92 | no | yes | Attempts - 1 | source |
| 10 | mini-SWE-agent SWE-agent | GPT 5.2 Codex OpenAI | 66.3 | 2026-02-20 | 198.65 | no | no | Attempts - 1 | source |
| 11 | mini-SWE-agent SWE-agent | Claude 4.5 Haiku Anthropic | 64.7 | 2026-02-13 | 113.85 | no | yes | Attempts - 1 | source |
| 12 | mini-SWE-agent SWE-agent | DeepSeek V3.2 DeepSeek | 59.0 | 2026-02-13 | 115.15 | yes | yes | Attempts - 1 | source |
| 13 | mini-SWE-agent SWE-agent | GPT 5 mini OpenAI | 39.7 | 2026-02-13 | 15.49 | no | yes | Attempts - 1 | source |
Multimodal · 480 official instances (count from swebench.com cards) · 22 rows in the official leaderboard JSON fetched 2026-11-22T12:28:32-05:00. Source: SWE-bench official leaderboard JSON. “Resolved” is the official resolved-% score. Cost is whatever the official board included (USD in their file); unknown if they left it null. “Checked” is the official field. Each row keeps its own date from SWE-bench.
| # | Agent | Model | Resolved % | Date | Cost | Open model | Checked | Attempts | Site |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GUIRepair GUIRepair | o3 OpenAI | 35.98 | 2025-07-01 | unknown | no | yes | Attempts - 1 | source |
| 2 | Codefuse_Pycfuse_SVR Codefuse | o3 OpenAI | 35.98 | 2025-11-17 | unknown | no | yes | Attempts - 1 | source |
| 3 | Refact.ai Agent Refact.ai | Claude 4 Sonnet Anthropic | 35.59 | 2025-06-11 | unknown | no | no | Attempts - 1 | source |
| 4 | OpenHands-Versa OpenHands | Claude Sonnet 4 Anthropic | 34.43 | 2025-05-28 | unknown | no | yes | Attempts - 1 | source |
| 5 | GUIRepair GUIRepair | o4-mini OpenAI | 33.85 | 2025-05-31 | unknown | no | yes | Attempts - 1 | source |
| 6 | OpenHands-Versa OpenHands | Claude 3.7 Sonnet Anthropic | 31.33 | 2025-05-09 | unknown | no | yes | Attempts - 1 | source |
| 7 | GUIRepair GUIRepair | GPT-4.1 OpenAI | 31.14 | 2025-05-31 | unknown | no | yes | Attempts - 1 | source |
| 8 | Zencoder | Claude 3.7 Sonnet Anthropic | 30.56 | 2025-04-01 | unknown | no | yes | Attempts - 1 | source |
| 9 | GUIRepair GUIRepair | GPT-4o OpenAI | 30.37 | 2025-05-31 | unknown | no | yes | Attempts - 1 | source |
| 10 | Globant Code Fixer Agent | Undisclosed | 29.59 | 2025-03-25 | unknown | no | yes | Attempts - 1 | source |
| 11 | Zencoder | Undisclosed | 27.08 | 2025-03-11 | unknown | no | yes | Attempts - 1 | source |
| 12 | Agentless Lite Agentless | Claude 3.5 Sonnet Anthropic | 25.34 | 2025-02-26 | unknown | no | yes | Attempts - 1 | source |
| 13 | SWE-agent Multimodal SWE-agent | GPT-4o OpenAI | 12.19 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 14 | SWE-agent SWE-agent | Claude Sonnet 3.5 Anthropic | 12.19 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 15 | SWE-agent JavaScript SWE-agent | Claude Sonnet 3.5 Anthropic | 11.99 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 16 | SWE-agent SWE-agent | GPT-4o OpenAI | 11.99 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 17 | SWE-agent Multimodal SWE-agent | Claude 3.5 Sonnet Anthropic | 11.41 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 18 | SWE-agent JavaScript SWE-agent | GPT-4o OpenAI | 9.28 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 19 | Agentless Agentless | Claude 3.5 Sonnet Anthropic | 6.19 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 20 | RAG | GPT-4o OpenAI | 6.0 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 21 | RAG | Claude 3.5 Sonnet Anthropic | 5.03 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
| 22 | Agentless Agentless | GPT-4o OpenAI | 3.09 | 2024-10-06 | unknown | no | yes | Attempts - 1 | source |
Price and context — OpenRouter
USD per 1M input/output tokens and context window in tokens, 476 models, from the OpenRouter models API, fetched 2026-11-22T12:28:32-05:00. Speed (tokens/sec) and quality scores are unknown in this table.
| Model | Org | Context | In $/M | Out $/M | Weights |
|---|---|---|---|---|---|
| Thinking Machines: Inkling 2 Mini | thinkingmachines | 1048576 | 0.075 | 0.3 | open |
| RadixArk: DaisyRose 27B | radixark | 262144 | 0.15 | 1.875 | open |
| Mistral: Pixtral Large 2502 | mistralai | 128000 | 2.0 | 6.0 | closed / API |
| Google: Gemini 4 Neon | 1048576 | 3.0 | 15.0 | closed / API | |
| Google: Gemini 4 Neon (batch) | 1048576 | 1.5 | 7.5 | closed / API | |
| OpenAI: GPT-6.2 Astra Pro | openai | 1050000 | 8.0 | 40.0 | closed / API |
| OpenAI: GPT-6.2 Astra Pro (batch) | openai | 1050000 | 4.0 | 20.0 | closed / API |
| OpenAI: GPT-6.2 Astra | openai | 1050000 | 8.0 | 40.0 | closed / API |
| OpenAI: GPT-6.2 Astra (batch) | openai | 1050000 | 4.0 | 20.0 | closed / API |
| OpenAI: GPT-6.2 Sol Pro | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-6.2 Sol Pro (batch) | openai | 1050000 | 1.0 | 5.0 | closed / API |
| OpenAI: GPT-6.2 Sol | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-6.2 Sol (batch) | openai | 1050000 | 1.0 | 5.0 | closed / API |
| Tencent: Hy4 flash | tencent | 1048576 | 0.035 | 0.14049999999999999 | open |
| Qwen: Qwen4 Max | qwen | 1000000 | 1.4249999999999998 | 4.2749999999999995 | closed / API |
| inclusionAI: Ling 3.2 Flash (free) | inclusionai | 1048576 | 0.0 | 0.0 | open |
| Google: Gemini 4 Hydrogen | 1048576 | 1.5 | 7.5 | closed / API | |
| Google: Gemini 4 Hydrogen (batch) | 1048576 | 0.75 | 3.75 | closed / API | |
| Baidu: ERNIE-5.5-Flash-Preview (free) | baidu | 1024000 | 0.0 | 0.0 | open |
| Baidu: ERNIE-Light-DT-1.0 | baidu | 1024000 | 1.13 | 3.9400000000000004 | closed / API |
| Google: Gemini 4 Carbon | 1048576 | 2.0 | 10.0 | closed / API | |
| Google: Gemini 4 Carbon (batch) | 1048576 | 1.0 | 5.0 | closed / API | |
| Nex AGI: Nex-N2.6 | nex-agi | 262144 | 0.12 | 0.39999999999999997 | open |
| Thinking Machines: Inkling 2 Small | thinkingmachines | 1048576 | 0.3 | 1.0 | open |
| Anthropic: Claude Fable 5.2 | anthropic | 1000000 | 8.0 | 40.0 | closed / API |
| Anthropic: Claude Fable 5.2 (batch) | anthropic | 1000000 | 4.0 | 20.0 | closed / API |
| Meta: Muse Ember | meta | 1048576 | 0.09999999999999999 | 0.5 | closed / API |
| LiquidAI: LFM3-1.7B | liquid | 262144 | 0.08 | 0.19999999999999998 | open |
| LiquidAI: LFM3-1.7B (free) | liquid | 262144 | 0.0 | 0.0 | open |
| Google: Gemini Omni 1.2 Flash | 131072 | 1.5 | 9.0 | closed / API | |
| KT: Mi:dm K 3.0 Pro | kt | 261795 | 0.19999999999999998 | 0.7999999999999999 | closed / API |
| Google: Gemini Omni 1.1 Flash | 131072 | 1.5 | 9.0 | closed / API | |
| Ricursive: Ripple 65B Preview | ricursiveai | 1048576 | 0.09999999999999999 | 0.3 | open |
| Tencent: Hy4 | tencent | 1048576 | 0.417 | 1.2505 | open |
| Motif Technologies: Motif 3 | motif-technologies | 262144 | 0.3 | 1.2 | open |
| Z.ai: GLM 5.3 FlashX Legal | z-ai | 1048576 | 0.37 | 1.25 | closed / API |
| Particle AI: Particle-1 | particle-ai | 1048576 | 0.07 | 0.5 | closed / API |
| Particle AI: Particle-1 (free) | particle-ai | 1048576 | 0.0 | 0.0 | closed / API |
| OpenAI: GPT-6.1 Terra Pro | openai | 1050000 | 1.0 | 5.0 | closed / API |
| OpenAI: GPT-6.1 Terra Pro (batch) | openai | 1050000 | 0.5 | 2.5 | closed / API |
| OpenAI: GPT-6.1 Terra | openai | 1050000 | 1.0 | 5.0 | closed / API |
| OpenAI: GPT-6.1 Terra (batch) | openai | 1050000 | 0.5 | 2.5 | closed / API |
| MoonshotAI: Kimi K3.1 Pro | moonshotai | 1048576 | 0.2403 | 8.402299999999999 | open |
| MoonshotAI: Kimi K3.1 Pro (batch) | moonshotai | 1048576 | 0.372 | 5.582999999999999 | open |
| Recursive: Flint (free) | recursive | 262144 | 0.0 | 0.0 | open |
| MiniMax: MiniMax M3.1 Flash | minimax | 1048576 | 0.075 | 0.3 | open |
| DeepSeek: DeepSeek V4.1 Flash 1017 | deepseek | 1048576 | 0.15 | 0.6 | open |
| DeepSeek: DeepSeek V4.1 Flash 1017 (batch) | deepseek | 1048576 | 0.035 | 0.149 | open |
| Elorian: Glimpse | elorian | 262144 | 0.6 | 3.0 | closed / API |
| Reflection: Beam 1.0 501B A23B | reflection | 1048576 | 0.25 | 1.0 | open |
| PrismML: Ternary Bonsai 2 9B | prism-ml | 262144 | 0.08 | 0.13 | open |
| Moondream: Moondream 3 Preview | moondream | 32000 | 0.3 | 2.5 | open |
| Moondream: Moondream 3.1 | moondream | 32000 | 0.3 | 2.5 | open |
| Meta: Muse Spark Open | meta | 1048576 | 1.25 | 4.25 | open |
| NVIDIA: Nemotron 3.5 Super VL | nvidia | 1048576 | 0.15 | 0.47 | open |
| NVIDIA: Nemotron 3.5 Super VL (free) | nvidia | 1000000 | 0.0 | 0.0 | open |
| Pareto 26.10 | unbiased | 1048576 | 0.7999999999999999 | 3.1999999999999997 | closed / API |
| Pareto Latest | ~unbiased | 1048576 | 0.7999999999999999 | 3.1999999999999997 | closed / API |
| Pareto 26.9 | unbiased | 262144 | 2.5 | 7.5 | closed / API |
| LiquidAI: LFM2.5-24B-A2B | liquid | 128000 | 0.049999999999999996 | 0.7 | open |
| Google: Gemini 4 Argon | 1048576 | 2.0 | 10.0 | closed / API | |
| Google: Gemini 4 Argon (batch) | 1048576 | 1.0 | 5.0 | closed / API | |
| ScaleDown: Summarize | scaledown | 1000000 | 0.04 | 0.0 | closed / API |
| ScaleDown: Extract | scaledown | 1000000 | 0.04 | 0.0 | closed / API |
| Meta: Muse Wonder | meta | 1048576 | 4.0 | 20.0 | closed / API |
| StepFun: Step 5 Preview | stepfun | 1000000 | 1.0 | 2.7 | open |
| Anthropic: Claude Haiku 5.5 | anthropic | 1000000 | 0.09999999999999999 | 0.5 | closed / API |
| Anthropic: Claude Haiku 5.5 (batch) | anthropic | 1000000 | 0.049999999999999996 | 0.25 | closed / API |
| Google: Nano Banana 2.1 | 65536 | 1.5 | 7.5 | closed / API | |
| Mistral: Mistral Large 4 | mistralai | 1048576 | 0.7 | 2.0999999999999996 | open |
| inclusionAI: Ling 3.1 Flash | inclusionai | 1000000 | 0.06 | 0.18 | open |
| Pareto 26.10 Preview | unbiased | 1048576 | 0.7999999999999999 | 3.1999999999999997 | closed / API |
| OpenAI: GPT-6.1 Sol Pro | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-6.1 Sol Pro (batch) | openai | 1050000 | 1.0 | 5.0 | closed / API |
| OpenAI: GPT-6.1 Sol | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-6.1 Sol (batch) | openai | 1050000 | 1.0 | 5.0 | closed / API |
| Anthropic: Claude Sonnet 5.5 | anthropic | 1000000 | 2.0 | 10.0 | closed / API |
| Anthropic: Claude Sonnet 5.5 (batch) | anthropic | 1000000 | 1.0 | 5.0 | closed / API |
| Perceptron: Perceptron Mk1.5 | perceptron | 36864 | 0.15 | 1.5 | closed / API |
| Fireworks: Ember-1 | fireworks | 1048576 | 3.0 | 15.0 | closed / API |
| Z.ai: GLM 5.3 Prime | z-ai | 1000000 | 1.68 | 5.28 | closed / API |
| Qwen: Qwen3.8 Max Prime | qwen | 1000000 | 4.0 | 12.0 | closed / API |
| AionLabs: Aion 3.5 Mini | aion-labs | 262144 | 0.7 | 1.4 | closed / API |
| AionLabs: Aion 3.5 | aion-labs | 262144 | 3.0 | 6.0 | closed / API |
| Upstage: Solar Mini 4 | upstage | 524288 | 0.049999999999999996 | 0.19999999999999998 | closed / API |
| Cohere: Command A+ | cohere | 192000 | 0.3 | 1.5 | open |
| OpenAI: GPT-6 Luna Pro | openai | 1050000 | 0.09999999999999999 | 0.5 | closed / API |
| OpenAI: GPT-6 Luna Pro (batch) | openai | 1050000 | 0.049999999999999996 | 0.25 | closed / API |
| OpenAI: GPT-6 Luna | openai | 1050000 | 0.09999999999999999 | 0.5 | closed / API |
| OpenAI: GPT-6 Luna (batch) | openai | 1050000 | 0.049999999999999996 | 0.25 | closed / API |
| OpenAI: GPT-6 Sol Pro | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-6 Sol Pro (batch) | openai | 1050000 | 1.0 | 5.0 | closed / API |
| OpenAI: GPT-6 Sol | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-6 Sol (batch) | openai | 1050000 | 1.0 | 5.0 | closed / API |
| Anthropic: Claude Opus 5.5 | anthropic | 1000000 | 4.0 | 20.0 | closed / API |
| Anthropic: Claude Opus 5.5 (batch) | anthropic | 1000000 | 2.0 | 10.0 | closed / API |
| Xiaomi: MiMo-V2.6-Pro-UltraSpeed | xiaomi | 1048576 | 4.35 | 8.7 | closed / API |
| Xiaomi: MiMo-V2.6-Flash | xiaomi | 1048576 | 0.14 | 0.28 | open |
| Xiaomi: MiMo-V2.6-Pro | xiaomi | 1048576 | 0.435 | 0.87 | open |
| SpaceXSI: Grok 4.7 | x-ai | 500000 | 2.0 | 6.0 | closed / API |
| SpaceXSI: Grok 4.7 (batch) | x-ai | 500000 | 1.5999999999999999 | 4.8 | closed / API |
| Qwen: Qwen3.8 Omni Flash | qwen | 1000000 | 0.15 | 0.47 | closed / API |
| PrismML: Ternary Bonsai 2 27B | prism-ml | 262144 | 0.075 | 0.5 | open |
| Z.ai: GLM 5.3 FlashX | z-ai | 1048576 | 0.37 | 1.25 | closed / API |
| DeepSeek: DeepSeek Pro Latest | ~deepseek | 1048576 | 0.1188 | 8.0 | closed / API |
| DeepSeek: DeepSeek Flash Latest | ~deepseek | 1048576 | 0.036 | 0.5 | closed / API |
| Inference.net: Schematron V2 Turbo | inference-net | 128000 | 0.03 | 0.15 | open |
| Inference.net: Schematron V2 Small | inference-net | 128000 | 0.049999999999999996 | 0.22999999999999998 | open |
| OpenAI: GPT Astra Latest | ~openai | 1050000 | 8.0 | 40.0 | closed / API |
| OpenAI: GPT Sol Latest | ~openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT Terra Latest | ~openai | 1050000 | 1.0 | 5.0 | closed / API |
| OpenAI: GPT Luna Latest | ~openai | 1050000 | 0.09999999999999999 | 0.5 | closed / API |
| Sakana: Fugu Ultra v2 | sakana | 1000000 | 5.0 | 30.0 | closed / API |
| Sakana: Fugu Max | sakana | 1000000 | 2.0 | 6.0 | closed / API |
| inclusionAI: Ling 3.0 Flash VL | inclusionai | 131072 | 0.06 | 0.18 | open |
| DeepSeek: DeepSeek V4.1 Flash | deepseek | 1048576 | 0.049800000000000004 | 0.19920000000000002 | open |
| DeepSeek: DeepSeek V4.1 Flash (batch) | deepseek | 1048576 | 0.049999999999999996 | 0.15 | open |
| Nex AGI: Nex-N2.5-Mini | nex-agi | 262144 | 0.024999999999999998 | 0.09999999999999999 | open |
| Nex AGI: Nex-N2.5-Pro | nex-agi | 262144 | 0.075 | 0.25 | open |
| OpenAI: GPT-6 Astra | openai | 1050000 | 10.0 | 50.0 | closed / API |
| OpenAI: GPT-6 Astra (batch) | openai | 1050000 | 5.0 | 25.0 | closed / API |
| OpenAI: GPT-6 Astra Pro | openai | 1050000 | 10.0 | 50.0 | closed / API |
| OpenAI: GPT-6 Astra Pro (batch) | openai | 1050000 | 5.0 | 25.0 | closed / API |
| Qwen: Qwen3.8 Max (0902) | qwen | 1000000 | 2.0 | 6.0 | closed / API |
| Qwen: Qwen3.8 Max (0902) (batch) | qwen | 262144 | 1.15 | 3.45 | closed / API |
| Meta: Muse Spark 1.3 Contributor | meta | 1048576 | 0.09999999999999999 | 0.19999999999999998 | closed / API |
| Meta: Muse Spark 1.3 | meta | 1048576 | 1.25 | 4.25 | closed / API |
| Google: Gemini 3.8 Flash | 1048576 | 0.8250000000000001 | 4.125 | closed / API | |
| Google: Gemini 3.8 Flash (batch) | 1048576 | 0.375 | 1.875 | closed / API | |
| Anthropic: Claude Fable 5.1 | anthropic | 1000000 | 10.0 | 50.0 | closed / API |
| Anthropic: Claude Fable 5.1 (batch) | anthropic | 1000000 | 5.0 | 25.0 | closed / API |
| IBM: Granite 4.2 8B | ibm-granite | 131072 | 0.06 | 0.15 | open |
| Tencent: Hy4 preview | tencent | 1048576 | 0.7505999999999999 | 2.2509 | open |
| Z.ai: GLM Flash Latest | ~z-ai | 1048576 | 0.0399 | 0.0784 | closed / API |
| Qwen: Qwen3.8 Flash | qwen | 1000000 | 0.15 | 0.47 | open |
| Z.ai: GLM 5.3 Flash | z-ai | 1048576 | 0.0399 | 0.0784 | open |
| Z.ai: GLM 5.3 Flash (batch) | z-ai | 1048576 | 0.023 | 0.068 | open |
| Meta: Muse Spark 1.2 Contributor | meta | 1048576 | 0.09999999999999999 | 0.19999999999999998 | closed / API |
| DeepSeek: DeepSeek V4 Flash Vision Exp | deepseek | 1048576 | 0.21559999999999999 | 0.6468 | open |
| Tencent: Hy-MT2-1.8B | tencent | 8192 | 0.044 | 0.17700000000000002 | open |
| Tencent: Hy-MT2-30B-A3B | tencent | 8192 | 0.074 | 0.295 | open |
| Z.ai: GLM Latest | ~z-ai | 1048576 | 0.0699 | 1.4 | closed / API |
| Tencent: Hy-MT2-7B | tencent | 8192 | 0.074 | 0.295 | open |
| Z.ai: GLM 5.3 | z-ai | 1048576 | 0.0699 | 1.4 | open |
| Z.ai: GLM 5.3 (batch) | z-ai | 1048576 | 0.28 | 1.249 | open |
| Qwen: Qwen3.8 27B | qwen | 1024000 | 0.035 | 1.5 | open |
| Qwen: Qwen3.8 27B (batch) | qwen | 262144 | 0.16 | 2.0 | open |
| Google: Gemini 3.7 Flash | 1048576 | 0.75 | 3.75 | closed / API | |
| Google: Gemini 3.7 Flash (batch) | 1048576 | 0.375 | 1.875 | closed / API | |
| ByteDance Seed: Seed 2.1 Turbo | bytedance-seed | 262144 | 0.5 | 2.5 | closed / API |
| Qwen: Qwen3.8 2.4T A95B | qwen | 1048576 | 2.0 | 6.0 | open |
| DeepSeek: DeepSeek V4 Pro 0813 | deepseek | 1048576 | 1.32 | 3.9600000000000004 | open |
| SpaceXSI: Grok 4.6 | x-ai | 500000 | 2.0 | 6.0 | closed / API |
| SpaceXSI: Grok 4.6 (batch) | x-ai | 500000 | 1.5999999999999999 | 4.8 | closed / API |
| NVIDIA: Nemotron 3.5 Lightning | nvidia | 1048576 | 0.07 | 0.19999999999999998 | open |
| NVIDIA: Nemotron 3.5 Lightning (free) | nvidia | 1000000 | 0.0 | 0.0 | open |
| Sakana: Sakana Namazu | sakana | 262144 | 0.95 | 4.0 | closed / API |
| Upstage: Solar Pro 4 | upstage | 524288 | 0.15 | 0.6 | closed / API |
| Meta: Muse Glimmer 30B | meta | 131072 | 0.35 | 1.5 | open |
| Meta: Muse Glimmer 30B (batch) | meta | 131072 | 0.24 | 0.96 | open |
| inclusionAI: Ling 3.0 Tiny | inclusionai | 262144 | 0.015 | 0.045 | open |
| Meta: Muse Spark 1.2 | meta | 1048576 | 1.25 | 4.25 | closed / API |
| DeepSeek: DeepSeek V4 Flash Latest | ~deepseek | 1048576 | 0.044 | 0.13199999999999998 | closed / API |
| DeepSeek: DeepSeek V4 Flash 0731 | deepseek | 1048576 | 0.0443 | 0.3 | open |
| DeepSeek: DeepSeek V4 Flash 0731 (batch) | deepseek | 1048576 | 0.048 | 0.144 | open |
| Thinking Machines: Inkling Small | thinkingmachines | 524288 | 0.44999999999999996 | 1.2 | open |
| Thinking Machines: Inkling Small (batch) | thinkingmachines | 524288 | 0.36 | 0.96 | open |
| Qwen: Qwen3.7 Flash | qwen | 1000000 | 0.03 | 0.13 | closed / API |
| Anthropic: Claude Opus 5 | anthropic | 1000000 | 5.0 | 25.0 | closed / API |
| Anthropic: Claude Opus 5 (batch) | anthropic | 1000000 | 2.5 | 12.5 | closed / API |
| inclusionAI: Ling 3.0 Flash | inclusionai | 262144 | 0.020999999999999998 | 0.063 | open |
| Google: Gemini 3.6 Flash | 1048576 | 0.75 | 3.75 | closed / API | |
| Google: Gemini 3.6 Flash (batch) | 1048576 | 0.375 | 1.875 | closed / API | |
| Google: Gemini 3.5 Flash Lite | 1048576 | 0.3 | 2.5 | closed / API | |
| Google: Gemini 3.5 Flash Lite (batch) | 1048576 | 0.15 | 1.25 | closed / API | |
| Thinking Machines: Inkling | thinkingmachines | 524288 | 1.0 | 4.05 | open |
| MoonshotAI: Kimi K3 | moonshotai | 1048576 | 0.192 | 16.0 | open |
| MoonshotAI: Kimi K3 (batch) | moonshotai | 1048576 | 2.2800000000000002 | 11.399999999999999 | open |
| Meta: Muse Spark 1.1 | meta | 1048576 | 1.25 | 4.25 | closed / API |
| OpenAI: GPT-5.6 Luna Pro | openai | 1050000 | 0.19999999999999998 | 1.2 | closed / API |
| OpenAI: GPT-5.6 Luna Pro (batch) | openai | 1050000 | 0.09999999999999999 | 0.6 | closed / API |
| OpenAI: GPT-5.6 Luna | openai | 1050000 | 0.19999999999999998 | 1.2 | closed / API |
| OpenAI: GPT-5.6 Luna (batch) | openai | 1050000 | 0.09999999999999999 | 0.6 | closed / API |
| OpenAI: GPT-5.6 Terra Pro | openai | 1050000 | 2.0 | 12.0 | closed / API |
| OpenAI: GPT-5.6 Terra Pro (batch) | openai | 1050000 | 1.0 | 6.0 | closed / API |
| OpenAI: GPT-5.6 Terra | openai | 1050000 | 2.0 | 12.0 | closed / API |
| OpenAI: GPT-5.6 Terra (batch) | openai | 1050000 | 1.0 | 6.0 | closed / API |
| OpenAI: GPT-5.6 Sol Pro | openai | 1050000 | 4.0 | 20.0 | closed / API |
| OpenAI: GPT-5.6 Sol Pro (batch) | openai | 1050000 | 2.0 | 10.0 | closed / API |
| OpenAI: GPT-5.6 Sol | openai | 1050000 | 4.0 | 20.0 | closed / API |
| OpenAI: GPT-5.6 Sol (batch) | openai | 1050000 | 2.0 | 10.0 | closed / API |
| SpaceXSI: Grok 4.5 | x-ai | 500000 | 2.0 | 6.0 | closed / API |
| xAI: Grok Latest | ~x-ai | 500000 | 2.0 | 6.0 | closed / API |
| AionLabs: Aion-3.0-Mini | aion-labs | 131072 | 0.7 | 1.4 | closed / API |
| AionLabs: Aion-3.0 | aion-labs | 131072 | 3.0 | 6.0 | closed / API |
| Tencent: Hy3 | tencent | 262144 | 0.14 | 0.58 | open |
| Anthropic: Claude Sonnet 5 | anthropic | 1000000 | 2.0 | 10.0 | closed / API |
| Anthropic: Claude Sonnet 5 (batch) | anthropic | 1000000 | 1.0 | 5.0 | closed / API |
| Google: Nano Banana 2 Lite (Gemini 3.1 Flash Lite Image) | 65536 | 0.25 | 1.5 | closed / API | |
| Sakana: Fugu Ultra | sakana | 1000000 | 5.0 | 30.0 | closed / API |
| Google: Nano Banana 2 (Gemini 3.1 Flash Image) | 131072 | 0.5 | 3.0 | closed / API | |
| Google: Nano Banana Pro (Gemini 3 Pro Image) | 131072 | 2.0 | 12.0 | closed / API | |
| Z.ai: GLM 5.2 | z-ai | 1048576 | 0.62 | 2.42 | open |
| Z.ai: GLM 5.2 (batch) | z-ai | 1048576 | 0.322 | 0.989 | open |
| MoonshotAI: Kimi K2.7 Code | moonshotai | 262144 | 0.6 | 3.0 | open |
| Anthropic: Claude Fable Latest | ~anthropic | 1000000 | 8.0 | 40.0 | closed / API |
| Anthropic: Claude Fable 5 | anthropic | 1000000 | 10.0 | 50.0 | closed / API |
| Anthropic: Claude Fable 5 (batch) | anthropic | 1000000 | 5.0 | 25.0 | closed / API |
| NVIDIA: Nemotron 3.5 Content Safety | nvidia | 131072 | 0.19999999999999998 | 0.19999999999999998 | open |
| NVIDIA: Nemotron 3 Ultra | nvidia | 262144 | 0.5 | 2.2 | open |
| NVIDIA: Nemotron 3 Ultra (free) | nvidia | 1000000 | 0.0 | 0.0 | open |
| Qwen: Qwen3.7 Plus | qwen | 1000000 | 0.32 | 1.28 | closed / API |
| Qwen: Qwen3.7 Plus (batch) | qwen | 262144 | 0.22999999999999998 | 0.9199999999999999 | closed / API |
| MiniMax: MiniMax M3 | minimax | 1048576 | 0.3 | 1.2 | open |
| MiniMax: MiniMax M3 (batch) | minimax | 524288 | 0.224 | 0.88 | open |
| StepFun: Step 3.7 Flash | stepfun | 262144 | 0.19999999999999998 | 1.15 | open |
| Anthropic: Claude Opus 4.8 | anthropic | 1000000 | 5.0 | 25.0 | closed / API |
| Anthropic: Claude Opus 4.8 (batch) | anthropic | 1000000 | 2.5 | 12.5 | closed / API |
| Qwen: Qwen3.7 Max | qwen | 1000000 | 1.475 | 4.425 | closed / API |
| Google: Gemini 3.5 Flash | 1048576 | 1.5 | 9.0 | closed / API | |
| Google: Gemini 3.5 Flash (batch) | 1048576 | 0.75 | 4.5 | closed / API | |
| Perceptron: Perceptron Mk1 | perceptron | 32768 | 0.15 | 1.5 | closed / API |
| Google: Gemini 3.1 Flash Lite | 1048576 | 0.25 | 1.5 | closed / API | |
| Google: Gemini 3.1 Flash Lite (batch) | 1048576 | 0.125 | 0.75 | closed / API | |
| OpenAI: GPT Chat Latest | openai | 400000 | 5.0 | 30.0 | closed / API |
| SpaceXSI: Grok 4.3 | x-ai | 1000000 | 1.25 | 2.5 | closed / API |
| SpaceXSI: Grok 4.3 (batch) | x-ai | 1000000 | 1.0 | 2.0 | closed / API |
| Mistral: Mistral Medium 3.5 | mistralai | 262144 | 1.5 | 7.5 | open |
| Mistral: Mistral Medium 3.5 (batch) | mistralai | 262144 | 0.75 | 3.75 | open |
| NVIDIA: Nemotron 3 Nano Omni (free) | nvidia | 256000 | 0.0 | 0.0 | open |
| Anthropic: Claude Haiku Latest | ~anthropic | 1000000 | 0.09999999999999999 | 0.5 | closed / API |
| OpenAI: GPT Mini Latest | ~openai | 400000 | 0.75 | 4.5 | closed / API |
| Google: Gemini Pro Latest | 1048576 | 2.0 | 10.0 | closed / API | |
| MoonshotAI: Kimi Latest | ~moonshotai | 1048576 | 0.18239999999999998 | 11.0055 | closed / API |
| Google: Gemini Flash Latest | 1048576 | 1.5 | 7.5 | closed / API | |
| Anthropic: Claude Sonnet Latest | ~anthropic | 1000000 | 2.0 | 10.0 | closed / API |
| Qwen: Qwen3.5 Plus 2026-04-20 | qwen | 1000000 | 0.3 | 1.7999999999999998 | closed / API |
| Qwen: Qwen3.6 Flash | qwen | 1000000 | 0.1875 | 1.125 | closed / API |
| Qwen: Qwen3.6 35B A3B | qwen | 262144 | 0.09999999999999999 | 0.8999999999999999 | open |
| Qwen: Qwen3.6 27B | qwen | 262144 | 0.32 | 3.1999999999999997 | open |
| OpenAI: GPT-5.5 Pro | openai | 1050000 | 30.0 | 180.0 | closed / API |
| OpenAI: GPT-5.5 Pro (batch) | openai | 1050000 | 15.0 | 90.0 | closed / API |
| OpenAI: GPT-5.5 | openai | 1050000 | 5.0 | 30.0 | closed / API |
| OpenAI: GPT-5.5 (batch) | openai | 1050000 | 2.5 | 15.0 | closed / API |
| DeepSeek: DeepSeek V4 Pro 0423 | deepseek | 1048576 | 0.8999999999999999 | 1.5 | open |
| DeepSeek: DeepSeek V4 Flash 0423 | deepseek | 1048576 | 0.09 | 0.18 | open |
| Xiaomi: MiMo-V2.5-Pro | xiaomi | 262144 | 0.48 | 1.7999999999999998 | open |
| Xiaomi: MiMo-V2.5 | xiaomi | 1000000 | 0.39999999999999997 | 2.0 | open |
| OpenAI: GPT-5.4 Image 2 | openai | 272000 | 8.0 | 15.0 | closed / API |
| Anthropic: Claude Opus Latest | ~anthropic | 1000000 | 4.0 | 20.0 | closed / API |
| MoonshotAI: Kimi K2.6 | moonshotai | 262144 | 0.65 | 3.41 | open |
| Anthropic: Claude Opus 4.7 | anthropic | 1000000 | 5.0 | 25.0 | closed / API |
| Anthropic: Claude Opus 4.7 (batch) | anthropic | 1000000 | 2.5 | 12.5 | closed / API |
| Z.ai: GLM 5.1 | z-ai | 204800 | 1.4 | 4.4 | open |
| Google: Gemma 4 26B A4B | 262144 | 0.0675 | 0.22499999999999998 | open | |
| Google: Gemma 4 26B A4B (free) | 262144 | 0.0 | 0.0 | open | |
| Google: Gemma 4 31B | 262144 | 0.19999999999999998 | 0.39999999999999997 | open | |
| Google: Gemma 4 31B (free) | 262144 | 0.0 | 0.0 | open | |
| Qwen: Qwen3.6 Plus | qwen | 1000000 | 0.325 | 1.95 | closed / API |
| Z.ai: GLM 5V Turbo | z-ai | 202752 | 1.2 | 4.0 | closed / API |
| SpaceXSI: Grok 4.20 Multi-Agent | x-ai | 2000000 | 1.25 | 2.5 | closed / API |
| SpaceXSI: Grok 4.20 | x-ai | 2000000 | 1.25 | 2.5 | closed / API |
| Google: Lyria 3 Pro Preview | 1048576 | 0.0 | 0.0 | closed / API | |
| Google: Lyria 3 Clip Preview | 1048576 | 0.0 | 0.0 | closed / API | |
| Reka Edge | rekaai | 16384 | 0.09999999999999999 | 0.09999999999999999 | open |
| MiniMax: MiniMax M2.7 | minimax | 196608 | 0.6 | 1.7999999999999998 | open |
| OpenAI: GPT-5.4 Nano | openai | 400000 | 0.19999999999999998 | 1.25 | closed / API |
| OpenAI: GPT-5.4 Nano (batch) | openai | 400000 | 0.09999999999999999 | 0.625 | closed / API |
| OpenAI: GPT-5.4 Mini | openai | 400000 | 0.75 | 4.5 | closed / API |
| OpenAI: GPT-5.4 Mini (batch) | openai | 400000 | 0.375 | 2.25 | closed / API |
| Mistral: Mistral Small 4 | mistralai | 262144 | 0.15 | 0.6 | open |
| Z.ai: GLM 5 Turbo | z-ai | 202752 | 1.2 | 4.0 | closed / API |
| NVIDIA: Nemotron 3 Super | nvidia | 262144 | 0.075 | 0.39999999999999997 | open |
| ByteDance Seed: Seed-2.0-Lite | bytedance-seed | 262144 | 0.25 | 2.0 | closed / API |
| Qwen: Qwen3.5-9B | qwen | 262144 | 0.16999999999999998 | 0.25 | open |
| OpenAI: GPT-5.4 Pro | openai | 1050000 | 30.0 | 180.0 | closed / API |
| OpenAI: GPT-5.4 Pro (batch) | openai | 1050000 | 15.0 | 90.0 | closed / API |
| OpenAI: GPT-5.4 | openai | 1050000 | 2.5 | 15.0 | closed / API |
| OpenAI: GPT-5.4 (batch) | openai | 1050000 | 1.25 | 7.5 | closed / API |
| Google: Gemini 3.1 Flash Lite Preview | 1048576 | 0.25 | 1.5 | closed / API | |
| ByteDance Seed: Seed-2.0-Mini | bytedance-seed | 262144 | 0.09999999999999999 | 0.39999999999999997 | closed / API |
| Google: Nano Banana 2 (Gemini 3.1 Flash Image Preview) | 65536 | 0.5 | 3.0 | closed / API | |
| Qwen: Qwen3.5-35B-A3B | qwen | 262144 | 0.14 | 1.0 | open |
| Qwen: Qwen3.5-27B | qwen | 262144 | 0.26 | 2.6 | open |
| Qwen: Qwen3.5-122B-A10B | qwen | 262144 | 0.26 | 2.08 | open |
| Qwen: Qwen3.5-Flash | qwen | 1000000 | 0.065 | 0.26 | closed / API |
| Google: Gemini 3.1 Pro Preview Custom Tools | 1048576 | 2.0 | 12.0 | closed / API | |
| OpenAI: GPT-5.3-Codex | openai | 400000 | 1.75 | 14.0 | closed / API |
| AionLabs: Aion-2.0 | aion-labs | 131072 | 0.7999999999999999 | 1.5999999999999999 | closed / API |
| Google: Gemini 3.1 Pro Preview | 1048576 | 2.0 | 12.0 | closed / API | |
| Google: Gemini 3.1 Pro Preview (batch) | 1048576 | 1.0 | 6.0 | closed / API | |
| Anthropic: Claude Sonnet 4.6 | anthropic | 1000000 | 3.0 | 15.0 | closed / API |
| Anthropic: Claude Sonnet 4.6 (batch) | anthropic | 1000000 | 1.5 | 7.5 | closed / API |
| Qwen: Qwen3.5 Plus 2026-02-15 | qwen | 1000000 | 0.26 | 1.56 | closed / API |
| Qwen: Qwen3.5 397B A17B | qwen | 262144 | 0.44999999999999996 | 3.0 | open |
| MiniMax: MiniMax M2.5 | minimax | 198000 | 0.27 | 0.95 | open |
| Z.ai: GLM 5 | z-ai | 204800 | 1.0 | 3.1999999999999997 | open |
| Anthropic: Claude Opus 4.6 | anthropic | 1000000 | 5.0 | 25.0 | closed / API |
| Anthropic: Claude Opus 4.6 (batch) | anthropic | 1000000 | 2.5 | 12.5 | closed / API |
| Qwen: Qwen3 Coder Next | qwen | 262144 | 0.12 | 0.7999999999999999 | open |
| MoonshotAI: Kimi K2.5 | moonshotai | 262144 | 0.5 | 2.5 | open |
| Writer: Palmyra X5 | writer | 1040000 | 0.6 | 6.0 | closed / API |
| OpenAI: GPT Audio | openai | 128000 | 2.5 | 10.0 | closed / API |
| OpenAI: GPT Audio Mini | openai | 128000 | 0.6 | 2.4 | closed / API |
| Z.ai: GLM 4.7 Flash | z-ai | 200000 | 0.060500000000000005 | 0.39999999999999997 | open |
| OpenAI: GPT-5.2-Codex | openai | 400000 | 1.75 | 14.0 | closed / API |
| Z.ai: GLM 4.7 | z-ai | 204800 | 0.6 | 2.2 | open |
| Google: Gemini 3 Flash Preview | 1048576 | 0.5 | 3.0 | closed / API | |
| Google: Gemini 3 Flash Preview (batch) | 1048576 | 0.25 | 1.5 | closed / API | |
| NVIDIA: Nemotron 3 Nano 30B A3B | nvidia | 262144 | 0.06 | 0.24 | open |
| OpenAI: GPT-5.2 Chat | openai | 128000 | 1.75 | 14.0 | closed / API |
| OpenAI: GPT-5.2 Pro | openai | 400000 | 21.0 | 168.0 | closed / API |
| OpenAI: GPT-5.2 Pro (batch) | openai | 400000 | 10.5 | 84.0 | closed / API |
| OpenAI: GPT-5.2 | openai | 400000 | 1.75 | 14.0 | closed / API |
| OpenAI: GPT-5.2 (batch) | openai | 400000 | 0.875 | 7.0 | closed / API |
| Mistral: Devstral 2 2512 | mistralai | 262144 | 0.39999999999999997 | 2.0 | open |
| Relace: Relace Search | relace | 256000 | 1.0 | 3.0 | closed / API |
| Z.ai: GLM 4.6V | z-ai | 131072 | 0.3 | 0.8999999999999999 | open |
| OpenAI: GPT-5.1-Codex-Max | openai | 400000 | 1.25 | 10.0 | closed / API |
| Amazon: Nova 2 Lite | amazon | 1000000 | 0.3 | 2.5 | closed / API |
| Mistral: Ministral 3 14B 2512 | mistralai | 262144 | 0.19999999999999998 | 0.19999999999999998 | open |
| Mistral: Ministral 3 8B 2512 | mistralai | 262144 | 0.15 | 0.15 | open |
| Mistral: Ministral 3 3B 2512 | mistralai | 131072 | 0.09999999999999999 | 0.09999999999999999 | open |
| Mistral: Mistral Large 3 2512 | mistralai | 262144 | 0.5 | 1.5 | open |
| DeepSeek: DeepSeek V3.2 | deepseek | 163840 | 0.26 | 0.38 | open |
| Anthropic: Claude Opus 4.5 | anthropic | 200000 | 5.0 | 25.0 | closed / API |
| Anthropic: Claude Opus 4.5 (batch) | anthropic | 200000 | 2.5 | 12.5 | closed / API |
| Google: Nano Banana Pro (Gemini 3 Pro Image Preview) | 65536 | 2.0 | 12.0 | closed / API | |
| OpenAI: GPT-5.1 | openai | 400000 | 1.25 | 10.0 | closed / API |
| OpenAI: GPT-5.1 (batch) | openai | 400000 | 0.625 | 5.0 | closed / API |
| OpenAI: GPT-5.1-Codex | openai | 400000 | 1.25 | 10.0 | closed / API |
| OpenAI: GPT-5.1-Codex-Mini | openai | 400000 | 0.25 | 2.0 | closed / API |
| MoonshotAI: Kimi K2 Thinking | moonshotai | 262144 | 0.6 | 2.5 | open |
| Amazon: Nova Premier 1.0 | amazon | 1000000 | 2.5 | 12.5 | closed / API |
| Perplexity: Sonar Pro Search | perplexity | 200000 | 3.0 | 15.0 | closed / API |
| Mistral: Voxtral Small 24B 2507 | mistralai | 32768 | 0.09999999999999999 | 0.3 | open |
| OpenAI: gpt-oss-safeguard-20b | openai | 131072 | 0.075 | 0.3 | open |
| OpenAI: GPT-5 Image Mini | openai | 400000 | 2.5 | 2.0 | closed / API |
| Anthropic: Claude Haiku 4.5 | anthropic | 200000 | 1.0 | 5.0 | closed / API |
| Anthropic: Claude Haiku 4.5 (batch) | anthropic | 200000 | 0.5 | 2.5 | closed / API |
| Qwen: Qwen3 VL 8B Instruct | qwen | 262144 | 0.25 | 0.75 | open |
| OpenAI: GPT-5 Image | openai | 400000 | 10.0 | 10.0 | closed / API |
| Google: Nano Banana (Gemini 2.5 Flash Image) | 32768 | 0.3 | 2.5 | closed / API | |
| Qwen: Qwen3 VL 30B A3B Instruct | qwen | 262144 | 0.15 | 0.6 | open |
| OpenAI: GPT-5 Pro | openai | 400000 | 15.0 | 120.0 | closed / API |
| OpenAI: GPT-5 Pro (batch) | openai | 400000 | 7.5 | 60.0 | closed / API |
| Z.ai: GLM 4.6 | z-ai | 204800 | 0.5 | 2.0 | open |
| Anthropic: Claude Sonnet 4.5 | anthropic | 1000000 | 3.0 | 15.0 | closed / API |
| Anthropic: Claude Sonnet 4.5 (batch) | anthropic | 1000000 | 1.5 | 7.5 | closed / API |
| DeepSeek: DeepSeek V3.2 Exp | deepseek | 163840 | 0.27 | 0.41 | open |
| TheDrummer: Cydonia 24B V4.1 | thedrummer | 131072 | 0.3 | 0.5 | open |
| Relace: Relace Apply 3 | relace | 256000 | 0.85 | 1.25 | closed / API |
| Qwen: Qwen3 VL 235B A22B Instruct | qwen | 262144 | 0.21 | 1.9 | open |
| DeepSeek: DeepSeek V3.1 Terminus | deepseek | 163840 | 0.27 | 1.0 | open |
| Qwen: Qwen3 Coder Flash | qwen | 1000000 | 0.195 | 0.975 | closed / API |
| Qwen: Qwen3 Next 80B A3B Instruct | qwen | 262144 | 0.09999999999999999 | 1.1 | open |
| MoonshotAI: Kimi K2 0905 | moonshotai | 262144 | 0.6 | 2.5 | open |
| Nous: Hermes 4 405B | nousresearch | 131072 | 1.0 | 3.0 | open |
| DeepSeek: DeepSeek V3.1 | deepseek | 163840 | 0.25 | 0.95 | open |
| Mistral: Mistral Medium 3.1 | mistralai | 131072 | 0.44 | 2.2 | closed / API |
| Z.ai: GLM 4.5V | z-ai | 65536 | 0.6 | 1.7999999999999998 | open |
| OpenAI: GPT-5 | openai | 400000 | 1.25 | 10.0 | closed / API |
| OpenAI: GPT-5 (batch) | openai | 400000 | 0.625 | 5.0 | closed / API |
| OpenAI: GPT-5 Mini | openai | 400000 | 0.25 | 2.0 | closed / API |
| OpenAI: GPT-5 Mini (batch) | openai | 400000 | 0.125 | 1.0 | closed / API |
| OpenAI: GPT-5 Nano | openai | 400000 | 0.049999999999999996 | 0.39999999999999997 | closed / API |
| OpenAI: GPT-5 Nano (batch) | openai | 400000 | 0.024999999999999998 | 0.19999999999999998 | closed / API |
| OpenAI: gpt-oss-120b | openai | 131072 | 0.09 | 0.36 | open |
| OpenAI: gpt-oss-120b (batch) | openai | 131072 | 0.0296 | 0.136 | open |
| OpenAI: gpt-oss-20b | openai | 131072 | 0.075 | 0.3 | open |
| OpenAI: gpt-oss-20b (batch) | openai | 131072 | 0.024 | 0.112 | open |
| Anthropic: Claude Opus 4.1 | anthropic | 200000 | 15.0 | 75.0 | closed / API |
| Anthropic: Claude Opus 4.1 (batch) | anthropic | 200000 | 7.5 | 37.5 | closed / API |
| Mistral: Codestral 2508 | mistralai | 256000 | 0.3 | 0.8999999999999999 | closed / API |
| Qwen: Qwen3 Coder 30B A3B Instruct | qwen | 262144 | 0.07 | 0.28 | open |
| Qwen: Qwen3 30B A3B Instruct 2507 | qwen | 262144 | 0.09999999999999999 | 0.3 | open |
| Z.ai: GLM 4.5 | z-ai | 131072 | 0.6 | 2.2 | open |
| Z.ai: GLM 4.5 Air | z-ai | 131072 | 0.13 | 0.85 | open |
| Qwen: Qwen3 235B A22B Thinking 2507 | qwen | 128000 | 0.44999999999999996 | 3.5 | open |
| Qwen: Qwen3 Coder 480B A35B | qwen | 262144 | 0.3 | 1.0 | open |
| ByteDance: UI-TARS 7B | bytedance | 128000 | 0.09999999999999999 | 0.19999999999999998 | open |
| Google: Gemini 2.5 Flash Lite | 1048576 | 0.09999999999999999 | 0.39999999999999997 | closed / API | |
| Qwen: Qwen3 235B A22B Instruct 2507 | qwen | 262144 | 0.14 | 0.7999999999999999 | open |
| MoonshotAI: Kimi K2 0711 | moonshotai | 131072 | 0.5700000000000001 | 2.3 | open |
| Venice: Uncensored | cognitivecomputations | 128000 | 0.19999999999999998 | 0.8999999999999999 | open |
| Morph: Morph V3 Large | morph | 262144 | 0.8999999999999999 | 1.9 | closed / API |
| Morph: Morph V3 Large (batch) | morph | 262144 | 0.44999999999999996 | 0.95 | closed / API |
| Morph: Morph V3 Fast | morph | 262144 | 0.7999999999999999 | 1.2 | closed / API |
| Morph: Morph V3 Fast (batch) | morph | 81920 | 0.39999999999999997 | 0.6 | closed / API |
| Mistral: Mistral Small 3.2 24B | mistralai | 256000 | 0.075 | 0.19999999999999998 | open |
| Google: Gemini 2.5 Flash | 1048576 | 0.3 | 2.5 | closed / API | |
| Google: Gemini 2.5 Pro | 1048576 | 1.25 | 10.0 | closed / API | |
| OpenAI: o3 Pro | openai | 200000 | 20.0 | 80.0 | closed / API |
| Google: Gemini 2.5 Pro Preview 06-05 | 1048576 | 1.25 | 10.0 | closed / API | |
| DeepSeek: R1 0528 | deepseek | 163840 | 0.5710000000000001 | 2.2859999999999996 | open |
| Mistral: Mistral Medium 3 | mistralai | 131072 | 0.44 | 2.2 | closed / API |
| Meta: Llama Guard 4 12B | meta-llama | 163840 | 0.18 | 0.18 | open |
| Qwen: Qwen3 30B A3B | qwen | 40960 | 0.12 | 0.5 | open |
| Qwen: Qwen3 14B | qwen | 40960 | 0.12 | 0.24 | open |
| Qwen: Qwen3 32B | qwen | 131072 | 0.08 | 0.28 | open |
| OpenAI: o3 | openai | 200000 | 2.0 | 8.0 | closed / API |
| OpenAI: o3 (batch) | openai | 200000 | 1.0 | 4.0 | closed / API |
| OpenAI: o4 Mini (batch) | openai | 200000 | 0.55 | 2.2 | closed / API |
| OpenAI: GPT-4.1 | openai | 1047576 | 2.0 | 8.0 | closed / API |
| OpenAI: GPT-4.1 (batch) | openai | 1047576 | 1.0 | 4.0 | closed / API |
| OpenAI: GPT-4.1 Mini | openai | 1047576 | 0.39999999999999997 | 1.5999999999999999 | closed / API |
| OpenAI: GPT-4.1 Mini (batch) | openai | 1047576 | 0.19999999999999998 | 0.7999999999999999 | closed / API |
| OpenAI: GPT-4.1 Nano | openai | 1047576 | 0.09999999999999999 | 0.39999999999999997 | closed / API |
| OpenAI: GPT-4.1 Nano (batch) | openai | 1047576 | 0.049999999999999996 | 0.19999999999999998 | closed / API |
| Meta: Llama 4 Maverick | meta-llama | 524288 | 0.35 | 1.0 | open |
| Meta: Llama 4 Scout | meta-llama | 1310720 | 0.09999999999999999 | 0.3 | open |
| DeepSeek: DeepSeek V3 0324 | deepseek | 163840 | 0.25 | 1.0 | open |
| Mistral: Mistral Small 3.1 24B | mistralai | 128000 | 0.351 | 0.5549999999999999 | open |
| Google: Gemma 3 4B | 131072 | 0.04 | 0.049999999999999996 | open | |
| Google: Gemma 3 12B | 131072 | 0.049999999999999996 | 0.15 | open | |
| Cohere: Command A | cohere | 256000 | 2.5 | 10.0 | open |
| Reka Flash 3 | rekaai | 65536 | 0.09999999999999999 | 0.19999999999999998 | open |
| Google: Gemma 3 27B | 131072 | 0.08 | 0.44999999999999996 | open | |
| TheDrummer: Skyfall 36B V2 | thedrummer | 32768 | 0.55 | 0.7999999999999999 | open |
| Perplexity: Sonar Reasoning Pro | perplexity | 128000 | 2.0 | 8.0 | closed / API |
| Perplexity: Sonar Pro | perplexity | 200000 | 3.0 | 15.0 | closed / API |
| Perplexity: Sonar Deep Research | perplexity | 128000 | 2.0 | 8.0 | closed / API |
| Mistral: Saba | mistralai | 32768 | 0.19999999999999998 | 0.6 | closed / API |
| AionLabs: Aion-RP 1.0 (8B) | aion-labs | 32768 | 0.7999999999999999 | 1.5999999999999999 | open |
| Qwen: Qwen2.5 VL 72B Instruct | qwen | 128000 | 0.7999999999999999 | 1.0 | open |
| Qwen: Qwen-Plus | qwen | 1000000 | 0.26 | 0.78 | closed / API |
| OpenAI: o3 Mini (batch) | openai | 200000 | 0.55 | 2.2 | closed / API |
| Mistral: Mistral Small 3 | mistralai | 32768 | 0.049999999999999996 | 0.08 | open |
| Perplexity: Sonar | perplexity | 127072 | 1.0 | 1.0 | closed / API |
| Microsoft: Phi 4 | microsoft | 16384 | 0.07 | 0.14 | open |
| DeepSeek: DeepSeek V3 | deepseek | 163840 | 0.2574 | 1.0287 | open |
| Cohere: Command R7B (12-2024) | cohere | 128000 | 0.0375 | 0.15 | open |
| Meta: Llama 3.3 70B Instruct | meta-llama | 131072 | 0.71 | 0.71 | open |
| Amazon: Nova Lite 1.0 | amazon | 300000 | 0.06 | 0.24 | closed / API |
| Amazon: Nova Micro 1.0 | amazon | 128000 | 0.035 | 0.14 | closed / API |
| Amazon: Nova Pro 1.0 | amazon | 300000 | 0.7999999999999999 | 3.1999999999999997 | closed / API |
| OpenAI: GPT-4o (2024-11-20) | openai | 128000 | 2.5 | 10.0 | closed / API |
| Mistral Large 2407 | mistralai | 131072 | 2.0 | 6.0 | open |
| Qwen2.5 Coder 32B Instruct | qwen | 32768 | 0.66 | 1.0 | open |
| TheDrummer: UnslopNemo 12B | thedrummer | 1024000 | 0.39999999999999997 | 0.39999999999999997 | open |
| Magnum v4 72B | anthracite-org | 32768 | 2.5 | 5.0 | open |
| Meta: Llama 3.2 3B Instruct | meta-llama | 131072 | 0.049999999999999996 | 0.33 | open |
| Qwen2.5 72B Instruct | qwen | 32768 | 0.36 | 0.39999999999999997 | open |
| Cohere: Command R (08-2024) | cohere | 128000 | 0.15 | 0.6 | open |
| Cohere: Command R+ (08-2024) | cohere | 128000 | 2.5 | 10.0 | open |
| Sao10K: Llama 3.1 Euryale 70B v2.2 | sao10k | 131072 | 0.85 | 0.85 | open |
| Sao10K: Llama 3 8B Lunaris | sao10k | 8192 | 0.04 | 0.049999999999999996 | open |
| OpenAI: GPT-4o (2024-08-06) | openai | 128000 | 2.5 | 10.0 | closed / API |
| Meta: Llama 3.1 70B Instruct | meta-llama | 131072 | 0.72 | 0.72 | open |
| Meta: Llama 3.1 8B Instruct | meta-llama | 131072 | 0.02 | 0.04 | open |
| Mistral: Mistral Nemo | mistralai | 131072 | 0.029 | 0.03 | open |
| OpenAI: GPT-4o-mini | openai | 128000 | 0.15 | 0.6 | closed / API |
| OpenAI: GPT-4o-mini (2024-07-18) | openai | 128000 | 0.15 | 0.6 | closed / API |
| OpenAI: GPT-4o-mini (batch) | openai | 128000 | 0.075 | 0.3 | closed / API |
| Google: Gemma 2 27B | 8192 | 0.65 | 0.65 | open | |
| OpenAI: GPT-4o | openai | 128000 | 2.5 | 10.0 | closed / API |
| OpenAI: GPT-4o (2024-05-13) | openai | 128000 | 5.0 | 15.0 | closed / API |
| OpenAI: GPT-4o (batch) | openai | 128000 | 1.25 | 5.0 | closed / API |
| Mistral: Mixtral 8x22B Instruct | mistralai | 65536 | 2.0 | 6.0 | open |
| OpenAI: GPT-4 Turbo (batch) | openai | 128000 | 5.0 | 15.0 | closed / API |
| Mistral Large | mistralai | 128000 | 2.0 | 6.0 | closed / API |
| OpenAI: GPT-3.5 Turbo (older v0613) | openai | 4095 | 1.0 | 2.0 | closed / API |
| OpenAI: GPT-3.5 Turbo 16k | openai | 16385 | 3.0 | 4.0 | closed / API |
| Mancer: Weaver (alpha) | mancer | 8000 | 0.3 | 0.5 | closed / API |
| ReMM SLERP 13B | undi95 | 6144 | 0.3 | 0.5 | open |
| MythoMax 13B | gryphe | 8192 | 0.3 | 0.5 | open |
| OpenAI: GPT-4 | openai | 8191 | 30.0 | 60.0 | closed / API |
| Auto Router | openrouter | 2000000 | -1000000.0 | -1000000.0 | closed / API |
| Auto Code | openrouter | 2000000 | -1000000.0 | -1000000.0 | closed / API |
| TypeSafe: Jev Router | typesafe | 1000000 | -1000000.0 | -1000000.0 | closed / API |
| NVIDIA: Switchyard | nvidia | 1000000 | -1000000.0 | -1000000.0 | closed / API |
| Auto Router (Beta) | openrouter | 2000000 | -1000000.0 | -1000000.0 | closed / API |
| OpenRouter: Fusion | openrouter | 1000000 | -1000000.0 | -1000000.0 | closed / API |
| Pareto Code Router | openrouter | 2000000 | -1000000.0 | -1000000.0 | closed / API |
| Free Models Router | openrouter | 200000 | 0.0 | 0.0 | closed / API |
| Body Builder (beta) | openrouter | 128000 | -1000000.0 | -1000000.0 | closed / API |
Speed
Unknown. OpenRouter publishes per-provider throughput_last_30m (tokens/sec) and latency_last_30m (time to first token) on /api/v1/models/{author}/{slug}/endpoints, but their docs say those fields are visible only when authenticated with an API key or cookie. An unauthenticated GET to openai/gpt-4o/endpoints on 2026-11-22T12:28:32-05:00 returned HTTP 200 with both fields null for every listed provider. We did not use a key, and we will not invent tokens/sec or latency.
Artificial Analysis
Unknown. Their free API (/api/v2/language/models/free) requires an API key: an unauthenticated GET on 2026-11-22T12:28:32-05:00 returned HTTP 401 (“API key is required”). Their data-platform terms also limit free-tier redistribution to internal use with attribution, which does not cover republishing a public rankings table. See the live Artificial Analysis leaderboard.
ABUZ8 Arena v1 (our own measured set)
120 contamination-checked agent tool tasks (pick the right tool and arguments), scored exact-match, greedy decoding, on one RTX 5090. Date on the snapshot: 2026-08-19.
data/bench.json.| System | Kind | Score |
|---|---|---|
| Muse Glimmer 30B (local) | model | 98.3 |
| 12-line regex baseline | baseline | 85.8 |
| Jabarr / MUTHALLATH-1 m0 (0.6B + triangle organs) | experiment | 77.5 |