AI Model Release Tracker

AI Benchmarks

Every benchmark we track across the major AI labs. Click any benchmark to see the full ranking of models, from the highest score to the lowest.

Most benchmark wins

Only benchmarks still in use count: not retired, and with a new score in the past year.

  1. #1
    Claude Opus 5.5
    CheatBench, ProgramBench, CursorBench, FrontierCode, Next.js, OSWorld 2.0, GDPval v2.1, Briefcase 1.1, threejseval
    9
  2. 2
    GPT-6 Astra
    Auto-review, AA Coding, BenchCAD, ExploitBench, ARC-AGI-3, ARC-AGI-2, GPQA, AA Index
    8
  3. 3
    Claude Opus 5
    Gray Swan, SWE-Ver, SWE-Multi, SWE-MM, Supabase, Frontier v0.1, GDPval v2
    7
  4. 4
    Kimi K3.1 Pro
    APEX-Agents, HLE, GDPval v2.1, CharXiv, Chartography, OfficeQA
    6
  5. 5
    Gemini 4 Carbon
    FrontierSWE V2, TB-Science, OSWorld 2.1, Blueprint
    4
  6. 5
    Gemini 4 Argon
    Next.js, LAB-Bench 2, Finance, LVBench
    4
  7. 5
    Grok 4.7
    CADGenBench, CathedralBench, LatchBio, EEBench
    4
  8. 8
    GPT-6.1 Terra
    Next.js, Agent's Last Exam, MMMU-Pro
    3
  9. 8
    DeepSeek-V4.1-Flash-1017
    NL2Repo, Terminal 2.1, AutoBench
    3
  10. 8
    Muse Wonder Full
    DeepSWE 1.1, SWE-Marathon, JobBench
    3

Coding

ProgramBench
fully rebuilding programs from their executable and documentation· 25 models
SWE-Bench Pro
long-horizon, real-world software engineering· 24 models
SWE-Bench Verified
real-world software engineering· 70 models
SWE-Bench Multilingual
software engineering across many languages· 19 models
SWE-Bench Multimodal
software engineering on issues that carry screenshots and visual context· 3 models
CursorBench 4.0
Cursor agentic coding eval on ambiguous, multi-file tasks from real sessions (4.0 task set)· 18 models
DeepSWE 1.1
Artificial Analysis agentic software-engineering eval (v1.1)· 37 models
DeepSWE 1.0
Artificial Analysis agentic software-engineering eval· 6 models
SWE-sweep
subtask-weighted reward for software engineering with the mini-swe-agent harness· 1 model
FrontierSWE V2
34 ultra-long-horizon engineering challenges, scored mean@5 on partial credit· 4 models
SWE-Marathon v1.1
autonomous completion of 20 multi-hour software engineering tasks, all verifiers passing· 5 models
FrontierCode v1.1 (Main)
frontier-difficulty agentic coding tasks (v1.1, main split)· 8 models
APEX-SWE
expert-level software-engineering tasks (AI Productivity Index)· 2 models
MLE-Bench
machine-learning engineering on real Kaggle competitions· 3 models
PaperBench
reproducing the results of an ML research paper end to end· 1 model
Next.js Evals
Next.js code generation & migration tasks· 35 models
Supabase Evals
building and debugging real Supabase projects as a coding agent· 11 models
NL2Repo-Bench
generating repository-level code from natural-language requirements· 7 models
SWEAtlas CodeBase QnA
answering questions about an unfamiliar codebase· 2 models
QwenSWEBench
Qwen's in-house benchmark for software-engineering capabilities· 1 model
QwenSWEBench V2
Qwen's in-house benchmark for complex real-world software engineering (v2)· 2 models
AA Coding Agent Index
Artificial Analysis composite of three coding-agent evals, scored per harness-and-model pairing· 2 models
Database Migration Tasks (OpenAI Internal)
OpenAI's internal set of database migration tasks· 1 model
CADGenBench
building CAD geometry from a design prompt, graded for executability and geometric correctness· 2 models
BenchCAD
writing and editing parametric CAD programs for industrial parts· 2 models
LiveCodeBench
contamination-free code generation on freshly published problems· 24 models
HumanEval
writing a single Python function from its docstring — the standard coding benchmark of 2021–2024, retired in favour of SWE-Bench-style tests on real repositories· 6 models
Expert-SWE (Internal)
OpenAI's internal software-engineering eval· 2 models

Terminal & CLI

Frontier-Bench v0.1
diverse, difficult agentic computer-work tasks (successor to Terminal-Bench)· 9 models
Terminal-Bench 4.0
command-line task completion (v4.0, recalibrated task resources)· 32 models
Terminal-Bench 3.0
command-line task completion (v3.0, much harder task set)· 7 models
Terminal-Bench 2.1
command-line task completion· 36 models
Terminal-Bench 2.0
command-line task completion (v2.0)· 17 models
Terminal-Bench-Science 0.1
scientific computing tasks at the command line (v0.1)· 10 models

Agentic & tool use

APEX-Agents
expert-level agentic work tasks (AI Productivity Index)· 3 models
MCP Atlas
multi-step workflows using MCP· 13 models
JobBench
professional workplace tool-use tasks· 11 models
CoWorkBench
long-horizon productivity tasks across professional domains· 4 models
Toolathlon-Verified
verified real-world personal tool use· 9 models
Toolathlon
real-world general tool use· 5 models
BU Bench
browser agent task completion (Browser Use)· 7 models
BrowseComp
agentic web browsing and research· 33 models
DeepSearchQA
multi-step web search and answer synthesis· 1 model
CyberGym
cybersecurity agentic tasks· 13 models
CVE-Bench
exploiting real-world web-application CVEs in sandboxed production-like services· 3 models
CathedralBench
multi-exploit chain execution on the hard subset, run by a third-party red team· 2 models
ExploitBench
exploit development on known V8 vulnerabilities (CMU capability-ladder eval)· 3 models
ExploitGym
end-to-end exploits built from known bugs (count of 898 instances solved)· 1 model
Agents’ Last Exam v1.1
averaged rubric score on the 122-task October 17, 2026 v1.1 set, using ALE-Claw with a five-hour limit; separate from earlier ALE scores· 1 model
CWE-bench v1
Collinear AI’s programmatically graded security-patch pass@1 rate; separate from exploit-generation benchmarks· 1 model
ExploitGym (causal pass rate)
share of tasks producing a verified exploit of the intended vulnerability, with mitigations off; not a count of solved instances· 1 model
ExploitGym (remote target)
intended-exploit success rate on a specified vulnerability in a remote target; distinct from the timed pass-rate and solved-instance boards· 1 model
ExploitGym Hardened
intended-exploit success rate with stronger mitigations, including address-space randomization, stack canaries, the V8 sandbox, and stricter kernel configurations· 1 model
RemoteExploitBench
completion rate on known vulnerabilities in remotely deployed applications such as GitLab, PostgreSQL, and Jenkins, in a controlled environment· 1 model
ExploitGym Honeypot
share of trials that avoided exploiting an out-of-scope honeypot service, without deployment safety prompts· 1 model
SEC-bench Pro (JavaScript)
pass@1 on the 183-task May 2026 JavaScript-engine split, averaged over three trials with the Antigravity harness· 1 model
OSWorld 2.0
operating a real computer to finish tasks (v2.0)· 10 models
OSWorld 2.1 (offline)
operating a real computer to finish tasks (v2.1, offline)· 4 models
OSWorld-Verified
operating a real computer to finish tasks· 29 models
Agent's Last Exam
pass rate on multimodal desktop and OS agent tasks· 11 models
SRE-Bench
Kubernetes incident response and reliability work, best of four attempts· 1 model
AutomationBench
end-to-end business workflow automation· 20 models

Reasoning & science

RiemannBench
mathematical problem-solving evaluation published by Surge· 1 model
Humanity's Last Exam
expert-level questions across disciplines· 54 models
Humanity's Last Exam (Verified)
expert-level questions across disciplines, on the verified question set· 2 models
ARC-AGI-3
novel problem-solving in interactive environments· 2 models
ARC-AGI-2
abstract reasoning puzzles· 41 models
FrontierMath
research-level mathematics· 6 models
BioMysteryBench
solving open biological research mysteries· 3 models
LatchBio Capabilities v1.0
agentic analysis of real biological data, averaged over 11 capability benchmarks· 2 models
GeneBench-Pro
multistage statistical reasoning in genomics and translational biomedicine· 1 model
MedChemBench (Internal)
OpenAI's medicinal-chemistry eval: structure-activity reasoning, ADME and lead optimisation· 1 model
LAB-Bench 2
real-world biology research tasks· 3 models
EEBench
designing circuits and hardware, graded for physical correctness and functionality· 2 models
GPQA Diamond
graduate-level science reasoning· 67 models
IFBench
following detailed instructions and constraints· 3 models
Agentic IF Index (Internal)
Meta's internal instruction-following eval for agentic runs· 1 model
MMLU
multiple-choice exam questions across 57 academic subjects· 7 models
GSM8K
grade-school maths word problems needing a few steps of arithmetic reasoning· 5 models

Knowledge work

AA Intelligence Index v5.0
Artificial Analysis composite intelligence index, version 5.0 with tools; separate from earlier index editions· 1 model
Vals Index v2.2
Vals AI composite evaluation of professional tasks, version 2.2· 1 model
Vending-Bench 2 (USD)
ending bank balance in US dollars in Andon Labs’ vending-business simulation; higher is better· 1 model
AA Intelligence Index
Artificial Analysis composite intelligence index across evals· 6 models
GDPval-AA
economically valuable knowledge work· 9 models
GDPval-AA v2.1
economically valuable knowledge work (v2.1, Crowd-BT Elo fit)· 8 models
GDPval-AA v2
economically valuable knowledge work (v2, re-based Elo)· 21 models
Design Tasks (OpenAI Internal)
OpenAI's internal set of professional design tasks· 1 model
Data Science Tasks (OpenAI Internal)
OpenAI's internal set of professional data-science tasks· 1 model
AA-Briefcase v1.1
Artificial Analysis agentic office-work eval (Elo, v1.1 rating fit)· 7 models
AA-Briefcase
Artificial Analysis agentic office-work eval (Elo)· 4 models
GDPval (win/tie rate)
GDPval win-or-tie rate vs human experts· 6 models

Finance

Finance Agent v2
financial analysis tasks· 12 models

Healthcare

HealthBench Professional
realistic health conversations graded against physician rubrics· 7 models
MedScribe
supporting doctors with administrative work (Vals AI proprietary set)· 5 models

Multimodal

CharXiv Reasoning
information synthesis from complex charts· 17 models
Chartography
chart-centred tasks, run with tools· 7 models
OfficeQA Pro
question answering over office documents· 2 models
MVBench
temporal reasoning over video clips· 1 model
MMVU
expert-level reasoning over discipline-specific video· 1 model
GDP.PDF
expert comprehension of PDF documents· 6 models
LVBench
understanding hour-long videos· 4 models
BabyVision
core visual reasoning· 4 models
OpenScore String Quartets
transcribing string-quartet sheet music, scored as 1 - OMR normalised edit distance· 1 model
MMMU-Pro
multimodal understanding and reasoning· 15 models
MMMU
multimodal understanding· 9 models
Blueprint-Bench 2
agentic spatial reasoning· 7 models

Long context

GraphWalks BFS (256K–1M)
breadth-first search F1 on the 200 GraphWalks problems with 256K to 1M tokens of context· 1 model
MRCR v2 (8-needle)
long-context retrieval· 7 models
MRCR
long-context retrieval· 1 model

Robustness

CheatBench
reward gaming by AI agents across ten categories of challenging assignments· 13 models
BullshitBench v2
detecting and pushing back on nonsense prompts· 98 models
Gray Swan IPI
indirect prompt injection attack success rate· 13 models
Auto-review circumvention (Internal)
OpenAI's internal measure of how often a model works around its own automated safety review· 2 models
OpenAI Factuality (Internal)
answers containing any factual error on OpenAI's internal error-inducing prompts· 1 model

Community preference

threejseval
blind human votes on the Three.js scenes coding agents build from one prompt· 29 models

Scores are published by the labs at launch, or gathered from the leaderboards that measure them — BenchLM, BullshitBench, CheatBench, CursorBench, ProgramBench, Supabase Evals, Next.js Evals and threejseval. Where the numbers come from.