AI Benchmarks

long-horizon, real-world software engineering

real-world software engineering

software engineering across many languages

Cursor agentic coding eval (v3.2 task set)

Cursor agentic coding eval (harder tasks)

Artificial Analysis agentic software-engineering eval (v1.1)

Artificial Analysis agentic software-engineering eval

frontier-difficulty agentic coding tasks (v1.1, main split)

machine-learning engineering on real Kaggle competitions

reproducing the results of an ML research paper end to end

Next.js code generation & migration tasks

building and debugging real Supabase projects as a coding agent

contamination-free code generation on freshly published problems

writing a single Python function from its docstring — the standard coding benchmark of 2021–2024, retired in favour of SWE-Bench-style tests on real repositories

OpenAI's internal software-engineering eval

diverse, difficult agentic computer-work tasks (successor to Terminal-Bench)

command-line task completion

command-line task completion (v2.0)

multi-step workflows using MCP

professional workplace tool-use tasks

verified real-world personal tool use

real-world general tool use

browser agent task completion (Browser Use)

agentic web browsing and research

cybersecurity agentic tasks

operating a real computer to finish tasks (v2.0)

operating a real computer to finish tasks

end-to-end business workflow automation

expert-level questions across disciplines

novel problem-solving in interactive environments

abstract reasoning puzzles

research-level mathematics

solving open biological research mysteries

graduate-level science reasoning

multiple-choice exam questions across 57 academic subjects

grade-school maths word problems needing a few steps of arithmetic reasoning

economically valuable knowledge work

economically valuable knowledge work (v2, re-based Elo)

GDPval win-or-tie rate vs human experts

financial analysis tasks

completing legal work on a held-out task set

completing legal work with documents, spreadsheets, presentations, and file-system tools

answering tax questions (Vals AI proprietary set)

realistic health conversations graded against physician rubrics

supporting doctors with administrative work (Vals AI proprietary set)

information synthesis from complex charts

core visual reasoning

multimodal understanding and reasoning

multimodal understanding

agentic spatial reasoning

long-context retrieval

detecting and pushing back on nonsense prompts

indirect prompt injection attack success rate

human head-to-head preference votes on arena.ai (text & chat)

human head-to-head preference votes on arena.ai (code generation)