Compare AI models

Specifications
Context window
200k—
Benchmarks
HumanEval
93.7%85.7%
MMLU
90.5%86.2%
MMMU
70.4%63.2%
Overview
CompanyAnthropicSpaceXSI
Release dateOct 22 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Sonnet (upgraded) leads Grok‑2 mini on 3 of the 3 benchmarks they both report (HumanEval, MMLU, MMMU). Grok‑2 mini shipped 70 days before Claude 3.5 Sonnet (upgraded), so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Claude 3.5 Sonnet (upgraded) leads at 93.7% vs Grok‑2 mini at 85.7%. On MMLU, Claude 3.5 Sonnet (upgraded) leads at 90.5% vs Grok‑2 mini at 86.2%. On MMMU, Claude 3.5 Sonnet (upgraded) leads at 70.4% vs Grok‑2 mini at 63.2%.