Compare AI models

Specifications
Context window
200k—
Benchmarks
HumanEval
92%85.7%
MMLU
90.4%86.2%
MMMU
68.3%63.2%
Overview
CompanyAnthropicSpaceXSI
Release dateJun 20 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Sonnet leads Grok‑2 mini on 3 of the 3 benchmarks they both report (HumanEval, MMLU, MMMU). Claude 3.5 Sonnet shipped 54 days before Grok‑2 mini, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Claude 3.5 Sonnet leads at 92% vs Grok‑2 mini at 85.7%. On MMLU, Claude 3.5 Sonnet leads at 90.4% vs Grok‑2 mini at 86.2%. On MMMU, Claude 3.5 Sonnet leads at 68.3% vs Grok‑2 mini at 63.2%.