Compare AI models

Specifications
Context window
200k—
Benchmarks
HumanEval
88.1%85.7%
MMLU
80.9%86.2%
Overview
CompanyAnthropicSpaceXSI
Release dateOct 22 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Haiku and Grok‑2 mini are evenly matched across the 2 benchmarks they both report (HumanEval, MMLU). Grok‑2 mini shipped 70 days before Claude 3.5 Haiku, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Claude 3.5 Haiku leads at 88.1% vs Grok‑2 mini at 85.7%. On MMLU, Grok‑2 mini leads at 86.2% vs Claude 3.5 Haiku at 80.9%.