Compare AI models

Specifications
Parameters
12B—
Context window
128k—
Benchmarks
HumanEval
72%85.7%
MATH
48.1%73%
MMLU
69.2%86.2%
MathVista
58%68.1%
MMMU
52.5%63.2%
Overview
CompanyMistralSpaceXSI
Release dateSep 11 2024Aug 13 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 mini leads Pixtral 24.09 on 5 of the 5 benchmarks they both report. Grok‑2 mini shipped 29 days before Pixtral 24.09, so benchmark comparisons should account for the intervening progress.

Pixtral 24.09 is open weight, while Grok‑2 mini is closed.

On HumanEval, Grok‑2 mini leads at 85.7% vs Pixtral 24.09 at 72%. On MATH, Grok‑2 mini leads at 73% vs Pixtral 24.09 at 48.1%. On MMLU, Grok‑2 mini leads at 86.2% vs Pixtral 24.09 at 69.2%. On MathVista, Grok‑2 mini leads at 68.1% vs Pixtral 24.09 at 58%. On MMMU, Grok‑2 mini leads at 63.2% vs Pixtral 24.09 at 52.5%.