Compare AI models

Specifications
Context window
32k—
Benchmarks
HumanEval
74.4%85.7%
MATH
53.2%73%
MMLU
83.7%86.2%
MathVista
53%68.1%
MMMU
59.4%63.2%
Overview
CompanyGoogleSpaceXSI
Release dateFeb 8 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 mini leads Gemini 1.0 Ultra on 5 of the 5 benchmarks they both report. Gemini 1.0 Ultra shipped 187 days before Grok‑2 mini, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Grok‑2 mini leads at 85.7% vs Gemini 1.0 Ultra at 74.4%. On MATH, Grok‑2 mini leads at 73% vs Gemini 1.0 Ultra at 53.2%. On MMLU, Grok‑2 mini leads at 86.2% vs Gemini 1.0 Ultra at 83.7%. On MathVista, Grok‑2 mini leads at 68.1% vs Gemini 1.0 Ultra at 53%. On MMMU, Grok‑2 mini leads at 63.2% vs Gemini 1.0 Ultra at 59.4%.