Compare AI models

Specifications
Context window
32k—
Benchmarks
HumanEval
67.7%85.7%
MATH
32.6%73%
MMLU
71.8%86.2%
MathVista
45.2%68.1%
MMMU
47.9%63.2%
Overview
CompanyGoogleSpaceXSI
Release dateDec 13 2023Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 mini leads Gemini 1.0 Pro on 5 of the 5 benchmarks they both report. Gemini 1.0 Pro shipped 244 days before Grok‑2 mini, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Grok‑2 mini leads at 85.7% vs Gemini 1.0 Pro at 67.7%. On MATH, Grok‑2 mini leads at 73% vs Gemini 1.0 Pro at 32.6%. On MMLU, Grok‑2 mini leads at 86.2% vs Gemini 1.0 Pro at 71.8%. On MathVista, Grok‑2 mini leads at 68.1% vs Gemini 1.0 Pro at 45.2%. On MMMU, Grok‑2 mini leads at 63.2% vs Gemini 1.0 Pro at 47.9%.