Compare AI models

Specifications
Context window
1M—
Benchmarks
HumanEval
71.9%85.7%
MATH
58.5%73%
MMLU
81.9%86.2%
MathVista
52.1%68.1%
MMMU
58.5%63.2%
Overview
CompanyGoogleSpaceXSI
Release dateFeb 15 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 mini leads Gemini 1.5 Pro on 5 of the 5 benchmarks they both report. Gemini 1.5 Pro shipped 180 days before Grok‑2 mini, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Grok‑2 mini leads at 85.7% vs Gemini 1.5 Pro at 71.9%. On MATH, Grok‑2 mini leads at 73% vs Gemini 1.5 Pro at 58.5%. On MMLU, Grok‑2 mini leads at 86.2% vs Gemini 1.5 Pro at 81.9%. On MathVista, Grok‑2 mini leads at 68.1% vs Gemini 1.5 Pro at 52.1%. On MMMU, Grok‑2 mini leads at 63.2% vs Gemini 1.5 Pro at 58.5%.