Compare AI models

Specifications
Context window
1M—
Benchmarks
HumanEval
74.3%85.7%
MATH
54.9%73%
MMLU
78.9%86.2%
MathVista
58.4%68.1%
MMMU
56.1%63.2%
Overview
CompanyGoogleSpaceXSI
Release dateMay 14 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 mini leads Gemini 1.5 Flash on 5 of the 5 benchmarks they both report. Gemini 1.5 Flash shipped 91 days before Grok‑2 mini, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, Grok‑2 mini leads at 85.7% vs Gemini 1.5 Flash at 74.3%. On MATH, Grok‑2 mini leads at 73% vs Gemini 1.5 Flash at 54.9%. On MMLU, Grok‑2 mini leads at 86.2% vs Gemini 1.5 Flash at 78.9%. On MathVista, Grok‑2 mini leads at 68.1% vs Gemini 1.5 Flash at 58.4%. On MMMU, Grok‑2 mini leads at 63.2% vs Gemini 1.5 Flash at 56.1%.