Compare AI models

Specifications
Parameters
8B—
Context window
1M—
Benchmarks
MATH
35.9%73%
MMLU
68.1%86.2%
MMMU
50.3%63.2%
Overview
CompanyGoogleSpaceXSI
Release dateOct 3 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 mini leads Gemini 1.5 Flash-8B on 3 of the 3 benchmarks they both report (MATH, MMLU, MMMU). Grok‑2 mini shipped 51 days before Gemini 1.5 Flash-8B, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On MATH, Grok‑2 mini leads at 73% vs Gemini 1.5 Flash-8B at 35.9%. On MMLU, Grok‑2 mini leads at 86.2% vs Gemini 1.5 Flash-8B at 68.1%. On MMMU, Grok‑2 mini leads at 63.2% vs Gemini 1.5 Flash-8B at 50.3%.