Compare AI models

Specifications
Parameters
5.1B/8B/26B/31B—
Context window
—128k
API pricing
Cheapest input
$0.04Darkbloom—
Cheapest output
$0.10Modal—
$15.50Darkbloom—
Benchmarks
GPQA Diamond
84.3%65.7%
MMLU-Pro
85.2%76.8%
MMMU-Pro
76.9%43.8%
Overview
CompanyGoogleMistral
Release dateApr 2 2026Aug 12 2025
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Gemma 4 leads Mistral Medium 3.1 on 3 of the 3 benchmarks they both report (GPQA Diamond, MMLU-Pro, MMMU-Pro). Mistral Medium 3.1 shipped 233 days before Gemma 4, so benchmark comparisons should account for the intervening progress.

Gemma 4 is open weight, while Mistral Medium 3.1 is closed.

On GPQA Diamond, Gemma 4 leads at 84.3% vs Mistral Medium 3.1 at 65.7%. On MMLU-Pro, Gemma 4 leads at 85.2% vs Mistral Medium 3.1 at 76.8%. On MMMU-Pro, Gemma 4 leads at 76.9% vs Mistral Medium 3.1 at 43.8%.