Compare AI models

Specifications
Parameters
9B/27B8B
Context window
8k8k
API pricing
Cheapest input
$0.65NextBit—
Cheapest output
$0.65NextBit—
$163NextBit · int4—
Benchmarks
HumanEval
51.8%36.22%
ARC-Challenge
71.4%64.42%
HellaSwag
86.4%83.03%
MMLU
75.2%69.51%
GSM8K
74%58.45%
Overview
CompanyGoogleNVIDIA
Release dateJun 27 2024Aug 21 2024
AccessOpen WeightOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemma 2 leads Mistral NeMo Minitron 8B on 5 of the 5 benchmarks they both report. Gemma 2 shipped 55 days before Mistral NeMo Minitron 8B, so benchmark comparisons should account for the intervening progress.

Gemma 2 has 9B/27B parameters, while Mistral NeMo Minitron 8B has 8B. Context windows are 8k (Gemma 2) vs 8k (Mistral NeMo Minitron 8B).

On HumanEval, Gemma 2 leads at 51.8% vs Mistral NeMo Minitron 8B at 36.22%. On ARC-Challenge, Gemma 2 leads at 71.4% vs Mistral NeMo Minitron 8B at 64.42%. On HellaSwag, Gemma 2 leads at 86.4% vs Mistral NeMo Minitron 8B at 83.03%. On MMLU, Gemma 2 leads at 75.2% vs Mistral NeMo Minitron 8B at 69.51%. On GSM8K, Gemma 2 leads at 74% vs Mistral NeMo Minitron 8B at 58.45%.