Compare AI models

Specifications
Parameters
—8B
Context window
32k8k
Benchmarks
HumanEval
67.7%36.22%
HellaSwag
84.7%83.03%
MMLU
71.8%69.51%
GSM8K
77.9%58.45%
Overview
CompanyGoogleNVIDIA
Release dateDec 13 2023Aug 21 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.0 Pro leads Mistral NeMo Minitron 8B on 4 of the 4 benchmarks they both report (HumanEval, HellaSwag, MMLU, GSM8K). Gemini 1.0 Pro shipped 252 days before Mistral NeMo Minitron 8B, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (Gemini 1.0 Pro) vs 8k (Mistral NeMo Minitron 8B). Gemini 1.0 Pro is closed, while Mistral NeMo Minitron 8B is open weight.

On HumanEval, Gemini 1.0 Pro leads at 67.7% vs Mistral NeMo Minitron 8B at 36.22%. On HellaSwag, Gemini 1.0 Pro leads at 84.7% vs Mistral NeMo Minitron 8B at 83.03%. On MMLU, Gemini 1.0 Pro leads at 71.8% vs Mistral NeMo Minitron 8B at 69.51%. On GSM8K, Gemini 1.0 Pro leads at 77.9% vs Mistral NeMo Minitron 8B at 58.45%.