Compare AI models

Specifications
Parameters
—8B
Context window
1M4k
Benchmarks
HellaSwag
92.5%76.4%
MMLU
81.9%54.4%
Overview
CompanyGoogleNVIDIA
Release dateFeb 15 2024Nov 15 2023
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.5 Pro leads Nemotron-3 8B on 2 of the 2 benchmarks they both report (HellaSwag, MMLU). Nemotron-3 8B shipped 92 days before Gemini 1.5 Pro, so benchmark comparisons should account for the intervening progress.

Context windows are 1M (Gemini 1.5 Pro) vs 4k (Nemotron-3 8B). Gemini 1.5 Pro is closed, while Nemotron-3 8B is open weight.

On HellaSwag, Gemini 1.5 Pro leads at 92.5% vs Nemotron-3 8B at 76.4%. On MMLU, Gemini 1.5 Pro leads at 81.9% vs Nemotron-3 8B at 54.4%.