Compare AI models

Specifications
Parameters
—8B
Context window
32k4k
Benchmarks
HellaSwag
87.8%76.4%
MMLU
83.7%54.4%
Overview
CompanyGoogleNVIDIA
Release dateFeb 8 2024Nov 15 2023
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.0 Ultra leads Nemotron-3 8B on 2 of the 2 benchmarks they both report (HellaSwag, MMLU). Nemotron-3 8B shipped 85 days before Gemini 1.0 Ultra, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (Gemini 1.0 Ultra) vs 4k (Nemotron-3 8B). Gemini 1.0 Ultra is closed, while Nemotron-3 8B is open weight.

On HellaSwag, Gemini 1.0 Ultra leads at 87.8% vs Nemotron-3 8B at 76.4%. On MMLU, Gemini 1.0 Ultra leads at 83.7% vs Nemotron-3 8B at 54.4%.