Compare AI models

Specifications
Parameters
51B—
Context window
128k8k
API pricing
Cheapest input
—$30.00Azure
Cheapest output
—$60.00Azure
—$9,000Azure
Benchmarks
ARC-Challenge
69.2%96.3%
HellaSwag
85.58%95.3%
WinoGrande
84.53%87.5%
MMLU
80.2%86.4%
GSM8K
91.43%92%
Overview
CompanyNVIDIAOpenAI
Release dateSep 23 2024Mar 14 2023
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

GPT-4 leads Llama 3.1 Nemotron 51B on 5 of the 5 benchmarks they both report. GPT-4 shipped 559 days before Llama 3.1 Nemotron 51B, so benchmark comparisons should account for the intervening progress.

Context windows are 128k (Llama 3.1 Nemotron 51B) vs 8k (GPT-4). Llama 3.1 Nemotron 51B is open weight, while GPT-4 is closed.

On ARC-Challenge, GPT-4 leads at 96.3% vs Llama 3.1 Nemotron 51B at 69.2%. On HellaSwag, GPT-4 leads at 95.3% vs Llama 3.1 Nemotron 51B at 85.58%. On WinoGrande, GPT-4 leads at 87.5% vs Llama 3.1 Nemotron 51B at 84.53%. On MMLU, GPT-4 leads at 86.4% vs Llama 3.1 Nemotron 51B at 80.2%. On GSM8K, GPT-4 leads at 92% vs Llama 3.1 Nemotron 51B at 91.43%.