Compare AI models

Specifications
Parameters
—51B
Context window
200k128k
Benchmarks
MMLU
90.4%80.2%
GSM8K
96.4%91.43%
Overview
CompanyAnthropicNVIDIA
Release dateJun 20 2024Sep 23 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Sonnet leads Llama 3.1 Nemotron 51B on 2 of the 2 benchmarks they both report (MMLU, GSM8K). Claude 3.5 Sonnet shipped 95 days before Llama 3.1 Nemotron 51B, so benchmark comparisons should account for the intervening progress.

Context windows are 200k (Claude 3.5 Sonnet) vs 128k (Llama 3.1 Nemotron 51B). Claude 3.5 Sonnet is closed, while Llama 3.1 Nemotron 51B is open weight.

On MMLU, Claude 3.5 Sonnet leads at 90.4% vs Llama 3.1 Nemotron 51B at 80.2%. On GSM8K, Claude 3.5 Sonnet leads at 96.4% vs Llama 3.1 Nemotron 51B at 91.43%.