Compare AI models

Specifications
Parameters
—51B
Context window
32k128k
Benchmarks
ARC-Challenge
94.2%69.2%
HellaSwag
89.2%85.58%
WinoGrande
86.7%84.53%
MMLU
81.2%80.2%
GSM8K
81%91.43%
Overview
CompanyMistralNVIDIA
Release dateFeb 26 2024Sep 23 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Mistral Large leads Llama 3.1 Nemotron 51B on 4 of the 5 benchmarks they both report. Mistral Large shipped 210 days before Llama 3.1 Nemotron 51B, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (Mistral Large) vs 128k (Llama 3.1 Nemotron 51B). Mistral Large is closed, while Llama 3.1 Nemotron 51B is open weight.

On ARC-Challenge, Mistral Large leads at 94.2% vs Llama 3.1 Nemotron 51B at 69.2%. On HellaSwag, Mistral Large leads at 89.2% vs Llama 3.1 Nemotron 51B at 85.58%. On WinoGrande, Mistral Large leads at 86.7% vs Llama 3.1 Nemotron 51B at 84.53%. On MMLU, Mistral Large leads at 81.2% vs Llama 3.1 Nemotron 51B at 80.2%. On GSM8K, Llama 3.1 Nemotron 51B leads at 91.43% vs Mistral Large at 81%.