Compare AI models

Specifications
Parameters
7B51B
Context window
8k128k
Benchmarks
ARC-Challenge
54.9%69.2%
HellaSwag
81%85.58%
WinoGrande
74.2%84.53%
MMLU
62.5%80.2%
GSM8K
36.5%91.43%
Overview
CompanyMistralNVIDIA
Release dateSep 27 2023Sep 23 2024
AccessOpen WeightOpen Weight
Model detailsView modelView model

Frequently asked questions

Llama 3.1 Nemotron 51B leads Mistral 7B on 5 of the 5 benchmarks they both report. Mistral 7B shipped 362 days before Llama 3.1 Nemotron 51B, so benchmark comparisons should account for the intervening progress.

Mistral 7B has 7B parameters, while Llama 3.1 Nemotron 51B has 51B. Context windows are 8k (Mistral 7B) vs 128k (Llama 3.1 Nemotron 51B).

On ARC-Challenge, Llama 3.1 Nemotron 51B leads at 69.2% vs Mistral 7B at 54.9%. On HellaSwag, Llama 3.1 Nemotron 51B leads at 85.58% vs Mistral 7B at 81%. On WinoGrande, Llama 3.1 Nemotron 51B leads at 84.53% vs Mistral 7B at 74.2%. On MMLU, Llama 3.1 Nemotron 51B leads at 80.2% vs Mistral 7B at 62.5%. On GSM8K, Llama 3.1 Nemotron 51B leads at 91.43% vs Mistral 7B at 36.5%.