Compare AI models

Specifications
Parameters
12B51B
Context window
128k128k
API pricing
Cheapest input
$0.0166Io Net—
Cheapest output
$0.0216Io Net—
$4.40Io Net · fp16—
Benchmarks
HellaSwag
83.5%85.58%
WinoGrande
76.8%84.53%
MMLU
68%80.2%
Overview
CompanyMistralNVIDIA
Release dateJul 18 2024Sep 23 2024
AccessOpen WeightOpen Weight
Model detailsView modelView model

Frequently asked questions

Llama 3.1 Nemotron 51B leads Mistral NeMo on 3 of the 3 benchmarks they both report (HellaSwag, WinoGrande, MMLU). Mistral NeMo shipped 67 days before Llama 3.1 Nemotron 51B, so benchmark comparisons should account for the intervening progress.

Mistral NeMo has 12B parameters, while Llama 3.1 Nemotron 51B has 51B. Context windows are 128k (Mistral NeMo) vs 128k (Llama 3.1 Nemotron 51B).

On HellaSwag, Llama 3.1 Nemotron 51B leads at 85.58% vs Mistral NeMo at 83.5%. On WinoGrande, Llama 3.1 Nemotron 51B leads at 84.53% vs Mistral NeMo at 76.8%. On MMLU, Llama 3.1 Nemotron 51B leads at 80.2% vs Mistral NeMo at 68%.