Compare AI models

Specifications
Parameters
—8B
Context window
200k8k
Benchmarks
HumanEval
92%36.22%
MMLU
90.4%69.51%
GSM8K
96.4%58.45%
Overview
CompanyAnthropicNVIDIA
Release dateJun 20 2024Aug 21 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Sonnet leads Mistral NeMo Minitron 8B on 3 of the 3 benchmarks they both report (HumanEval, MMLU, GSM8K). Claude 3.5 Sonnet shipped 62 days before Mistral NeMo Minitron 8B, so benchmark comparisons should account for the intervening progress.

Context windows are 200k (Claude 3.5 Sonnet) vs 8k (Mistral NeMo Minitron 8B). Claude 3.5 Sonnet is closed, while Mistral NeMo Minitron 8B is open weight.

On HumanEval, Claude 3.5 Sonnet leads at 92% vs Mistral NeMo Minitron 8B at 36.22%. On MMLU, Claude 3.5 Sonnet leads at 90.4% vs Mistral NeMo Minitron 8B at 69.51%. On GSM8K, Claude 3.5 Sonnet leads at 96.4% vs Mistral NeMo Minitron 8B at 58.45%.