Compare AI models

Specifications
Parameters
—8B
Context window
200k8k
Benchmarks
HumanEval
93.7%36.22%
MMLU
90.5%69.51%
Overview
CompanyAnthropicNVIDIA
Release dateOct 22 2024Aug 21 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Sonnet (upgraded) leads Mistral NeMo Minitron 8B on 2 of the 2 benchmarks they both report (HumanEval, MMLU). Mistral NeMo Minitron 8B shipped 62 days before Claude 3.5 Sonnet (upgraded), so benchmark comparisons should account for the intervening progress.

Context windows are 200k (Claude 3.5 Sonnet (upgraded)) vs 8k (Mistral NeMo Minitron 8B). Claude 3.5 Sonnet (upgraded) is closed, while Mistral NeMo Minitron 8B is open weight.

On HumanEval, Claude 3.5 Sonnet (upgraded) leads at 93.7% vs Mistral NeMo Minitron 8B at 36.22%. On MMLU, Claude 3.5 Sonnet (upgraded) leads at 90.5% vs Mistral NeMo Minitron 8B at 69.51%.