Compare AI models

Specifications
Parameters
—8B
Context window
200k8k
Benchmarks
HumanEval
88.1%36.22%
MMLU
80.9%69.51%
Overview
CompanyAnthropicNVIDIA
Release dateOct 22 2024Aug 21 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Haiku leads Mistral NeMo Minitron 8B on 2 of the 2 benchmarks they both report (HumanEval, MMLU). Mistral NeMo Minitron 8B shipped 62 days before Claude 3.5 Haiku, so benchmark comparisons should account for the intervening progress.

Context windows are 200k (Claude 3.5 Haiku) vs 8k (Mistral NeMo Minitron 8B). Claude 3.5 Haiku is closed, while Mistral NeMo Minitron 8B is open weight.

On HumanEval, Claude 3.5 Haiku leads at 88.1% vs Mistral NeMo Minitron 8B at 36.22%. On MMLU, Claude 3.5 Haiku leads at 80.9% vs Mistral NeMo Minitron 8B at 69.51%.