Compare AI models

Specifications
Parameters
8B—
Context window
8k8k
API pricing
Cheapest input
—$30.00Azure
Cheapest output
—$60.00Azure
—$9,000Azure
Benchmarks
HumanEval
36.22%67%
ARC-Challenge
64.42%96.3%
HellaSwag
83.03%95.3%
WinoGrande
80.35%87.5%
MMLU
69.51%86.4%
GSM8K
58.45%92%
Overview
CompanyNVIDIAOpenAI
Release dateAug 21 2024Mar 14 2023
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

GPT-4 leads Mistral NeMo Minitron 8B on 6 of the 6 benchmarks they both report. GPT-4 shipped 526 days before Mistral NeMo Minitron 8B, so benchmark comparisons should account for the intervening progress.

Context windows are 8k (Mistral NeMo Minitron 8B) vs 8k (GPT-4). Mistral NeMo Minitron 8B is open weight, while GPT-4 is closed.

On HumanEval, GPT-4 leads at 67% vs Mistral NeMo Minitron 8B at 36.22%. On ARC-Challenge, GPT-4 leads at 96.3% vs Mistral NeMo Minitron 8B at 64.42%. On HellaSwag, GPT-4 leads at 95.3% vs Mistral NeMo Minitron 8B at 83.03%. On WinoGrande, GPT-4 leads at 87.5% vs Mistral NeMo Minitron 8B at 80.35%. On MMLU, GPT-4 leads at 86.4% vs Mistral NeMo Minitron 8B at 69.51%. On GSM8K, GPT-4 leads at 92% vs Mistral NeMo Minitron 8B at 58.45%.