Compare AI models

Specifications
Parameters
8B/70B/405B—
Context window
128k—
API pricing
Cheapest input
$0.02DeepInfra—
Cheapest output
$0.04DeepInfra—
$6.00DeepInfra · fp8—
Benchmarks
HumanEval
89%85.7%
MATH
73.8%73%
MMLU-Pro
73.3%72%
MMLU
87.3%86.2%
Overview
CompanyMetaSpaceXSI
Release dateJul 23 2024Aug 13 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

LLaMA 3.1 leads Grok‑2 mini on 4 of the 4 benchmarks they both report (HumanEval, MATH, MMLU-Pro, MMLU). LLaMA 3.1 shipped 21 days before Grok‑2 mini, so benchmark comparisons should account for the intervening progress.

LLaMA 3.1 is open weight, while Grok‑2 mini is closed.

On HumanEval, LLaMA 3.1 leads at 89% vs Grok‑2 mini at 85.7%. On MATH, LLaMA 3.1 leads at 73.8% vs Grok‑2 mini at 73%. On MMLU-Pro, LLaMA 3.1 leads at 73.3% vs Grok‑2 mini at 72%. On MMLU, LLaMA 3.1 leads at 87.3% vs Grok‑2 mini at 86.2%.