Compare AI models

Specifications
Parameters
24B—
Context window
32k1M
API pricing
Cheapest input
$0.04DeepInfra—
Cheapest output
$0.064DeepInfra—
$14.00DeepInfra · fp8—
Benchmarks
HumanEval
84.8%86.6%
IFEval
82.9%76.3%
MATH
70.6%81.1%
MMLU-Pro
66.3%64.5%
Overview
CompanyMistralQwen
Release dateJan 30 2025Nov 15 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Mistral Small 3 and Qwen2.5-Turbo (1M) are evenly matched across the 4 benchmarks they both report (HumanEval, IFEval, MATH, MMLU-Pro). Qwen2.5-Turbo (1M) shipped 76 days before Mistral Small 3, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (Mistral Small 3) vs 1M (Qwen2.5-Turbo (1M)). Mistral Small 3 is open weight, while Qwen2.5-Turbo (1M) is closed.

On HumanEval, Qwen2.5-Turbo (1M) leads at 86.6% vs Mistral Small 3 at 84.8%. On IFEval, Mistral Small 3 leads at 82.9% vs Qwen2.5-Turbo (1M) at 76.3%. On MATH, Qwen2.5-Turbo (1M) leads at 81.1% vs Mistral Small 3 at 70.6%. On MMLU-Pro, Mistral Small 3 leads at 66.3% vs Qwen2.5-Turbo (1M) at 64.5%.