Compare AI models

Specifications
Parameters
24B7B/14B
Context window
32k1M
API pricing
Cheapest input
$0.04DeepInfra—
Cheapest output
$0.064DeepInfra—
$14.00DeepInfra · fp8—
Benchmarks
HumanEval
84.8%88.4%
IFEval
82.9%84.3%
MATH
70.6%79.5%
MMLU-Pro
66.3%63.3%
Overview
CompanyMistralQwen
Release dateJan 30 2025Jan 27 2025
AccessOpen WeightOpen Weight
Model detailsView modelView model

Frequently asked questions

Qwen2.5-1M leads Mistral Small 3 on 3 of the 4 benchmarks they both report (HumanEval, IFEval, MATH, MMLU-Pro). Qwen2.5-1M shipped 3 days before Mistral Small 3, so benchmark comparisons should account for the intervening progress.

Mistral Small 3 has 24B parameters, while Qwen2.5-1M has 7B/14B. Context windows are 32k (Mistral Small 3) vs 1M (Qwen2.5-1M).

On HumanEval, Qwen2.5-1M leads at 88.4% vs Mistral Small 3 at 84.8%. On IFEval, Qwen2.5-1M leads at 84.3% vs Mistral Small 3 at 82.9%. On MATH, Qwen2.5-1M leads at 79.5% vs Mistral Small 3 at 70.6%. On MMLU-Pro, Mistral Small 3 leads at 66.3% vs Qwen2.5-1M at 63.3%.