Compare AI models

Specifications
Parameters
1B/4B/12B/27B7B/14B
Context window
—1M
API pricing
Cheapest input
$0.04DeepInfra—
Cheapest output
$0.05Parasail—
$10.50Parasail · bf16—
Benchmarks
LiveCodeBench
29.7%38.6%
HumanEval
87.8%88.4%
IFEval
90.4%84.3%
MATH
89%79.5%
MMLU-Pro
67.5%63.3%
GSM8K
95.9%94.8%
Overview
CompanyGoogleQwen
Release dateMar 12 2025Jan 27 2025
AccessOpen WeightOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemma 3 leads Qwen2.5-1M on 4 of the 6 benchmarks they both report. Qwen2.5-1M shipped 44 days before Gemma 3, so benchmark comparisons should account for the intervening progress.

Gemma 3 has 1B/4B/12B/27B parameters, while Qwen2.5-1M has 7B/14B.

On LiveCodeBench, Qwen2.5-1M leads at 38.6% vs Gemma 3 at 29.7%. On HumanEval, Qwen2.5-1M leads at 88.4% vs Gemma 3 at 87.8%. On IFEval, Gemma 3 leads at 90.4% vs Qwen2.5-1M at 84.3%. On MATH, Gemma 3 leads at 89% vs Qwen2.5-1M at 79.5%. On MMLU-Pro, Gemma 3 leads at 67.5% vs Qwen2.5-1M at 63.3%. On GSM8K, Gemma 3 leads at 95.9% vs Qwen2.5-1M at 94.8%.