Compare AI models

Specifications
Parameters
1B/4B/12B/27B—
Context window
—1M
API pricing
Cheapest input
$0.04DeepInfra—
Cheapest output
$0.05Parasail—
$10.50Parasail · bf16—
Benchmarks
LiveCodeBench
29.7%37.8%
HumanEval
87.8%86.6%
IFEval
90.4%76.3%
MATH
89%81.1%
MMLU-Pro
67.5%64.5%
GSM8K
95.9%93.8%
Overview
CompanyGoogleQwen
Release dateMar 12 2025Nov 15 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Gemma 3 leads Qwen2.5-Turbo (1M) on 5 of the 6 benchmarks they both report. Qwen2.5-Turbo (1M) shipped 117 days before Gemma 3, so benchmark comparisons should account for the intervening progress.

Gemma 3 is open weight, while Qwen2.5-Turbo (1M) is closed.

On LiveCodeBench, Qwen2.5-Turbo (1M) leads at 37.8% vs Gemma 3 at 29.7%. On HumanEval, Gemma 3 leads at 87.8% vs Qwen2.5-Turbo (1M) at 86.6%. On IFEval, Gemma 3 leads at 90.4% vs Qwen2.5-Turbo (1M) at 76.3%. On MATH, Gemma 3 leads at 89% vs Qwen2.5-Turbo (1M) at 81.1%. On MMLU-Pro, Gemma 3 leads at 67.5% vs Qwen2.5-Turbo (1M) at 64.5%. On GSM8K, Gemma 3 leads at 95.9% vs Qwen2.5-Turbo (1M) at 93.8%.