Compare AI models

Specifications
Parameters
7B/14B—
Context window
1M128k
Benchmarks
HumanEval
88.4%88.4%
MATH
79.5%76.1%
MMLU-Pro
63.3%75.5%
Overview
CompanyQwenSpaceXSI
Release dateJan 27 2025Aug 13 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Qwen2.5-1M and Grok‑2 are evenly matched across the 3 benchmarks they both report (HumanEval, MATH, MMLU-Pro). Grok‑2 shipped 167 days before Qwen2.5-1M, so benchmark comparisons should account for the intervening progress.

Context windows are 1M (Qwen2.5-1M) vs 128k (Grok‑2). Qwen2.5-1M is open weight, while Grok‑2 is closed.

On HumanEval, both models score 88.4%. On MATH, Qwen2.5-1M leads at 79.5% vs Grok‑2 at 76.1%. On MMLU-Pro, Grok‑2 leads at 75.5% vs Qwen2.5-1M at 63.3%.