Compare AI models

Specifications
Context window
1M128k
Benchmarks
HumanEval
86.6%88.4%
MATH
81.1%76.1%
MMLU-Pro
64.5%75.5%
Overview
CompanyQwenSpaceXSI
Release dateNov 15 2024Aug 13 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Grok‑2 leads Qwen2.5-Turbo (1M) on 2 of the 3 benchmarks they both report (HumanEval, MATH, MMLU-Pro). Grok‑2 shipped 94 days before Qwen2.5-Turbo (1M), so benchmark comparisons should account for the intervening progress.

Context windows are 1M (Qwen2.5-Turbo (1M)) vs 128k (Grok‑2).

On HumanEval, Grok‑2 leads at 88.4% vs Qwen2.5-Turbo (1M) at 86.6%. On MATH, Qwen2.5-Turbo (1M) leads at 81.1% vs Grok‑2 at 76.1%. On MMLU-Pro, Grok‑2 leads at 75.5% vs Qwen2.5-Turbo (1M) at 64.5%.