Compare AI models

Specifications
Context window
2M1M
Benchmarks
LiveCodeBench
34.3%37.8%
MATH
86.5%81.1%
MMLU-Pro
75.8%64.5%
Overview
CompanyGoogleQwen
Release dateSep 24 2024Nov 15 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

Gemini 1.5 Pro-002 leads Qwen2.5-Turbo (1M) on 2 of the 3 benchmarks they both report (LiveCodeBench, MATH, MMLU-Pro). Gemini 1.5 Pro-002 shipped 52 days before Qwen2.5-Turbo (1M), so benchmark comparisons should account for the intervening progress.

Context windows are 2M (Gemini 1.5 Pro-002) vs 1M (Qwen2.5-Turbo (1M)).

On LiveCodeBench, Qwen2.5-Turbo (1M) leads at 37.8% vs Gemini 1.5 Pro-002 at 34.3%. On MATH, Gemini 1.5 Pro-002 leads at 86.5% vs Qwen2.5-Turbo (1M) at 81.1%. On MMLU-Pro, Gemini 1.5 Pro-002 leads at 75.8% vs Qwen2.5-Turbo (1M) at 64.5%.