Compare AI models

Specifications
Parameters
—0.5B/1.5B/3B/7B/14B/32B
Context window
128k128k
API pricing
Cheapest input
—$0.66Cloudflare
Cheapest output
—$1.00Cloudflare
—$182Cloudflare
Benchmarks
HumanEval
92.4%92.7%
MMLU
85.2%77.6%
Overview
CompanyOpenAIQwen
Release dateSep 12 2024Sep 19 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

o1-mini and Qwen2.5-Coder are evenly matched across the 2 benchmarks they both report (HumanEval, MMLU). o1-mini shipped 7 days before Qwen2.5-Coder, so benchmark comparisons should account for the intervening progress.

Context windows are 128k (o1-mini) vs 128k (Qwen2.5-Coder). o1-mini is closed, while Qwen2.5-Coder is open weight.

On HumanEval, Qwen2.5-Coder leads at 92.7% vs o1-mini at 92.4%. On MMLU, o1-mini leads at 85.2% vs Qwen2.5-Coder at 77.6%.