Compare AI models

Specifications
Parameters
12B—
Context window
128k—
Benchmarks
HumanEval
72%72%
MATH
48.1%47.9%
MMLU
69.2%81.5%
Overview
CompanyMistralZ.ai
Release dateSep 11 2024Jan 16 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Pixtral 24.09 and GLM-4 are evenly matched across the 3 benchmarks they both report (HumanEval, MATH, MMLU). GLM-4 shipped 239 days before Pixtral 24.09, so benchmark comparisons should account for the intervening progress.

Pixtral 24.09 is open weight, while GLM-4 is closed.

On HumanEval, both models score 72%. On MATH, Pixtral 24.09 leads at 48.1% vs GLM-4 at 47.9%. On MMLU, GLM-4 leads at 81.5% vs Pixtral 24.09 at 69.2%.