Compare AI models

Specifications
Parameters
12B—
Context window
128k128k
Benchmarks
HumanEval
72%75.7%
IFEval
61.3%75.2%
MATH
48.1%57.9%
MMLU
69.2%81.9%
Overview
CompanyMistralZ.ai
Release dateSep 11 2024Jun 5 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

GLM-4-Air leads Pixtral 24.09 on 4 of the 4 benchmarks they both report (HumanEval, IFEval, MATH, MMLU). GLM-4-Air shipped 98 days before Pixtral 24.09, so benchmark comparisons should account for the intervening progress.

Context windows are 128k (Pixtral 24.09) vs 128k (GLM-4-Air). Pixtral 24.09 is open weight, while GLM-4-Air is closed.

On HumanEval, GLM-4-Air leads at 75.7% vs Pixtral 24.09 at 72%. On IFEval, GLM-4-Air leads at 75.2% vs Pixtral 24.09 at 61.3%. On MATH, GLM-4-Air leads at 57.9% vs Pixtral 24.09 at 48.1%. On MMLU, GLM-4-Air leads at 81.9% vs Pixtral 24.09 at 69.2%.