Compare AI models

Specifications
Context window
32k—
Benchmarks
HumanEval
67.7%72%
MATH
32.6%47.9%
MMLU
71.8%81.5%
GSM8K
77.9%87.6%
Overview
CompanyGoogleZ.ai
Release dateDec 13 2023Jan 16 2024
AccessClosedClosed
Model detailsView modelView model

Frequently asked questions

GLM-4 leads Gemini 1.0 Pro on 4 of the 4 benchmarks they both report (HumanEval, MATH, MMLU, GSM8K). Gemini 1.0 Pro shipped 34 days before GLM-4, so benchmark comparisons should account for the intervening progress.

Published specifications for these two models are limited — see each model page for the latest details.

On HumanEval, GLM-4 leads at 72% vs Gemini 1.0 Pro at 67.7%. On MATH, GLM-4 leads at 47.9% vs Gemini 1.0 Pro at 32.6%. On MMLU, GLM-4 leads at 81.5% vs Gemini 1.0 Pro at 71.8%. On GSM8K, GLM-4 leads at 87.6% vs Gemini 1.0 Pro at 77.9%.