Compare AI models

Specifications
Parameters
9B/27B6B
Context window
8k32k
API pricing
Cheapest input
$0.65NextBit—
Cheapest output
$0.65NextBit—
$163NextBit · int4—
Benchmarks
HumanEval
51.8%9.8%
MATH
42.3%6.9%
HellaSwag
86.4%57%
MMLU
75.2%45.2%
GSM8K
74%25.9%
Overview
CompanyGoogleZ.ai
Release dateJun 27 2024Jun 25 2023
AccessOpen WeightOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemma 2 leads ChatGLM2-6B on 5 of the 5 benchmarks they both report. ChatGLM2-6B shipped 368 days before Gemma 2, so benchmark comparisons should account for the intervening progress.

Gemma 2 has 9B/27B parameters, while ChatGLM2-6B has 6B. Context windows are 8k (Gemma 2) vs 32k (ChatGLM2-6B).

On HumanEval, Gemma 2 leads at 51.8% vs ChatGLM2-6B at 9.8%. On MATH, Gemma 2 leads at 42.3% vs ChatGLM2-6B at 6.9%. On HellaSwag, Gemma 2 leads at 86.4% vs ChatGLM2-6B at 57%. On MMLU, Gemma 2 leads at 75.2% vs ChatGLM2-6B at 45.2%. On GSM8K, Gemma 2 leads at 74% vs ChatGLM2-6B at 25.9%.