Compare AI models

Specifications
Parameters
—16B
Context window
1M8k
Benchmarks
HumanEval
71.9%48.1%
MATH
58.5%45.3%
MMLU
81.9%70%
GSM8K
91.7%77.4%
Overview
CompanyGoogleMoonshot AI
Release dateFeb 15 2024Feb 22 2025
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.5 Pro leads Moonlight on 4 of the 4 benchmarks they both report (HumanEval, MATH, MMLU, GSM8K). Gemini 1.5 Pro shipped 373 days before Moonlight, so benchmark comparisons should account for the intervening progress.

Context windows are 1M (Gemini 1.5 Pro) vs 8k (Moonlight). Gemini 1.5 Pro is closed, while Moonlight is open weight.

On HumanEval, Gemini 1.5 Pro leads at 71.9% vs Moonlight at 48.1%. On MATH, Gemini 1.5 Pro leads at 58.5% vs Moonlight at 45.3%. On MMLU, Gemini 1.5 Pro leads at 81.9% vs Moonlight at 70%. On GSM8K, Gemini 1.5 Pro leads at 91.7% vs Moonlight at 77.4%.