Compare AI models

Specifications
Parameters
16B—
Context window
32k32k
Benchmarks
HumanEval
57.3%74.4%
MATH
27.9%53.2%
MMLU
55.7%83.7%
GSM8K
72%88.9%
Overview
CompanyDeepSeekGoogle
Release dateMay 16 2024Feb 8 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Gemini 1.0 Ultra leads DeepSeek V2 Lite on 4 of the 4 benchmarks they both report (HumanEval, MATH, MMLU, GSM8K). Gemini 1.0 Ultra shipped 98 days before DeepSeek V2 Lite, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (DeepSeek V2 Lite) vs 32k (Gemini 1.0 Ultra). DeepSeek V2 Lite is open weight, while Gemini 1.0 Ultra is closed.

On HumanEval, Gemini 1.0 Ultra leads at 74.4% vs DeepSeek V2 Lite at 57.3%. On MATH, Gemini 1.0 Ultra leads at 53.2% vs DeepSeek V2 Lite at 27.9%. On MMLU, Gemini 1.0 Ultra leads at 83.7% vs DeepSeek V2 Lite at 55.7%. On GSM8K, Gemini 1.0 Ultra leads at 88.9% vs DeepSeek V2 Lite at 72%.