Compare AI models

Specifications
Parameters
16B—
Context window
32k1M
Benchmarks
HumanEval
57.3%71.9%
MATH
27.9%58.5%
MMLU
55.7%81.9%
GSM8K
72%91.7%
Overview
CompanyDeepSeekGoogle
Release dateMay 16 2024Feb 15 2024
AccessOpen WeightClosed
Model detailsView modelView model

Frequently asked questions

Gemini 1.5 Pro leads DeepSeek V2 Lite on 4 of the 4 benchmarks they both report (HumanEval, MATH, MMLU, GSM8K). Gemini 1.5 Pro shipped 91 days before DeepSeek V2 Lite, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (DeepSeek V2 Lite) vs 1M (Gemini 1.5 Pro). DeepSeek V2 Lite is open weight, while Gemini 1.5 Pro is closed.

On HumanEval, Gemini 1.5 Pro leads at 71.9% vs DeepSeek V2 Lite at 57.3%. On MATH, Gemini 1.5 Pro leads at 58.5% vs DeepSeek V2 Lite at 27.9%. On MMLU, Gemini 1.5 Pro leads at 81.9% vs DeepSeek V2 Lite at 55.7%. On GSM8K, Gemini 1.5 Pro leads at 91.7% vs DeepSeek V2 Lite at 72%.