Compare AI models

Specifications
Parameters
—16B
Context window
200k32k
Benchmarks
HumanEval
92%57.3%
MMLU
90.4%55.7%
GSM8K
96.4%72%
Overview
CompanyAnthropicDeepSeek
Release dateJun 20 2024May 16 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Claude 3.5 Sonnet leads DeepSeek V2 Lite on 3 of the 3 benchmarks they both report (HumanEval, MMLU, GSM8K). DeepSeek V2 Lite shipped 35 days before Claude 3.5 Sonnet, so benchmark comparisons should account for the intervening progress.

Context windows are 200k (Claude 3.5 Sonnet) vs 32k (DeepSeek V2 Lite). Claude 3.5 Sonnet is closed, while DeepSeek V2 Lite is open weight.

On HumanEval, Claude 3.5 Sonnet leads at 92% vs DeepSeek V2 Lite at 57.3%. On MMLU, Claude 3.5 Sonnet leads at 90.4% vs DeepSeek V2 Lite at 55.7%. On GSM8K, Claude 3.5 Sonnet leads at 96.4% vs DeepSeek V2 Lite at 72%.