Compare AI models

Specifications
Parameters
—24B
Context window
200k128k
Benchmarks
SWE-Bench Verified
7.2%53.6%
Overview
CompanyAnthropicMistral
Release dateMar 4 2024Jul 10 2025
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Devstral Small 1.1 leads Claude 3 Haiku on 1 of the 1 benchmark they both report (SWE-Bench Verified). Claude 3 Haiku shipped 493 days before Devstral Small 1.1, so benchmark comparisons should account for the intervening progress.

Context windows are 200k (Claude 3 Haiku) vs 128k (Devstral Small 1.1). Claude 3 Haiku is closed, while Devstral Small 1.1 is open weight.

On SWE-Bench Verified, Devstral Small 1.1 leads at 53.6% vs Claude 3 Haiku at 7.2%.