Compare AI models

Specifications
Parameters
—7B
Context window
32k32k
Benchmarks
HumanEval
74.4%38.4%
MATH
53.2%13.2%
Overview
CompanyGoogleMistral
Release dateFeb 8 2024May 22 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.0 Ultra leads Mistral 7B v0.3 on 2 of the 2 benchmarks they both report (HumanEval, MATH). Gemini 1.0 Ultra shipped 104 days before Mistral 7B v0.3, so benchmark comparisons should account for the intervening progress.

Context windows are 32k (Gemini 1.0 Ultra) vs 32k (Mistral 7B v0.3). Gemini 1.0 Ultra is closed, while Mistral 7B v0.3 is open weight.

On HumanEval, Gemini 1.0 Ultra leads at 74.4% vs Mistral 7B v0.3 at 38.4%. On MATH, Gemini 1.0 Ultra leads at 53.2% vs Mistral 7B v0.3 at 13.2%.