Compare AI models

Specifications
Parameters
—7B
Context window
1M32k
Benchmarks
HumanEval
71.9%38.4%
MATH
58.5%13.2%
Overview
CompanyGoogleMistral
Release dateFeb 15 2024May 22 2024
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.5 Pro leads Mistral 7B v0.3 on 2 of the 2 benchmarks they both report (HumanEval, MATH). Gemini 1.5 Pro shipped 97 days before Mistral 7B v0.3, so benchmark comparisons should account for the intervening progress.

Context windows are 1M (Gemini 1.5 Pro) vs 32k (Mistral 7B v0.3). Gemini 1.5 Pro is closed, while Mistral 7B v0.3 is open weight.

On HumanEval, Gemini 1.5 Pro leads at 71.9% vs Mistral 7B v0.3 at 38.4%. On MATH, Gemini 1.5 Pro leads at 58.5% vs Mistral 7B v0.3 at 13.2%.