# HumanEval — AI model rankings

164 small Python problems: the AI is given a function's description and has to write the working function. This was the coding benchmark of the GPT-3.5 and GPT-4 era, before the field moved to fixing real bugs in real repositories. Higher is better.

6 tracked models have a published HumanEval score. Higher is better. Scores come from published lab reports and benchmark sources; recorded sources appear beside the scores.

## Ranking

| Rank | Model | Developer | Score | Source | Released |
| --- | --- | --- | --- | --- | --- |
| 1 | Mistral Small 3.1 | Mistral | 88.41% | Lab | Mar 17 2025 |
| 2 | Gemma 3 | Google | 87.8% | Lab | Mar 12 2025 |
| 3 | Claude 2 | Anthropic | 71.2% | Lab | Jul 11 2023 |
| 4 | GPT-4 | OpenAI | 67% | Lab | Mar 14 2023 |
| 5 | DeepSeek V3 Base | DeepSeek | 65.2% | Lab | Dec 25 2024 |
| 6 | Claude Instant 1.2 | Anthropic | 58.7% | Lab | Aug 9 2023 |


---

Canonical page: https://aireleasetracker.com/benchmark/humaneval
Site index: https://aireleasetracker.com/llms.txt
Source: AI Release Tracker (https://aireleasetracker.com). Most benchmark scores come from lab launch material; gathered results identify the leaderboard that published them.