# Okapi Multilingual MMLU — AI model rankings

five-shot accuracy on the Okapi translations of MMLU used in the Claude 3 model card

6 tracked models have a published Okapi Multilingual MMLU score. Higher is better. Scores come from published lab reports and benchmark sources; recorded sources appear beside the scores.

## Ranking

| Rank | Model | Developer | Score | Source | Released |
| --- | --- | --- | --- | --- | --- |
| 1 | Claude 3 Opus | Anthropic | 79.1% | Lab | Mar 4 2024 |
| 2 | Claude 3 Sonnet | Anthropic | 69% | Lab | Mar 4 2024 |
| 3 | Claude 3 Haiku | Anthropic | 65.2% | Lab | Mar 4 2024 |
| 4 | Claude 2.1 | Anthropic | 63.4% | Lab | Nov 21 2023 |
| 5 | Claude 2 | Anthropic | 63.1% | Lab | Jul 11 2023 |
| 6 | Claude Instant 1.2 | Anthropic | 61.2% | Lab | Aug 9 2023 |


---

Canonical page: https://aireleasetracker.com/benchmark/okapi-mmlu
Site index: https://aireleasetracker.com/llms.txt
Source: AI Release Tracker (https://aireleasetracker.com). Most benchmark scores come from lab launch material; gathered results identify the leaderboard that published them.