GSM8K

Grade-school maths word problems that take a few steps of arithmetic to work through. It separated the models of 2022 and 2023 sharply, then saturated. One caveat on the historical numbers: OpenAI included part of the GSM8K training set in GPT-4's pre-training mix, so GPT-4's score is not a clean few-shot result. Higher is better.

GPT-4 by OpenAI holds the best GSM8K result among tracked models, at 92% (released Mar 14 2023). Higher scores are better on this benchmark.

1. GPT-4 (OpenAI) — 92%; 2. Claude 2 (Anthropic) — 88%; 3. Claude Instant 1.2 (Anthropic) — 86.7%.

3 tracked models have a published GSM8K score. Scores are the figures reported by each lab at that model's release, so this page is a record of results over time rather than a re-run leaderboard.