Which AI model scores highest on HellaSwag?

GPT-4 by OpenAI holds the best HellaSwag result among tracked models, at 95.3% (released Mar 14 2023). Higher scores are better on this benchmark.

What are the top 5 models on HellaSwag?

1. GPT-4 (OpenAI) — 95.3%; 2. Gemini 1.5 Pro (Google) — 92.5%; 3. Gemini 1.0 Ultra (Google) — 87.8%; 4. Gemini 1.5 Flash (Google) — 86.5%; 5. Gemma 2 (Google) — 86.4%.

How many models have a published HellaSwag score?

9 tracked models have a published HellaSwag score. Scores are collected from published lab reports and benchmark sources, with recorded sources credited alongside the results. We do not re-run the evaluations.

What is the best open model on HellaSwag?

Gemma 2 by Google is the highest-ranked model with downloadable weights on HellaSwag, scoring 86.4% at rank 5 overall.

Reasoning

HellaSwag

commonsense completion of short scenarios

Rankings

Higher is better

HellaSwag — frequently asked questions

: commonsense completion of short scenarios
: GPT-4 by OpenAI holds the best HellaSwag result among tracked models, at 95.3% (released Mar 14 2023). Higher scores are better on this benchmark.
: 1. GPT-4 (OpenAI) — 95.3%; 2. Gemini 1.5 Pro (Google) — 92.5%; 3. Gemini 1.0 Ultra (Google) — 87.8%; 4. Gemini 1.5 Flash (Google) — 86.5%; 5. Gemma 2 (Google) — 86.4%.
: 9 tracked models have a published HellaSwag score. Scores are collected from published lab reports and benchmark sources, with recorded sources credited alongside the results. We do not re-run the evaluations.
: Gemma 2 by Google is the highest-ranked model with downloadable weights on HellaSwag, scoring 86.4% at rank 5 overall.