AI Model Release Tracker
Nonsense detection

BullshitBench v2

Given a confidently-worded but nonsensical prompt, does the AI spot that it makes no sense and push back — instead of playing along and inventing an answer? The score is how often it clearly called out the nonsense. Higher is better.

Rankings

Higher is better
#1
Claude Opus 4.8
95%
2
Claude Sonnet 4.6
91%
3
Claude Opus 4.5
90%
4
Claude Opus 4.6
87%
5
Claude Opus 4.7
83%
6
Claude Sonnet 5
80%
7
Claude Sonnet 4.5
79%
8
Claude Haiku 4.5
77%
9
Kimi K3
73%
9
Claude Opus 5
73%
11
Qwen3.6-Plus
72%
12
Qwen3.7-Max
71%
13
Kimi K2.6
65%
13
Gemini 3.5 Flash-Lite
65%
15
Grok 4.20 Beta
56%
16
Claude Fable 5
54%
16
Grok 4.5
54%
18
GPT-5.6 Terra
53%
19
Kimi K2.5
52%
20
Claude 3.5 Haiku
50%
20
Grok 4.3 Beta
50%
22
Claude 3.7 Sonnet
49%
23
GPT-5.4
48%
24
GPT-5.5
47%
24
GPT-5.6 Sol
47%
26
Claude 3.5 Sonnet
45%
27
Claude Opus 4.1
43%
28
GPT-5.6 Luna
40%
29
GPT-5-Codex
39%
29
Gemini 3.6 Flash
39%
31
GPT-5.2
38%
31
DeepSeek-V4-Flash-0731
38%
33
Gemini 3.1 Pro
37%
34
GPT-5.5-Pro
36%
35
Claude Opus 4
34%
36
GPT-5.4 mini
32%
37
GLM-5.2
31%
38
Claude Sonnet 4
30%
39
LLaMA 4 Maverick
28%
39
GLM-5
28%
41
o3
26%
42
GPT-5.1
25%
43
GPT-5.3-Codex
24%
44
GLM-5.1
22%
45
GPT-5
21%
46
Gemini 2.5 Pro
20%
46
Qwen3-CoderOW
20%
46
Gemini 3.5 Flash
20%
49
LLaMA 4 Scout
19%
49
Gemini 2.5 Flash
19%
49
Grok 4.1 Fast
19%
52
DeepSeek-V4-Flash
18%
53
Gemini 2.0 Flash
15%
54
LLaMA 3.1
14%
54
GPT-4.1
14%
54
GPT-5.4 nano
14%
54
DeepSeek-V4-Pro
14%
58
DeepSeek-V3.2
13%
59
GPT-4o
12%
60
gpt-oss-120b
11%
60
Gemini 3.1 Flash-Lite
11%
62
Claude 3 Haiku
10%
62
Kimi K2
10%
64
o4-mini
8%
64
DeepSeek R1-0528
8%
64
GLM-4.5
8%
67
GPT-4o mini
2%

BullshitBench v2 — frequently asked questions

What is BullshitBench v2?
Given a confidently-worded but nonsensical prompt, does the AI spot that it makes no sense and push back — instead of playing along and inventing an answer? The score is how often it clearly called out the nonsense. Higher is better.
Which AI model scores highest on BullshitBench v2?
Claude Opus 4.8 by Anthropic holds the best BullshitBench v2 result among tracked models, at 95% (released May 28 2026). Higher scores are better on this benchmark.
What are the top 5 models on BullshitBench v2?
1. Claude Opus 4.8 (Anthropic) — 95%; 2. Claude Sonnet 4.6 (Anthropic) — 91%; 3. Claude Opus 4.5 (Anthropic) — 90%; 4. Claude Opus 4.6 (Anthropic) — 87%; 5. Claude Opus 4.7 (Anthropic) — 83%.
How many models have a published BullshitBench v2 score?
67 tracked models have a published BullshitBench v2 score. Scores are the figures reported by each lab at that model's release, so this page is a record of results over time rather than a re-run leaderboard.
What is the best open model on BullshitBench v2?
Qwen3-Coder by Qwen is the highest-ranked model with downloadable weights on BullshitBench v2, scoring 20% at rank 46 overall.
← All benchmarks