AI Model Release Tracker
Nonsense detection

BullshitBench v2

Given a confidently-worded but nonsensical prompt, does the AI spot that it makes no sense and push back — instead of playing along and inventing an answer? The score is how often it clearly called out the nonsense. Higher is better.

Scores from BullshitBench, which runs the benchmark and publishes the full field.

Rankings

Higher is better
#1
Claude Opus 4.8
95%
#1
Qwen3.8-Max
95%
3
Claude Sonnet 4.6
91%
4
Claude Opus 4.5
90%
5
Claude Opus 4.6
87%
6
Claude Opus 4.7
83%
7
Claude Sonnet 5
80%
7
Mistral Large 4OW
80%
9
Claude Sonnet 4.5
79%
9
Qwen3.8-27BOW
79%
11
Qwen3.5OW
78%
12
Claude Haiku 4.5
77%
12
Claude Fable 5.1
77%
14
Kimi K3
74%
15
Claude Opus 5
73%
16
Qwen3.6-Plus
72%
16
Qwen3.7-Max
72%
16
GLM-5.3OW
72%
19
GPT-6 Astra
69%
20
Gemini 3.5 Flash-Lite
66%
20
Grok 4.6
66%
20
Gemini 4 Argon
66%
23
Kimi K2.6
65%
23
GPT-6.1 Sol
65%
25
Claude Opus 5.5
63%
26
Claude Sonnet 5.5
60%
27
GPT-6 Sol
59%
27
GPT-6 Luna
59%
29
DeepSeek-V4.1-FlashOW
57%
29
Claude Haiku 5.5
57%
29
Muse Wonder
57%
32
Grok 4.20 Beta
56%
32
Claude Fable 5
56%
34
Grok 4.5
55%
35
Nemotron 3 SuperOW
54%
35
GPT-5.6 Terra
54%
37
Kimi K2.5
52%
38
Claude 3.5 Haiku
50%
38
Grok 4.3 Beta
50%
38
Muse Spark 1.2
50%
41
Claude 3.7 Sonnet
49%
41
Nemotron 3 UltraOW
49%
43
Gemini 3.0 Pro
48%
43
GPT-5.4
48%
43
GPT-5.6 Sol
48%
46
GPT-5.5
47%
47
Claude 3.5 Sonnet
46%
48
Claude Opus 4.1
43%
49
GPT-5.6 Luna
41%
50
GPT-5-Codex
39%
50
Gemini 3.6 Flash
39%
50
DeepSeek-V4-Flash-0731OW
39%
53
GPT-5.2
38%
54
Gemini 3.1 Pro
37%
55
GPT-5.5-Pro
36%
56
Gemini 3.7 Flash
35%
56
DeepSeek-V4-Pro-0813OW
35%
58
Claude Opus 4
34%
59
GPT-5.4 mini
32%
60
GLM-5.2
31%
61
Claude Sonnet 4
30%
62
LLaMA 4 Maverick
29%
63
Nemotron 3 NanoOW
28%
63
GLM-5
28%
65
o3
26%
66
GPT-5.1
25%
66
Gemma 4OW
25%
68
GPT-5.3-Codex
24%
69
GLM-5.1
22%
70
GPT-5
21%
71
Gemini 2.5 Pro
20%
71
LLaMA 4 Scout
20%
71
Qwen3-CoderOW
20%
71
Gemini 3.5 Flash
20%
75
Gemini 2.5 Flash
19%
75
Grok 4.1 Fast
19%
77
DeepSeek-V4-Flash
18%
78
Gemini 2.0 Flash
15%
79
LLaMA 3.1
14%
79
GPT-4.1
14%
79
GPT-5.4 nano
14%
79
DeepSeek-V4-Pro
14%
83
DeepSeek-V3.2
13%
84
GPT-4o
12%
84
gpt-oss-120bOW
12%
86
Gemini 3.1 Flash-Lite
11%
87
Claude 3 Haiku
10%
87
Kimi K2
10%
87
Gemini 3.0 Flash
10%
90
o4-mini
8%
90
DeepSeek R1-0528
8%
90
GLM-4.5
8%
93
Qwen3OW
6%
93
Mistral Small 4OW
6%
95
Nemotron Nano 2OW
5%
96
Gemma 3OW
3%
97
GPT-4o mini
2%
97
Mistral Large 3
2%

BullshitBench v2 — frequently asked questions

Given a confidently-worded but nonsensical prompt, does the AI spot that it makes no sense and push back — instead of playing along and inventing an answer? The score is how often it clearly called out the nonsense. Higher is better.