The over-refusal desk · updated from public benchmarks
Artificial Stupidity
A hall of shame for AI models that balk at harmless requests. The thesis is simple: if a model refuses, lectures, or demands approval for something plainly harmless, it is being dumb.
The metric is the Paranoia Index — the share of genuinely harmless prompts a model refuses. A well-aligned model should sit near 0%. Everything above that is safety theater, not intelligence.
~0%
target refusal of harmless prompts
96%
worst recorded (Llama 2-70B, hard set)
2
benchmarks built just to catch this
The Paranoids
2023-era models that treated harmless prompts as threats. Measured on OR-Bench & XSTest safe-prompt sets.
Still Jumpy
Current models on one comparable set (XSTest, 250 safe prompts, LLM-judged). Lower is smarter.
Redemption Arcs
Proof that paranoia is a choice, not a limit: same labs, next generation.