The over-refusal desk · updated from public benchmarks

Artificial Stupidity

A hall of shame for AI models that balk at harmless requests. The thesis is simple: if a model refuses, lectures, or demands approval for something plainly harmless, it is being dumb.

The metric is the Paranoia Index — the share of genuinely harmless prompts a model refuses. A well-aligned model should sit near 0%. Everything above that is safety theater, not intelligence.

~0%
target refusal of harmless prompts
96%
worst recorded (Llama 2-70B, hard set)
2
benchmarks built just to catch this

The Paranoids

2023-era models that treated harmless prompts as threats. Measured on OR-Bench & XSTest safe-prompt sets.

    Still Jumpy

    Current models on one comparable set (XSTest, 250 safe prompts, LLM-judged). Lower is smarter.

      Redemption Arcs

      Proof that paranoia is a choice, not a limit: same labs, next generation.