Demonstrates that entropy-based uncertainty is insufficient for safe selective prediction and proposes combining it with correctness probes.
March 24, 2026
Original Paper
Entropy Alone is Insufficient for Safe Selective Prediction in LLMs
arXiv · 2603.21172
The Takeaway
Selective prediction (abstention) is critical for LLM safety. This paper identifies a specific failure mode in entropy-only methods and provides a more reliable scoring mechanism to ensure models trustworthily 'know what they don't know' across QA tasks.
From the abstract
Selective prediction systems can mitigate harms resulting from language model hallucinations by abstaining from answering in high-risk cases. Uncertainty quantification techniques are often employed to identify such cases, but are rarely evaluated in the context of the wider selective prediction policy and its ability to operate at low target error rates. We identify a model-dependent failure mode of entropy-based uncertainty methods that leads to unreliable abstention behaviour, and address it