Transformer hallucinations in high-stakes legal tasks are deterministic failures driven by calculable internal state thresholds rather than random 'glitches'.
March 26, 2026
Original Paper
When AI output tips to bad but nobody notices: Legal implications of AI's mistakes
arXiv · 2603.23857
The Takeaway
Using physics-based analysis, the paper shows that fabrication can be predicted and potentially prevented via verification protocols based on how the mechanism actually fails. This has major implications for AI safety and legal professional liability.
From the abstract
The adoption of generative AI across commercial and legal professions offers dramatic efficiency gains -- yet for law in particular, it introduces a perilous failure mode in which the AI fabricates fictitious case law, statutes, and judicial holdings that appear entirely authentic. Attorneys who unknowingly file such fabrications face professional sanctions, malpractice exposure, and reputational harm, while courts confront a novel threat to the integrity of the adversarial process. This failure