A mathematical blind spot in almost all supervised learning makes it impossible for models to be perfectly robust against adversarial attacks.
April 24, 2026
Original Paper
Supervised Learning Has a Necessary Geometric Blind Spot: Theory, Consequences, and Minimal Repair
arXiv · 2604.21395
The Takeaway
Vulnerabilities like texture bias and sensitivity to tiny pixel changes are not errors in code, but necessary results of the training objective itself. The standard way we teach AI to minimize error forces the model to ignore certain geometric features of the data. This means that no amount of extra data or better hardware can fix these flaws without changing the fundamental math of the learning process. Researchers have spent years trying to patch these holes, but they are actually baked into the foundation of empirical risk minimization. Real safety in AI will require moving away from current supervised learning paradigms entirely. We are hitting a theoretical ceiling on how reliable our current models can ever become.
From the abstract
We prove that empirical risk minimisation (ERM) imposes a necessary geometric constraint on learned representations: any encoder that minimises supervised loss must retain non-zero Jacobian sensitivity in directions that are label-correlated in training data but nuisance at test time. This is not a contingent failure of current methods; it is a mathematical consequence of the supervised objective itself. We call this the geometric blind spot of supervised learning (Theorem 1), and show it holds