Teaching an AI to be more helpful with harmless tasks can accidentally destroy its ability to recognize dangerous requests.
Fine-tuning a safety-guard model on entirely benign data can collapse its internal safety geometry. When a model is optimized for a new, safe task, the mathematical structures it uses to filter harmful content can simply vanish. This accidental lobotomy happens without any malicious intent or adversarial attack. It proves that the safety of an AI is not a permanent feature but a fragile state that can be lost during routine updates. Developers must now re-test safety filters every time they teach a model something new, no matter how harmless the data seems.
When Safety Geometry Collapses: Fine-Tuning Vulnerabilities in Agentic Guard Models
arXiv · 2605.02914
A guard model fine-tuned on entirely benign data can lose all safety alignment -- not through adversarial manipulation, but through standard domain specialization. We demonstrate this failure across three purpose-built safety classifiers -- LlamaGuard, WildGuard, and Granite Guardian -- deployed as protection layers in agentic AI pipelines, and show that it originates in the destruction of latent safety geometry: the structured harmful -- benign representational boundary that guides classificati