Just because your model converged during fine-tuning doesn't mean it actually learned your data.
April 14, 2026
Original Paper
Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models
arXiv · 2604.10079
The Takeaway
The study identifies 'Incomplete Learning,' where LLMs fail to internalize specific subsets of training data even after full convergence. This suggests that the standard SFT recipe is fundamentally leaky and requires new methods to ensure data 'sticks' reliably.
From the abstract
Supervised Fine-Tuning (SFT) is the standard approach for adapting large language models (LLMs) to downstream tasks. However, we observe a persistent failure mode: even after convergence, models often fail to correctly reproduce a subset of their own supervised training data. We refer to this behavior as the Incomplete Learning Phenomenon(ILP). This paper presents the first systematic study of ILP in LLM fine-tuning. We formalize ILP as post-training failure to internalize supervised instances a