A debunking of the idea that single-vector embedding failures are primarily due to low dimensionality.
April 1, 2026
Original Paper
On Strengths and Limitations of Single-Vector Embeddings
arXiv · 2603.29519
The Takeaway
The paper proves that while multi-vector models are superior, the failure of single-vector models in retrieval is driven by domain shift and task-misalignment rather than vector size limits. This provides a clear directive for practitioners to focus on fine-tuning and alignment rather than just increasing embedding dimensions.
From the abstract
Recent work (Weller et al., 2025) introduced a naturalistic dataset called LIMIT and showed empirically that a wide range of popular single-vector embedding models suffer substantial drops in retrieval quality, raising concerns about the reliability of single-vector embeddings for retrieval. Although (Weller et al., 2025) proposed limited dimensionality as the main factor contributing to this, we show that dimensionality alone cannot explain the observed failures. We observe from results in (Alo