Solves the 'vanishing gradient' problem in 3D Gaussian Splatting (3DGS) tracking by optimizing in the frequency domain using spectral moments.
March 26, 2026
Original Paper
SpectralSplats: Robust Differentiable Tracking via Spectral Moment Supervision
arXiv · 2603.24036
The Takeaway
Standard 3DGS tracking fails when there is no pixel overlap between the model and the target; this paper creates a global basin of attraction that ensures valid gradients across the entire image. This makes 3DGS-based tracking significantly more robust for real-world robotics and video applications where large camera movements are common.
From the abstract
3D Gaussian Splatting (3DGS) enables real-time, photorealistic novel view synthesis, making it a highly attractive representation for model-based video tracking. However, leveraging the differentiability of the 3DGS renderer "in the wild" remains notoriously fragile. A fundamental bottleneck lies in the compact, local support of the Gaussian primitives. Standard photometric objectives implicitly rely on spatial overlap; if severe camera misalignment places the rendered object outside the target'