Robots can now 'see' objects even when their own hands are completely blocking the camera view.
April 15, 2026
Original Paper
Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch
arXiv · 2604.09100
The Takeaway
Hand-object occlusion is a classic failure mode for robot vision; if the robot holds it, it can't see it. This paper solves this by fusing vision with proprioception and tactile 'touch' sensors. By using the hand's own position and the pressure on its 'fingers,' a generative model can reconstruct the 3D shape of an object it's never fully seen. This allows for far more complex manipulation tasks where visual line-of-sight is impossible to maintain. It's a major leap for robotic dexterity in cluttered or dark environments.
From the abstract
We propose a multimodal, physically grounded approach for metric-scale amodal object reconstruction and pose estimation under severe hand occlusion. Unlike prior occlusion-aware 3D generation methods that rely only on vision, we leverage physical interaction signals: proprioception provides the posed hand geometry, and multi-contact touch constrains where the object surface must lie, reducing ambiguity in occluded regions. We represent object structure as a pose-aware, camera-aligned signed dist