Generative models are now extracting the geometry of the physical world to help control the scorching plasma inside a nuclear fusion reactor.
April 25, 2026
Original Paper
Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning
arXiv · 2604.20627
The Takeaway
Reinforcement learning often fails in physical tasks because it cannot figure out which specific action led to a positive result. This method uses world geometry to shape rewards, allowing the AI to learn complex maneuvers in high-stakes environments like a Tokamak. Traditional control systems rely on rigid mathematical models that struggle with the chaotic nature of fusion. The AI can now navigate sparse reward settings where it only receives feedback after a successful long-term sequence. This move from theory to practical energy generation signals a major shift in how we manage the most complex machines on Earth.
From the abstract
The temporal lag between actions and their long-term consequences makes credit assignment a challenge when learning goal-directed behaviors from data. Generative world models capture the distribution of future states an agent may visit, indicating that they have captured temporal information. How can that temporal information be extracted to perform credit assignment? In this paper, we formalize how the temporal information stored in world models encodes the underlying geometry of the world. Lev