Replacing the standard next-token guess with a set of multiple learned options boosted AI math accuracy from 51% to 70%.
April 23, 2026
Original Paper
OLLM: Options-based Large Language Models
arXiv · 2604.19087
The Takeaway
Options-based Large Language Models challenge the core architecture of almost every AI in existence today. Instead of predicting just one word at a time, the model considers a set of plausible future paths indexed by a latent variable. This allows the AI to explicitly model multiple futures before committing to one. The jump in math performance shows that next-token prediction is a ceiling that we can now break. It suggests the next generation of AI will be much better at long-term planning and complex logic.
From the abstract
We introduce Options LLM (OLLM), a simple, general method that replaces the single next-token prediction of standard LLMs with a \textit{set of learned options} for the next token, indexed by a discrete latent variable. Instead of relying on temperature or sampling heuristics to induce diversity, OLLM models variation explicitly: a small latent space parametrizes multiple plausible next-token options which can be selected or searched by a downstream policy. Architecturally, OLLM is a lightweight