Stop treating language as discrete tokens; continuous diffusion just proved it can beat autoregressive models.
April 15, 2026
Original Paper
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
arXiv · 2604.11748
The Takeaway
We've long assumed 'next-token prediction' was the only way to build competitive LLMs because language is discrete. LangFlow flips this, showing that mapping text into continuous space and using diffusion can match or exceed standard autoregressive baselines in zero-shot transfer. This isn't just a research curiosity—it's a fundamental challenge to the GPT architecture. By moving beyond the discrete bottleneck, we might finally solve the sampling speed and 'hallucination-by-probability' issues inherent in traditional decoders. This opens the door to a new generation of non-autoregressive models that are cheaper to run and easier to control.
From the abstract
Continuous diffusion has been the foundation of high-fidelity, controllable, and few-step generation of many data modalities such as images. However, in language modeling, prior continuous diffusion language models (DLMs) lag behind discrete counterparts due to the sparse data space and the underexplored design space. In this work, we close this gap with LangFlow, the first continuous DLM to rival discrete diffusion, by connecting embedding-space DLMs to Flow Matching via Bregman divergence, alo