A dual-path architecture that combines speculative speech-to-speech prefixes with cascaded LLM continuations for zero-latency, high-quality dialogue.
March 25, 2026
Original Paper
RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialogue
arXiv · 2603.23346
The Takeaway
It resolves the 'latency vs. quality' trade-off in voice assistants. By streaming a fast S2S 'prefix' immediately while a slower LLM computes the complex response, it achieves human-like response times with the intelligence of large-scale models.
From the abstract
Real-time spoken dialogue systems face a fundamental tension between latency and response quality. End-to-end speech-to-speech (S2S) models respond immediately and naturally handle turn-taking, backchanneling, and interruption, but produce semantically weaker outputs. Cascaded pipelines (ASR -> LLM) deliver stronger responses at the cost of latency that grows with model size. We present RelayS2S, a hybrid architecture that runs two paths in parallel upon turn detection. The fast path -- a duplex