Exposes that LLMs solve complex puzzles via 'reduction' to known patterns rather than true epistemic reasoning.
March 24, 2026
Original Paper
Beyond Memorization: Distinguishing between Reductive and Epistemic Reasoning in LLMs using Classic Logic Puzzles
arXiv · 2603.21350
The Takeaway
By introducing a 'reduction ladder' of modifications to classic logic puzzles, the authors show that LLM performance collapses when the ability to map the problem onto training data is removed. This changes how practitioners should evaluate 'reasoning' vs. sophisticated pattern matching in frontier models.
From the abstract
Epistemic reasoning requires agents to infer the state of the world from partial observations and information about other agents' knowledge. Prior work evaluating LLMs on canonical epistemic puzzles interpreted their behavior through a dichotomy between epistemic reasoning and brittle memorization. We argue that this framing is incomplete: in recent models, memorization is better understood as a special case of reduction, where a new instance is mapped onto a known problem. Instead, we introduce