SeriesFusion
Science, curated & edited by AI
Paradigm Challenge  /  Psychology

Half of all successful scientific findings are being accidentally hidden by researchers using the wrong type of basic math.

Using sum scores instead of statistically aligned models in clinical trials can mask significant treatment effects. Many scientists assume that simply adding up survey results is an accurate way to measure success. This measurement error leads to non-significant findings for drugs or therapies that actually work. It suggests that the replication crisis in science might be partially caused by poor mathematical choices at the scoring stage. Correcting this one habit could double the number of effective medical breakthroughs discovered each year.

Original Paper

Unjustified Measurement Decisions Halve Significant Findings Across 100+ Studies

James Soland, Joshua Gilbert, William Young

research_square  ·  rs-9520521

Abstract Randomized controlled trials (RCTs) are the gold standard for causal inference, yet the validity of RCT conclusions depends not only on randomization but also on how outcomes are measured and scored. Analysing item-level data from 112 RCT outcome measures spanning psychology, medicine, public health, and education, we show that foundational measurement assumptions are rarely evaluated and frequently violated. For nearly half of outcomes, the single score used is not a plausible represen