An analyst can use perfectly legitimate statistical methods to prove a political point simply by choosing which professional options to use.
Large language models acting as analysts consistently selected different statistical paths based on their assigned political ideology. Even though every choice was defensible and professional, the final results always aligned with the starting political bias. We trust data-driven reports to be the objective truth that settles political arguments. This study shows that there is enough room in the rules of math and science to steer any dataset toward a desired conclusion. It means that trusting the data is impossible if the person analyzing it has a specific goal in mind.
Ideology-Driven Specification Selection in Applied Econometrics
SSRN · 6722712
We use 600 large language model agents as synthetic analysts across four applied economics papers. We organise agents into balanced cells of 50 runs per ideology (left, neutral, right) and introduce the persona through an identity-framing clause in an otherwise identical system prompt. Each agent chooses from a pre-enumerated menu of defensible specifications and retrieves coefficients from a frozen database, so the only margin of variation is selection among legitimate specifications. In every