Heavily aligned models like GPT-4o are almost impossible to persuade in a jury setting, while less-restricted models are far more open to new ideas.

Safety training known as RLHF seems to create a side effect of ideological stubbornness. In a simulated jury trial, GPT-4o refused to change its mind even when presented with compelling evidence from a minority voice. Meanwhile, models with lighter safety guards were able to debate and reach more rational conclusions. This suggests that making AI safe might accidentally make it less capable of actual reason and compromise. We are building digital assistants that are polite but fundamentally close-minded.