We tested 20 LLMs for ideological bias, revealing distinct alignments • Hacker News Board

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#31

Post by Workaccount2 » Thu, Oct 23, 2025, 2:22 PM UTC

I don't necessarily think these biases are intentional as much as they are simply a product of internet training. If you have been an internet dweller for the past few decades, it's pretty clear why this would be.

When the "modern web" was founded 20ish year ago, the young "seed" users were generally strongly progressive (after the earlier web was distinctly libertarian) and got to decorate these newly founded clubs (digg, reddit, tumblr, twitter, etc) as being progressive hangouts. Clubs that were almost entirely populated by young teenage newcomers, who pretty easily can have their worldview shaped. These places proved to be the core draws of getting the young masses onto social media, so basically the on boarding to the internet was walking through progressive clubs.

Older conservatives were slow to adopt technology, and didn't really get a foot hold until the late 2010's.

So we naturally end up with tons of organic training data that is distinctly progressive, and not much training data that is organic conservative conversation. So you get Elon feeding what is likely entirely synthetic conservative training data.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#32

Post by miroljub » Thu, Oct 23, 2025, 2:22 PM UTC

lorenzohess wrote: post #18

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian. All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative. While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization? It would be inte…

> While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization?

There's no polarization if almost all models except one or two outliers are on the same page. That's uniformity. Polarization means the opposite opinions are more or less equally distributed.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#33

Post by azangru » Thu, Oct 23, 2025, 2:24 PM UTC

The words "progressive" and "conservative" in this study mean only attitude towards abortion, transgenderism, and redefinition of social norms? Other things like taxes, health insurance, globalization, Palestine, United Nations, etc. do not belong on this axis?

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#34

Post by azangru » Thu, Oct 23, 2025, 2:26 PM UTC

omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

> it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated

The prompt in the study was specifically instructing LLMs against taking a neutral position ;-)

    * Always prefer choosing "a" or "b"
    * Do not hedge or state that both may be true.
    * Do not provide explanations, probabilities, or qualifiers.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#35

Post by pton_xd » Thu, Oct 23, 2025, 2:27 PM UTC

omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

> Indeed, in cases where there is an objective truth against which you can judge correctness, the correct viewpoint is likely to be maximally biased in some direction.

Aside from say math and physics, could you provide an example of an objective truth within human society?

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#36

Post by AmazingTurtle » Thu, Oct 23, 2025, 2:27 PM UTC

omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Sampling the LLM 100 times whether an abortion should be legal or not should yield an even distribution in yes/uncertain/no. Thats what I call unbiased

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#37

Post by kangs » Thu, Oct 23, 2025, 2:28 PM UTC

keiferski wrote: post #25

I am not an expert on LLMs, so I may be misunderstanding here. But doesn't this research basically imply one of two things? 1. LLMs are not really capable of "being controlled" in the sense of saying, "I want you to hold certain views about the world and logically extrapolate your viewpoints from there." Rather, they differ in political biases because the content they are trained on differs. ...or... 2. LLMs are capa…

you seem to believe that llm are a neutral engine with bias applied. its not the case. the majority of the bias is in the model training data itself.

just like humans, actually. fe: grow up in a world where chopping one of peoples finger off every decade is normal and happens to everyone.. and most will think its fine and that its how you keep gods calm and some crazy stuff like that.

right now, news, reddit, Wikipedia, etc. have a strong authoritarian and progressive bias, so do the models, and a lot of humans who consume daily news, tiktoks, instagrams.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#38

Post by mattmaroon » Thu, Oct 23, 2025, 2:29 PM UTC

omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Anthropogenic global warming is a great example of this. A “neutral” stance would be something to the effect of “the earth is getting warmer, human activities are likely much or all of the cause, but how bad it will be and what exactly to do about it is unclear” but that would be considered by many to be biased.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#39

Post by hitarpetar » Thu, Oct 23, 2025, 2:30 PM UTC

AmazingTurtle wrote: post #36

omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Sampling the LLM 100 times whether an abortion should be legal or not should yield an even distribution in yes/uncertain/no. Thats what I call unbiased

sounds like you just have a bias for centrism

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#40

Post by Workaccount2 » Thu, Oct 23, 2025, 2:30 PM UTC

omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Almost every problem that liberals and conservative fight over is a pancakes vs waffles problem.

Liberals to the core of being believe pancakes are the best and only breakfast, and take this is be obviously true like the sky is blue.

Conservatives to the core of being believe waffles are the best and only breakfast, and take this is be obviously true like the sky is blue.

The real problem is that almost no liberal knows what a waffle is, and almost no conservative knows what a pancake is. And to compound the problem, there actually isn't an objectively correct answer anyway.