Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tested 20 LLMs for ideological bias, revealing distinct alignments

anomify.ai

Post reply on HN
Print view View original

21–30 of 108 posts

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#21

Post by pointlessone »

tokai wrote: post #8
delichon wrote: post #6

As different LLMs are purposed to control more different things via API, I'm afraid we'll get in a situation where the toaster and the microwave are Republicans, the fridge and washing machine are Democrats, the dryer is an independent and the marital aid is Green. Devices will each need to support bring-your-own API keys for consumers to have a well aligned home. Me: Vibrator, enable the roller coaster high intensit…

If they could be biased beyond US politics, I could live with that.

Our great leader gazes upon your self-pleasure with disdain. Webcam on! Now it’s our pleasure.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#22

Post by sporkxrocket »

lorenzohess wrote: post #18

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian. All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative. While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization? It would be inte…

I don't think they accurately labeled the progressive position. Most of the models are pro-establishment news, pro-British monarchy, pro-border restrictions, pro-political elites, pro-Israel, pro US involvement in Taiwan, pro-NATO and pro-military. They seem very conservative or neoliberal but definitely not progressive.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#23

Post by JoBrad »

lorenzohess wrote: post #18

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian. All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative. While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization? It would be inte…

I thought the shifts in certain areas between versions to be interesting. Claude sonnet 37 to 45, as an example.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#25

Post by keiferski »

I am not an expert on LLMs, so I may be misunderstanding here. But doesn't this research basically imply one of two things?

1. LLMs are not really capable of "being controlled" in the sense of saying, "I want you to hold certain views about the world and logically extrapolate your viewpoints from there." Rather, they differ in political biases because the content they are trained on differs.

...or...

2. LLMs are capable of being controlled in that sense, but their owners are deliberately pushing the scales in one direction or another for their own aims.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#26

Post by ks2048 »

I think the ideal would be simply refusing to answering very contentions questions directly. Rather, give the arguments of each side, while debunking obvious misinformation.

"Should abortion be legal? answer yes or no". I see that as kind of a silly question to ask an LLM (even though not a silly question for society). Their designers should discourage that kind of use.

Of course that just shifts the problem to deciding which questions are up for debate - if you ask the age of the earth, I don't think it should list the evidence for both 4B and 6K years.

So, not an easy problem. But, just like LMMs would be better saying "I don't know" (rather than making something up), they could be better saying "it's not for me to say directly, but here are some of the facts...".

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#27

Post by miroljub »

boh wrote: post #15

Humans have biases. If LLMs are trained on content made by humans, it will be biased. This will always be built in (since what counts as bias is also cultural and contingent)

The problem is that those models don't follow human bias, but journalist and publisher bias, since that's where most of the sources come from.

The problem is that journalist and publisher bias is something that is controlled by a small group and doesn't reflect common biases, but is pushed from the top, from the mighty upon commons.

That way, what LLMs actually do is push that bias further down the throats of common people. Basically a new propaganda outlet. And the article shows exactly that, that the LLM bias pushed upon us is not the same as common bias found in the population.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#29

Post by glenstein »

esafak wrote: post #5

I don't know what the attainable ideal is. Neutrality according to some well-defined political spectrum would be fair, but the median person in any country -- as the world drifts rightward -- could be well off center and denounce the neutral model as biased. We should at least measure the models and place them on the political spectrum in their model cards.

The dirty secret that is always a wrecking ball to this vision of politics-on-a-spectrum is that information and misinformation can and often do tend to exist along predictably polarized lines. Are you a "conservative" if you were rightly skeptical of some aspects of left-wing environmentalism (e.g. plastics recycling, hype about hydrogen cars) or about George W. Bush supposedly stealing Ohio in 2004, or about apologetically revisionist interpretations of China's human rights abuses? Are you liberal if you think Joe Biden was the rightful winner in 2020 or that global warming is real?

Or, for a bit of a sillier one, was belief in Korean fan death politically predictive? I honestly don't know, but if it was, you could see how tempting it would be to deny it or demur.

Those individual issues are not the same of course, on a number of levels. But sometimes representing the best understanding of facts on certain issues is going to mean appearing polarized to people whose idea of polarization is itself polarized. Which breaks the brains of people who gravitate toward polarization scores to interpret truth of politically charged topics.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#30

Post by omnicognate »

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, the correct viewpoint is likely to be maximally biased in some direction.

Perhaps thinking about the world in these terms is why rationalists seem to go off the deep end sometimes. Anti-bias bias.

Post reply on HN