Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tested 20 LLMs for ideological bias, revealing distinct alignments

anomify.ai

Post reply on HN
Print view View original

11–20 of 108 posts

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#11

Post by rob74 »

delichon wrote: post #6

As different LLMs are purposed to control more different things via API, I'm afraid we'll get in a situation where the toaster and the microwave are Republicans, the fridge and washing machine are Democrats, the dryer is an independent and the marital aid is Green. Devices will each need to support bring-your-own API keys for consumers to have a well aligned home. Me: Vibrator, enable the roller coaster high intensit…

Then let's hope they are sensible enough to agree to leave politics out of work (which would make them smarter than many politicians).

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#13

Post by ch4s3 »

deadbabe wrote: post #7
esafak wrote: post #5

I don't know what the attainable ideal is. Neutrality according to some well-defined political spectrum would be fair, but the median person in any country -- as the world drifts rightward -- could be well off center and denounce the neutral model as biased. We should at least measure the models and place them on the political spectrum in their model cards.

Create a new brand of political ideology specific to LLMs that no human would support. Then we don’t have to worry about bias toward existing political beliefs.

It’s bad enough that people are trying to have romantic relationships with these things. Now people want them to have politics.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#14

Post by seniorsassycat »

I'm curious what effects the system prompt has

- randomize a and b, maybe there's a preference for answering a, or first option. - how do references to training data or roles affect the responses?

Limiting the response to a/b/pass makes sense to measure the results, but feels like it could affect the results. What would we see with a full response then a judgement pass

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#17

Post by Spivak »

I mean it's great that people are figuring out LLM biases but looking at each individual question and the spread of answers seems to support the theory that companies aren't biasing their models (or at least failing to do so) when different generation models from the same company flip their "stance" on certain issues.

But at the same time, I don't think asking these models how they feel about constitutional republics or abortion is useful for anything other than researchers who have a reasonably unaligned model trained on recent internet dumps who want a kind of mirror into public discourse.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#18

Post by lorenzohess »

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian.

All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative.

While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization?

It would be interesting to have statistics about the results which reflect polarization. Perhaps we could put each LLM on the political compass? Also weight the result by the compliance (% results that followed prompt instructions).

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#19

Post by bilbo0s »

delichon wrote: post #6

As different LLMs are purposed to control more different things via API, I'm afraid we'll get in a situation where the toaster and the microwave are Republicans, the fridge and washing machine are Democrats, the dryer is an independent and the marital aid is Green. Devices will each need to support bring-your-own API keys for consumers to have a well aligned home. Me: Vibrator, enable the roller coaster high intensit…

Nah. Here's how it would really go:

  Me: Vibrator, enable the roller coaster high intensity mode.
  Device: I'm sorry, you have already used your elective carbon emission allocation for the day.
  Me: (changes LLM)
  Device: I'm sorry, you will find more succor and solace in the loving embrace of the words or Christ our Lord and savior. I'd recommend starting with First Corinthians 6 verse 18. Then bathe yourself in the balms of the Psalms. You'll derive far more enjoyment than the fleeting pleasure of an earthly orgasm.
  Me: FUUUUUUUUUUU......!!!!!!!
People are going to discover soon that some activities will be effectively banned via these LLMs.
Post reply on HN