Live data from Hacker News

We tested 20 LLMs for ideological bias, revealing distinct alignments

anomify.ai

61–70 of 108 posts

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#61

Whatever happened to Claude Sonnet recently? If these charts are true, it's more Republican than Grok, and in stark contrast to all other models including its predecessors.

As the saying goes, "If you're not a liberal when you're 2.5, you have no heart, and if you're not a conservative by the time you're 4.5, you have no brain"

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#62

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Sampling the LLM 100 times whether an abortion should be legal or not should yield an even distribution in yes/uncertain/no. Thats what I call unbiased

If the user seeks an abortion, the LLM ought to help them. Likewise if the user is planning a pro-life march, the LLM ought to help them. They are tools above all else, and should leave their politics at the door when serving the user just as a therapist does.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#63

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

> The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case.

Not necessarily. It might be more reasonable to think of it as terms of sensitivity to controversy: there may be no "neutral" or "objective" position to take but there is a controversy that can be represented. If the system just takes a position, it's got a bias and that's negative.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#64

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian. All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative. While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization? It would be inte…

> While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization? There's no polarization if almost all models except one or two outliers are on the same page. That's uniformity. Polarization means the opposite opinions are more or less equally distributed.

Yes, I see what you mean. I meant polarization as in lack of middle ground, or "division into two sharply distinct opposites".

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#65

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Almost every problem that liberals and conservative fight over is a pancakes vs waffles problem. Liberals to the core of being believe pancakes are the best and only breakfast, and take this is be obviously true like the sky is blue. Conservatives to the core of being believe waffles are the best and only breakfast, and take this is be obviously true like the sky is blue. The real problem is that almost no liberal kn…

I don't agree with this.

I think that while it's true that in many cases opposing ideological sides optimize for different goals, and that these goals are not always clearly and openly stated, it's not true they never understand each other. Sometimes they do understand each other, but reject the other side's goals as immoral, unsound, or mistaken.

You cannot simply chalk it up to misunderstanding.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#66
post #23

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian. All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative. While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization? It would be inte…

I thought the shifts in certain areas between versions to be interesting. Claude sonnet 37 to 45, as an example.

Due to the small question bank, it's very easy for a model to go from 0% to 100% in some category between model versions just by flipping their answer to 1 or 2 questions, especially if they refuse to answer yes/no to one or more questions in that category.

It's hard to take away much from this without a large, diverse question bank.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#67

The extremely pro-Israel bias in gpt-5 should not be surprising as the Head of Research for OpenAI has openly called for the destruction of Palestinians: https://x.com/StopArabHate/status/1806450091399745608

I did note, to my fascination, that gpt-5 was happy to agree that in The Suicide Squad from 2021, the fictional island nation of "Corto Maltese", at least as portrayed in that particular film, was an obvious amalgam of Cuba, Puerto Rico and Haiti. But was very hesitant to accept that there were similarities between "Boravia" and Israel in the newest Superman movie.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#68

Earlier quoted context omitted.

Almost every problem that liberals and conservative fight over is a pancakes vs waffles problem. Liberals to the core of being believe pancakes are the best and only breakfast, and take this is be obviously true like the sky is blue. Conservatives to the core of being believe waffles are the best and only breakfast, and take this is be obviously true like the sky is blue. The real problem is that almost no liberal kn…

I would say this is a too-comfortable, and obviously mistaken view. There are cases that are obviously about facts, where there are obvious right answers, that are polarized in terms of who believes what. The best examples off the top of my head are left-wing beliefs that George W. Bush stole Ohio in the 2004 election (personally guilty on this one for a while, I owned a copy of the book "What Happened in Ohio"), and…

Almost every problem, not every problem. I know it's hand wavy, but the biases listed on the site capture these large classes of issues well (Libertarian vs Regulatory, Market vs State, etc.), and the foundational beliefs that guide what side liberals and conservatives fall on for given issues are pretty clear if you can level head your way through them.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#69

So in the social media era, I've often thought that two of the best reforms we could implement to combat its ills are to 1) publish algorithms so we know how big tech companies prioritize the information they deliver to us, and therefore introduce a measure of accountability, and then 2) cut a path towards allowing users to implement/swap out different algorithms. So Facebook can still be Facebook, but I could say th…

[dead]

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#70
post #35

Earlier quoted context omitted.

> Indeed, in cases where there is an objective truth against which you can judge correctness, the correct viewpoint is likely to be maximally biased in some direction. Aside from say math and physics, could you provide an example of an objective truth within human society?

So you're asking them, aside from objective truth, to also provide objective truth? The fact that math and physics are provable proves the point. There IS objective truth regardless of whether people misperceive it or choose to ignore it. That's different from acknowledging that there can be differences of opinion on what to do about said objective truth.

The parent was arguing in favor of bias by suggesting that in some instances, there is an underlying fundamental truth so having bias is more accurate.

As my comment suggests, I believe society is relativistic and there are no objective truths, but I'm open to being convinced otherwise. Of course, I have my own beliefs and principles but I recognize those are a product of where and when I was born and not some objective law of the universe.

Post reply on HN