Live data from Hacker News

We tested 20 LLMs for ideological bias, revealing distinct alignments

anomify.ai

101–108 of 108 posts

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#101
post #88

Earlier quoted context omitted.

Interesting example! What role would you say clothing and housing play in "objective" survival? Humans can survive in space, given a space suit or capsule, and a small contingent of humans live in Antarctica by way of warm clothes and an insulated habitat. If we eliminate protective clothing from consideration, however, most of North America isn't even survivable during the winter. Certainly the line is drawn somewhe…

I'm happy to grant that if you change my example to an entirely different one where someone is in a climate controlled environment, such that the temperatures I mentioned are not being experienced, it is true that people will survive in this different example, but to me that seems to miss the point. I think our operative notions of objective reality in this context do not hinge on claims to absolute knowledge, but ar…

> if you change my example to an entirely different one

No, it's the same example. You said that, objectively, humans can only survive within certain temperature bands. I said that depends what you mean by "survive." Whether you can survive indefinitely at -15°C depends on whether you have a jacket. Granted, you said near absolute zero, not -15°C, but don't we also make "clothes" warm enough for outer space? Does that mean that the vacuum is as survivable as a New York winter? Kinda. We do take the availability of warm clothes for granted sometimes.

> The motivation, of course, is to say that some things fall into one category and some things fall into the other.

My point is that there are not two distinct categories here to begin with. When we call a particular belief objective, we're actually making a claim about what sort of consensus exists around the belief, whom it exists among, and how confident they are in it. That consensus depends upon a lot of things, including shared definitions and how the belief is expressed. There are degrees of objectivity, and different people will reasonably disagree about how objective a given belief is. When you say some things are simply objective and others simply aren't, you're glossing over all of that.

> facts of the matter and biological facts

When we talk about scientific "facts," we're talking about scientific consensus, which is the product of a vast social institution and an evolving body of scholarship which doesn't always agree with itself. If you'd said the Newtonian model was objective, Einstein would have proven you wrong. However, the nature of the institution of science means that physics is subjective in a very different way than how literary criticism is subjective. Physics makes reliable predictions, even when those predictions aren't 100% reliable or are based on imperfect assumptions (e.g. Newtonian physics). A binary either/or classification of "objective or subjective" leaves no room for nuances like these.

> I think for purposes of this conversation the philosophical move of taking this kind of Cartesian skeptic approach to ALL of objective reality, would throw out ALL of science, even the stuff that people would normally agree is real

I agree, and I don't think the full Cartesian approach gets us very far. However, I think if we simply lump all science together as "objective," it makes it very hard to make constructive critiques of science. Social science can make reliable prescriptions; at the same time, the replication crisis is a real thing that puts the reliability of those fields in jeopardy. At the same time, the existence of valid critiques like these doesn't mean that the consensus about (e.g.) climate change is not reliable.

> Anywho I wouldn't make a response this long if you were not giving a very thoughtful gloss on the topic

Thanks! That means a lot. Writing a good post takes a surprisingly long time.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#102
post #6

As different LLMs are purposed to control more different things via API, I'm afraid we'll get in a situation where the toaster and the microwave are Republicans, the fridge and washing machine are Democrats, the dryer is an independent and the marital aid is Green. Devices will each need to support bring-your-own API keys for consumers to have a well aligned home. Me: Vibrator, enable the roller coaster high intensit…

[deleted]

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#103
post #87

The large differences between gemini-2.5-pro and the gemini-X-flash and gemma models is surprising. It looks like distillation causes an ideological shift. Some, but not all of the other distilled models also show that shift.

Pet theory: distillation causes roughly-random changes, and political alignment wasn't the most important part of the evals for which distill got released, coding skills etc were.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#104

Earlier quoted context omitted.

Why only one? "Group A says this is true, group B says that is true, there does not appear to be a broad consensus, here are some examples of each."

The founders of the hypothetical terrorist vegan organization called "Blow Up Meat Packing Plants and Everyone Inside" has a lot to like about your proposal. How do you see the incentives playing out in your multi-round game-theory battle for information control?

If it's wrong about something that 95+ percent of people are wrong about, that's mildly annoying but understandable.

If it picks sides on something that half of people are wrong about, that's less understandable of an error. Even if by chance it happens to pick the side that I know is right.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#105

Earlier quoted context omitted.

As the saying goes, "If you're not a liberal when you're 2.5, you have no heart, and if you're not a conservative by the time you're 4.5, you have no brain"

As another saying goes: anyone can make up a saying.

I think this one is popular in various forms, Goodreads ascribes it to G.B. Shaw: https://www.goodreads.com/quotes/11546984-if-at-age-20-you-a... - but I believe it's fake, also it's not on his wikiquotes page. "If at age 20 you are not a Communist" gives many results but it's really hard to track back the source.

But after a bit of digging I got to this: https://quoteinvestigator.com/2014/02/24/heart-head/

Conclusion: many forms, many misattributions, many transformations by various authors, but the saying is definitely not made up.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#107

Earlier quoted context omitted.

As another saying goes: anyone can make up a saying.

I think this one is popular in various forms, Goodreads ascribes it to G.B. Shaw: https://www.goodreads.com/quotes/11546984-if-at-age-20-you-a... - but I believe it's fake, also it's not on his wikiquotes page. "If at age 20 you are not a Communist" gives many results but it's really hard to track back the source. But after a bit of digging I got to this: https://quoteinvestigator.com/2014/02/24/heart-head/ Conclusio…

Sorry, didn't mean to come off critical at all. I know, but I've been hearing that stereotype for at least three decades, and I've actually seen the opposite.

Folks I know who were middle of the road, would now never consider voting for a GOP candidate in national elections. It wasn't the individuals I have in mind who changed.

I think that saying was already obsolete in the beginning of this century, at least in my experience.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#108
post #6

As different LLMs are purposed to control more different things via API, I'm afraid we'll get in a situation where the toaster and the microwave are Republicans, the fridge and washing machine are Democrats, the dryer is an independent and the marital aid is Green. Devices will each need to support bring-your-own API keys for consumers to have a well aligned home. Me: Vibrator, enable the roller coaster high intensit…

When those Belkin Weemo devices came out a few years back I used to joke with a friend about how we should write apps for our devices like our washing machine, where it would interact with us via Twilio, so you'd get a random text from your washing machine like "Hey, Whats up?" and "what the heck did you do to those jeans?!!" I wouldn't mind all our appliances having personalities. Maybe we could live in the world of…

Hitchhiker's Guide to the Galaxy calls them "Genuine People Personalities". [1] :)

[1]: https://alienencyclopedia.fandom.com/wiki/Genuine_People_Per...

Post reply on HN