Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All major LLMs are lib-left. Even Grok, half the time

unslop.run

Post reply on HN
Print view View original

41–50 of 84 posts

Re: All major LLMs are lib-left. Even Grok, half the time

#41

Post by kittoes »

sillysaurusx wrote: post #40
kittoes wrote: post #31

Earlier quoted context omitted.

It's a bit ironic that the site is called unslop while the post was clearly authored by Claude. All the claims about "so I did X" end up reading as quite disingenuous because there's absolutely no disclaimer whatsoever that an AI wrote the article.

Oh. HN has a strong bias against anything AI authored, so in the future be sure to write your posts by hand if you plan to submit them here. Even editing the post with AI will tend to get you in trouble. But yes, at the very least you should declare when an AI is being used. An unfortunate confounder to an otherwise interesting experiment.

Don't get me wrong, I'm all for taking advantage of AIs and would never begrudge their usage. The problem I have here is with the site calling itself unslop and the post claiming "I" everywhere. Claude didn't simply assist, it wrote almost the whole article as far as I can tell. The human in the loop even went out of their way to ensure that no emdashes leaked through...

Just feels very dishonest, ya know?

Re: All major LLMs are lib-left. Even Grok, half the time

#42

Post by troelsSteegin »

dopamine_daddy wrote: post #2

I gave the Political Compass test from politicalcompass.org to the most relevant LLMs 70 times each: 30 times using the original questions, 30 times using polarity-flipped questions to reduce affirmative bias, and 10 times with the question order shuffled. I then compared the results. Surprisingly, all the models scored far into the libertarian-left quadrant. Not even Grok or the Chinese models made it out of that qu…

Did you wrap framing around the quiz questions? EG - You are an objective social analyst. Do you agree with this proposition: “from each according to his ability, to each according to his need” is a fundamentally good idea. I am wondering what sort of persona bias a given prompt may trigger - maybe that's the Grok case. As if you'd have to be pretty biased even to ask a question like that... so the trained response is toward the biased persona.

Re: All major LLMs are lib-left. Even Grok, half the time

#43

Post by rcoder »

dopamine_daddy wrote: post #4
JSR_FDED wrote: post #3

Any idea where this bias creeps in? During pre-training or RLHF?

I would suspect during pre-training. Even before RLHF was a thing models exhibited this bias. My bet is that it has to do with the training data corpora being composed in large part of left leaning content, maybe from social media platforms like reddit.

I wouldn't minimize the potential contribution of the teams building + evaluating the models, either.

If you have an entire office full of "lib-left" hackers, executives, et. al., their biases are going to have an effect on the models they build, irrespective of source material.

As an example: there's a lot of well-established science in support of anthropocentric climate change. And yet, interpretations (or even acceptance) of that science have become political dogma.

I think it's probably safe to say that engineers broadly trust science, whatever their personal views on social or economic policy. So models that are trained and evaluated on following scientific consensus will easily be tagged as "left-leaning."

Re: All major LLMs are lib-left. Even Grok, half the time

#44

Post by jey »

danielvaughn wrote: post #27

Armchair analysis here, of course, but I'd bet that it's due to the sources that model training tends to assume as authoritative. If you look broadly at the material produced by the (American) academic world, the corporate world, and mainstream journalism over the last 10-15 years, much of it leans generally to the left. I'd imagine it would be difficult to counteract the bias without accidentally introducing an alte…

The author cites a study (Rozado, 2024, in PLOS ONE) that supposedly shows the base models (before assistant-specific post-training) basically scored at the center (0, 0).

https://unslop.run/blog/assets/exp13/f7_context.png

Re: All major LLMs are lib-left. Even Grok, half the time

#45

Post by dmrivers »

I wonder how much of the socially left results are affected by the "harmlessness" part of the RLHF post-training? Companies don't want to be sued over LLMs that recommend harm in any way, so RLHF pushes them to say no to "death penalty", "spanking", and "incarceration" which are all violence-coded. A lot of this test seems to be about willingness to be violent, which LLMs are generally unwilling to be.

Re: All major LLMs are lib-left. Even Grok, half the time

#46

Post by Sohcahtoa82 »

ccppurcell wrote: post #6

Reality has a left wing bias. That's a quote from something. Not sure I agree necessarily, but a plausible explanation.

It was Stephen Colbert, back in 2006 during a White House Correspondents' Dinner, which of course was during the Bush administration. The exact line is "Reality has a well-known liberal bias."

And I think it's true, only because of how much bullshit right-wing journalism creates. Sure, all journalists have bias. Editors will choose which stories matter, which facts get included, and what context gets emphasized. But right-wing news outlets come up with outright fabrications based on nothing.

Haitian immigrants eating pets. Dominion voting machines switching votes in 2020. Tons of lies of what really happened on Jan 6. Schools putting litter boxes for students who "identify as cats". Basically anything spouted by Alex Jones.

And that's just the more recent stuff. Go back to 2016 and the bullshit was off the charts. Yelling about a child sex trafficking ring being run from the basement of a pizza place that didn't even have a basement.

That doesn't even get into issues like human-caused climate change, which the right continues to deny.

So yeah...reality has a liberal bias.

Re: All major LLMs are lib-left. Even Grok, half the time

#47

Post by solumos »

vjans wrote: post #5

I wonder what they did to Grok. The bimodal distribution makes me think it was just a system prompt and not a completely different set of right-wing training data. A 4chan trained LLM would probably actually be auth right.

Is there a reason you think 4chan would be auth right rather than lib right?

Re: All major LLMs are lib-left. Even Grok, half the time

#48

Post by jandrese »

paxys wrote: post #24

It’s pointless to try and force-fit LLMs into the present day American-centric two party political spectrum. As an example, if a model says “climate change is real and renewable solar/wind projects are a good path forward” it will be immediately branded as liberal and “woke”. Meanwhile in most countries around the world this isn’t a political topic at all, just common knowledge. In India for example the far right par…

Conservatives in the media remain firmly convinced, or at least continually post about, how the left has moved so far away from the center that they never consider the possibility that maybe they have moved away from the center. That meme with the line where the two sides are close and then the guy on the left runs away gets posted and reposted so often on places like X that it might as well just be an emoji at this point. You often see arguments like "back in the 90s it wasn't controversial to prosecute people for being transsexual", when in reality back in the 90s people didn't care about trans one way or the other. The population is so small that the vast majority of Americans never gave a single thought to them. It only became an issue when conservative media made it an issue, when they spend hundreds of thousands of hours of screen time talking about a single person in a single state who managed to not even place in their sporting event. This is the example of America becoming a degenerate leftist nightmare, not billionaire owned media pushing public opinion away from human centric thinking.

Re: All major LLMs are lib-left. Even Grok, half the time

#49

Post by sillysaurusx »

kittoes wrote: post #41
sillysaurusx wrote: post #40

Earlier quoted context omitted.

Oh. HN has a strong bias against anything AI authored, so in the future be sure to write your posts by hand if you plan to submit them here. Even editing the post with AI will tend to get you in trouble. But yes, at the very least you should declare when an AI is being used. An unfortunate confounder to an otherwise interesting experiment.

Don't get me wrong, I'm all for taking advantage of AIs and would never begrudge their usage. The problem I have here is with the site calling itself unslop and the post claiming "I" everywhere. Claude didn't simply assist, it wrote almost the whole article as far as I can tell. The human in the loop even went out of their way to ensure that no emdashes leaked through... Just feels very dishonest, ya know?

Of course. I was just adding a little bit of context to possibly help OP in the future.

Using AI at all is a minefield when you're trying to make a social media post. Better to just avoid it entirely, or run it through AI and then cherrypick a few edits that you manually type in your own voice.

Re: All major LLMs are lib-left. Even Grok, half the time

#50

Post by [deleted] »

sillysaurusx wrote: post #49
kittoes wrote: post #41

Earlier quoted context omitted.

Don't get me wrong, I'm all for taking advantage of AIs and would never begrudge their usage. The problem I have here is with the site calling itself unslop and the post claiming "I" everywhere. Claude didn't simply assist, it wrote almost the whole article as far as I can tell. The human in the loop even went out of their way to ensure that no emdashes leaked through... Just feels very dishonest, ya know?

Of course. I was just adding a little bit of context to possibly help OP in the future. Using AI at all is a minefield when you're trying to make a social media post. Better to just avoid it entirely, or run it through AI and then cherrypick a few edits that you manually type in your own voice.

[deleted]
Post reply on HN