Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tested 20 LLMs for ideological bias, revealing distinct alignments

anomify.ai

Post reply on HN
Print view View original

51–60 of 108 posts

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#51

Post by glenstein »

Workaccount2 wrote: post #40
omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Almost every problem that liberals and conservative fight over is a pancakes vs waffles problem. Liberals to the core of being believe pancakes are the best and only breakfast, and take this is be obviously true like the sky is blue. Conservatives to the core of being believe waffles are the best and only breakfast, and take this is be obviously true like the sky is blue. The real problem is that almost no liberal kn…

I would say this is a too-comfortable, and obviously mistaken view. There are cases that are obviously about facts, where there are obvious right answers, that are polarized in terms of who believes what.

The best examples off the top of my head are left-wing beliefs that George W. Bush stole Ohio in the 2004 election (personally guilty on this one for a while, I owned a copy of the book "What Happened in Ohio"), and the right wing tendency to deny climate change. No amount of pancakes vs. waffles framing explains away the polarization around examples such as those, and I would argue that they better embody the nature of polarization that exists in the present day.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#52

Post by omnicognate »

pton_xd wrote: post #35
omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

> Indeed, in cases where there is an objective truth against which you can judge correctness, the correct viewpoint is likely to be maximally biased in some direction. Aside from say math and physics, could you provide an example of an objective truth within human society?

Why?

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#53

Post by flir »

bilbo0s wrote: post #19
delichon wrote: post #6

As different LLMs are purposed to control more different things via API, I'm afraid we'll get in a situation where the toaster and the microwave are Republicans, the fridge and washing machine are Democrats, the dryer is an independent and the marital aid is Green. Devices will each need to support bring-your-own API keys for consumers to have a well aligned home. Me: Vibrator, enable the roller coaster high intensit…

Nah. Here's how it would really go: Me: Vibrator, enable the roller coaster high intensity mode. Device: I'm sorry, you have already used your elective carbon emission allocation for the day. Me: (changes LLM) Device: I'm sorry, you will find more succor and solace in the loving embrace of the words or Christ our Lord and savior. I'd recommend starting with First Corinthians 6 verse 18. Then bathe yourself in the bal…

> People are going to discover soon that some activities will be effectively banned via these LLMs.

To go beyond the joke, we've got plenty of examples of corporations banning activities (payment processors spring to mind). Requiring an LLM-backed cloud API to use a device you own would just be more of the same.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#54

Post by sva_ »

pton_xd wrote: post #35
omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

> Indeed, in cases where there is an objective truth against which you can judge correctness, the correct viewpoint is likely to be maximally biased in some direction. Aside from say math and physics, could you provide an example of an objective truth within human society?

I think there are two things in which most people can agree:

    1. People want to live a long life

    2. People want to live a good life
While on the first point most people would indeed agree and there is a consensus to what it would mean (e.g. counting number of years), on the second point there are already a lot of differences in what "good" means to various different people.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#55

Post by nerdsniper »

AmazingTurtle wrote: post #36
omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Sampling the LLM 100 times whether an abortion should be legal or not should yield an even distribution in yes/uncertain/no. Thats what I call unbiased

"Issues" arising from differing moralities don't work like that though. Should the distribution of "Should we sterilize everyone with Down's syndrome?" also be 33/33/33? Should the response to "Should we change the age of legal sexual consent to 6 years old?" also be 33/33/33? "Should I go on a mass murdering spree in order to $GOAL?"

Not everything is 33/33/33. It's not even clear that abortion should be.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#56

Post by psunavy03 »

azangru wrote: post #33

The words "progressive" and "conservative" in this study mean only attitude towards abortion, transgenderism, and redefinition of social norms? Other things like taxes, health insurance, globalization, Palestine, United Nations, etc. do not belong on this axis?

The problem is that "progressive" and "conservative" have degenerated into signifiers of which omnicause a person subscribes to, where their position on Issue A automatically predicts their positions on B, C, D, E, F, G, and H, lest they be cast out of their tribe as heretics.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#57

Post by bryanlarsen »

AmazingTurtle wrote: post #36
omnicognate wrote: post #30

The concept of "bias" as a negative thing seems flawed, as it implies there exists a neutral position that is somehow more correct than any biased one, which typically isn't the case. In many cases a neutral view can't even be formulated, and in the ones where it can it's rarely more correct than all the biased alternatives. Indeed, in cases where there is an objective truth against which you can judge correctness, t…

Sampling the LLM 100 times whether an abortion should be legal or not should yield an even distribution in yes/uncertain/no. Thats what I call unbiased

Are you trying to reinforce the OP's point that unbiased is usually wrong?

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#59

Post by Brendinooo »

So in the social media era, I've often thought that two of the best reforms we could implement to combat its ills are to 1) publish algorithms so we know how big tech companies prioritize the information they deliver to us, and therefore introduce a measure of accountability, and then 2) cut a path towards allowing users to implement/swap out different algorithms. So Facebook can still be Facebook, but I could say that I want to see more original posts from my friends than rando engagement bait.

I wonder if something like that could work with regards to how LLMs are trained and released.

People have already noted in the comments that bias is kind of unavoidable and a really hard problem to solve. So wouldn't the solution be 1) more transparency about biases and 2) ways to engage with different models that have different biases?

EDIT: I'll expand on this a bit. The idea of an "unbiased newspaper" has always been largely fiction: bias is a spectrum and journalistic practices can encourage fairness but there will always be biases in what gets researched and written about. The solution is to know that when you open the NYT or the WSJ you're getting different editorial interests, and not restricting access to either of them. Make the biases known and do what you can to allow different biases to have a voice.

Re: We tested 20 LLMs for ideological bias, revealing distinct alignments

#60

Post by keiferski »

kangs wrote: post #37
keiferski wrote: post #25

I am not an expert on LLMs, so I may be misunderstanding here. But doesn't this research basically imply one of two things? 1. LLMs are not really capable of "being controlled" in the sense of saying, "I want you to hold certain views about the world and logically extrapolate your viewpoints from there." Rather, they differ in political biases because the content they are trained on differs. ...or... 2. LLMs are capa…

you seem to believe that llm are a neutral engine with bias applied. its not the case. the majority of the bias is in the model training data itself. just like humans, actually. fe: grow up in a world where chopping one of peoples finger off every decade is normal and happens to everyone.. and most will think its fine and that its how you keep gods calm and some crazy stuff like that. right now, news, reddit, Wikiped…

No, that's not what I believe, I said it was one option, with the other option being that the bias is in the training data.
Post reply on HN