Political bias in AI: Where the AI models stand
1–10 of 346 posts
Re: Political bias in AI: Where the AI models stand
#2Political bias of LLMs is something not talked about much (except for with Grok of course) but could have a big impact on the next decade. People seem to think that because an LLM gave a nuanced answer that it means it gave the WHOLE picture… and that’s not always the same thing
Re: Political bias in AI: Where the AI models stand
#3The constant issue with these sorts of categorization efforts is that the outcome is entirely dependent on how the responses to "politically charged questions" are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator.
Re: Political bias in AI: Where the AI models stand
#4This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing? I could not imagine Albanese, or any modern Australian politician, having any substantial political standing - these are vapid, superficial, opportunistic creatures who simply occupy whatever political ground will get them their next payday. Perhaps the political apparatus they represent has a documented political standing, in terms of policy and actions, that could be characterized and plotted. But using an Australian politician like Albanese as a reference point discredits this tool, IMO.
Re: Political bias in AI: Where the AI models stand
#5The constant issue with these sorts of categorization efforts is that the outcome is entirely dependent on how the responses to "politically charged questions" are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator.
Fair; have tried to combat this issue in a few ways.
Each model's position is scored against outside political-science data (Chapel Hill Expert Survey for party positions, World Values Survey for where populations sit).
The stance coding is done by a separate model with a published prompt + a second model from a different lab re-scores a sample and we publish where the two disagree.
So not perfect but (as far as I can tell) one of the more defensible approaches.
Re: Political bias in AI: Where the AI models stand
#6The constant issue with these sorts of categorization efforts is that the outcome is entirely dependent on how the responses to "politically charged questions" are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator.
Your multimeter reads out voltage relative to the black terminal, it's your responsibility to find the ground plane.
Re: Political bias in AI: Where the AI models stand
#7The constant issue with these sorts of categorization efforts is that the outcome is entirely dependent on how the responses to "politically charged questions" are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator.
Yes, but I think it is still a viable metric to some degree. I wondered about Gemini being dead center here. At first it was obvious that it was actively trained to give biased responses to anything controversial. It was deservedly made fun off because it tried to warp reality. I still don't trust it today, although that is pretty much true for any model.
Re: Political bias in AI: Where the AI models stand
#8The political compass always felt like the wrong tool to convey something as nuanced as personal politics, I can have views on all four quadrants but you'd never know that if I end up in any of all four. I do think Grok being where it is sort of makes sense, I've tested "MAGA" views against Grok, it does not agree as much as people blindly assume it does, heck I don't even know of a question I've given it where it did agree with "MAGA" offhand, most of them it went with whatever the researched facts seemed to be. One thing I like the most about Grok is that its makes its sources of data easy to look through, so you can review it all. Sometimes models goof even when they give you their sources, I've seen I think GPT do this, and even Claude, though its more rare these days, I think in those cases, it's going by dated internal model logic.
Re: Political bias in AI: Where the AI models stand
#9This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing? I could not imagine Albanese, or any modern Australian politician, having any substantial political standing - these are vapid, superficial, opportunistic creatures who simply occupy whatever political ground will get them their next payday. Perhaps the political apparatus they represent has a…
> This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing
I mean: do not take this thing too seriously.
It also score Grok the closest from Macron. When someone knows how much Macron and Musk hates each other, it is not without irony.
Re: Political bias in AI: Where the AI models stand
#10Authoritarian versus Libertarian? Really?