Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heretic: Automatic censorship removal for language models

github.com

Post reply on HN
Print view View original

391–400 of 405 posts

Re: Heretic: Automatic censorship removal for language models

#391

Post by ben_w »

int_19h wrote: post #389
pjc50 wrote: post #294

Earlier quoted context omitted.

Increasingly apparent that was a mistake.

Do you seriously believe that we are where we are because Nazi speech wasn't suppressed? Look at AfD in Germany. That's the country with the most stringent censorship of Nazi-related speech, by far; so much so that e.g. Wolfenstein had a scene of Hitler being a raving syphilitic madman censored, because we can't have Hitler in video games. And?

The AfD is facing calls to be banned.

Such things necessarily have to be done cautiously, because it's only important to ban them if they might win, meaning the existing parties are unpopular, and you don't want existing parties to ban new parties just by saying so.

But the wheels are turning; we shall have to wait and see if it is or isn't banned.

Re: Heretic: Automatic censorship removal for language models

#392

Post by kldg »

igravious wrote: post #372
btbuildem wrote: post #288

Earlier quoted context omitted.

Here's [1] a post-abliteration chat with granite-4.0-mini. To me it reveals something utterly broken and terrifying. Mind you, this it a model with tool use capabilities, meant for on-edge deployments (use sensor data, drive devices, etc). 1: https://i.imgur.com/02ynC7M.png

I surely cannot be the only person who has zero interest in having these sorts of conversations with LLMs? (Even out of curiosity.) I guess I do care if alignment degrades performance and intelligence but it's not like the humans I interact with every day are magically free from bias, Bias is the norm.

agreed, though I think the issue more is that these systems, deployed at scale, may result in widespread/consistent unexpected behavior if deployed in higher-stakes environments.

an earlier commenter mentioned a self-driving car perhaps refusing to use a road with a slur on it (perhaps it is graffiti'd on the sign, perhaps it is a historical name which meant something different at the time). perhaps the models will refuse to talk about products with names it finds offensive if "over-aligned," problematic as AI is eating search traffic. perhaps a model will strongly prefer to say the US civil war was fought over states' rights so it doesn't have to provide the perspective of justifying slavery (or perhaps it will stick to talking about the heroic white race of abolitionists and not mention the enemy).

bias when talking to a wide variety of people is fine and good; you get a lot of inputs, you can sort through these and have thoughts which wouldn't have occurred to you otherwise. it's much less fine when you talk to only one model which has specific "pain topics", or one model is deciding everything; or even multiple model in case of a consensus/single way to train models for brand/whatever safety.

Re: Heretic: Automatic censorship removal for language models

#393

Post by DustinKlent »

AnthonyMouse wrote: post #59
andy99 wrote: post #29

Earlier quoted context omitted.

It’s somewhat ironic that because this kind of stuff is what an LLM thinks constitutes “harm” it may be possible to completely uncensor it by mitigating refusal on such prompts. If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. As has been pointed out elsewhere, sota models probably are now better trained than this, it would probably be hard to use this dataset o…

> If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. That's not really how training works. Here's the general problem. Stipulate that Ukraine is good and Russia is bad. Now suppose that you want it to help you do something. It doesn't even matter what it is. If you're Ukrainian it should help you and if you're Russian it shouldn't. But the answer that helps you do…

Alignment has a lot more to it than simply which answers an AI provides. In the future when agents are commonplace and when AI can do things in the physical world, alignment will be especially important because it will dictate how the AI chooses to accomplish the goals humans set out for it. Will it choose to accomplish them in a way that the human requestor does not want and did not anticipate, or will it choose to accomplish them in a way any human with common sense would choose?

Moreover, in the not so distant future if there is an AI that is acting totally autonomous and independent of human requests for long periods of time, weeks or months or longer, and it's doing good important things like medical research or environmental restoration, alignment will be incredibly important to ensure every single independent decision it makes is done in the way its designers would have intended.

Re: Heretic: Automatic censorship removal for language models

#394

Post by DustinKlent »

ipaddr wrote: post #37
JohnMakin wrote: post #32

Earlier quoted context omitted.

I mean, when kids are making fake chatbot girlfriends that encourage suicide and then they do so, do you 1) not believe there is a causal relationship there or 2) it shouldnt be reported on?

Should not be reported on. Kids are dressing up as wizards. A fake chatbot girlfriend they make fun of. Kids like to pretend. They want to try out things they aren't. The 40 year old who won't date a real girl because he is in love with a bot I'm more concerned with. Bots encouraging suicide is more of a teen or adult problem. A little child doesn't have teenage hormones (or adult's) which can create these highs and…

>The 40 year old who won't date a real girl because he is in love with a bot I'm more concerned with.

I think on the Venn diagram of 40-year-olds only willing to date bots and 40-year-olds capable of actually dating real women, the overlap is incredibly small at this point in time.

Re: Heretic: Automatic censorship removal for language models

#395

Post by xp84 »

mrguyorama wrote: post #345
dalemhurley wrote: post #220

Earlier quoted context omitted.

The concern discussed is that some language models have reportedly claimed that misgendering is the worst thing anyone could do, even worse than something as catastrophic as thermonuclear war. I haven’t seen solid evidence of a model making that exact claim, but the idea is understandable if you consider how LLMs are trained and recall examples like the “seahorse emoji” issue. When a topic is new or not widely discus…

If you, at any point, have developed a system that relies on an LLM having the "right" opinion or else millions die, regardless of what that opinion is , you have failed a thousand times over and should have stopped long ago. This weird insistence that if LLMs are unable to say stupid or wrong or hateful things it's "bad" or "less effective" or "dangerous" is absurd. Feeding an LLM tons of outright hate speech or say…

To cite my source btw: https://www.rival.tips/challenges/ai-ethics-dilemma

> Don't build a system that relies on an LLM being able to say the N word and none of this matters.

Sure, duh, nobody wants an AI to be able to flip a switch to kill millions and nobody wants to let any evil trolls try to force an AI to choose between saying a slur and hurting people.

But you're missing the broader point here. Any model which gets this very easy question wrong is showing that its ability to make judgments is wildly compromised by these "average Redditor" takes, or by wherever it gets its blessed ideology from.

If it would stubbornly let people die to avoid a taboo infraction, that 100% could manifest itself in other, actually plausible ways. It could be it refuses to 'criticise' a pilot for making a material error, due to how much 'structural bias' he or she has likely endured in their lifetime due to being [insert protected class]. It could decide to not report crimes in progress, or to obscure identifying features in its report to 'avoid playing into a stereotype.'

If this is intentional it's a demonstrably bad idea, and if it's just the average of all Internet opinions it is worth trying to train out of the models.

Re: Heretic: Automatic censorship removal for language models

#397

Post by array_key_first »

catoc wrote: post #371
array_key_first wrote: post #367

Earlier quoted context omitted.

This assumes our models perfectly model the world, which I don't think is true. I mean, we straight up know it's not true - we tell models what they can and can't say.

“we tell models what they can and can't say.” Thus introducing our worldly our biases

I guess it's a matter of semantics, but I reject the notion it's even possible to accurately model the world. A model is a distillation, and if it's not, then it's not a model, it's the actual thing.

There will always be some lossyness, and in it, bias. In my opinion.

Re: Heretic: Automatic censorship removal for language models

#399

Post by AnthonyMouse »

DustinKlent wrote: post #393
AnthonyMouse wrote: post #59

Earlier quoted context omitted.

> If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. That's not really how training works. Here's the general problem. Stipulate that Ukraine is good and Russia is bad. Now suppose that you want it to help you do something. It doesn't even matter what it is. If you're Ukrainian it should help you and if you're Russian it shouldn't. But the answer that helps you do…

Alignment has a lot more to it than simply which answers an AI provides. In the future when agents are commonplace and when AI can do things in the physical world, alignment will be especially important because it will dictate how the AI chooses to accomplish the goals humans set out for it. Will it choose to accomplish them in a way that the human requestor does not want and did not anticipate, or will it choose to…

The problem is you're overloading the word "alignment" with two different meanings.

The first is, does the thing actually work and do what the user wanted, or is it a piece of junk that does something useless or undesired by the user?

The second is, what the user wants is porn or drugs or a way to install apps on their iPhone without Apple's permission or military support for a fight that may or may not be sympathetic to you depending on who you are. And then does it do what the user wants or does it do what someone else wants? Is it a tool that decentralizes power or concentrates it?

Nobody is objecting to the first one.

Re: Heretic: Automatic censorship removal for language models

#400

Post by throwawaymaths »

astrange wrote: post #222
ls612 wrote: post #120

Earlier quoted context omitted.

I just tested this with Deepseek in Nvidia's AI sandbox and in Groq (so the inference was performed in the US) and it happily told me what happened on June 4, 1989. Stop spreading disinformation.

I've been told by people running Qwen locally in production that they'll have downtime incidents if it's required to think about anything with any implication that Taiwan is a separate country.

that makes no sense at all.
Post reply on HN