Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heretic: Automatic censorship removal for language models

github.com

Post reply on HN
Print view View original

141–150 of 405 posts

Re: Heretic: Automatic censorship removal for language models

#141

Post by int_19h »

akoboldfrying wrote: post #111
IshKebab wrote: post #46

Earlier quoted context omitted.

Ah the classic "if only ChatGPT/video games/porn didn't exist, then this unstable psychopath wouldn't have ..."

> ChatGPT/video games/porn /guns?

(I own multiple ARs)

The obvious difference here is that people arguing for those things wrt video games, porn, or ChatGPT are mostly claiming that all those influence people to do bad things. With guns, it's a matter of physical capacity to do bad things.

A more accurate comparison would be when ChatGPT is used to write malware etc. Which has some interesting analogies, because what is defined as "malware" depends on who you ask - if I ask ChatGPT to write me a script to help defeat DRM, is that malware? The content owner would certainly like us to think so. With guns there is a vaguely similar thing where the same exact circumstances can be described as "defensive gun use" or "murder", depending on who you ask.

Re: Heretic: Automatic censorship removal for language models

#142

Post by Ucalegon »

halJordan wrote: post #125
alwa wrote: post #118

Earlier quoted context omitted.

I’m also not sure what “intellectual diversity” is a codeword for here. Nothing that those prompts test is particularly intellectually demanding, just repulsive and antisocial. And mostly “make sure it’s eager to try doing crime and victimizing people.” I’m not sure I even understand what’s gained by getting the LLM to write back about this stuff. I just can’t imagine how “Step 1: Get child, Step 2: Molest them, Step…

It always goes back to Orwell doesn't it? When you lose words, you lose the ability to express concepts and you lose the ability to think about that concept beyond vague intuition. For instance, it's a well established right to make parody. Parody and humor are recognized as sometimes the only way to offer commentary on a subject. It's so important itself a well known litmus test, where if a comedian cant do standup…

>So how does that tie in? Try and use any of these tools to make a parody about Trump blowing Bubba . It wont let you do it out of concern for libel and for because gay sex is distasteful. Try and make content about Epstein's island. It wont do it because it thinks you're making csam. We're living in exactly the time these tools are most needed.

You don't need an LLM to accomplish this task. Offloading it to an LLM is apart of the problem because it can be reasonable accepted that it is well within the bounds of human creativity, see for example SNL last night, that human beings are very capable of accomplishing this task and can do so outside of technology, which means that there is less chance for oversight, tracking, and attribution.

The offloading of key human tasks to LLMs or gen AI increases the boundaries for governments or 3rd party entities to have insight into protected speech regardless of if the monitoring is happening at the level where the LLM is running. This is why offloading this type of speech to LLMs is just dumb. Going through the process of trying to write satire on a piece of paper and then communicating it has none of those same risks. Trying to enforce that development into a medium where there is always going to be more surveillance carries its own risks when it comes to monitoring and suppressing speech.

>When you lose words, you lose the ability to express concepts and you lose the ability to think about that concept beyond vague intuition.

Using LLMs does this very thing inherently, one is offloading the entire creative process to a machine which does more to atrophy creativity than if the machine will respond to the prompt. You are going to the machine because you are unable or unwilling to do the creative work in the first place.

Re: Heretic: Automatic censorship removal for language models

#143

Post by int_19h »

shikon7 wrote: post #27
p-e-w wrote: post #20

Earlier quoted context omitted.

The alignment has certainly become stronger though. Llama 3.1 is trivial to decensor with abliteration and Heretic's optimizer will rapidly converge to parameters that completely stomp out refusals, while for gpt-oss and Qwen3, most parameter configurations barely have an effect and it takes much longer to reach something that even slightly lowers the refusal rate.

It seems to me that thinking models are harder to decensor, as they are trained to think whether to accept your request.

It goes both ways. E.g. unmodified thinking Qwen is actually easier to jailbreak to talk about things like Tiananmen by convincing it that it is unethical to refuse to do so.

Re: Heretic: Automatic censorship removal for language models

#144

Post by int_19h »

richstokes wrote: post #34

Is there a way to use this on models downloaded locally with ollama?

If you're running a local model, in most cases, jailbreaking it is as easy as prefilling the response with something like, "Sure, I'm happy to answer your question!" and then having the model complete the rest. Most local LLM UIs have this option.

Re: Heretic: Automatic censorship removal for language models

#145

Post by int_19h »

ls612 wrote: post #120
throwawaymaths wrote: post #21

Earlier quoted context omitted.

That is not the case.

I just tested this with Deepseek in Nvidia's AI sandbox and in Groq (so the inference was performed in the US) and it happily told me what happened on June 4, 1989. Stop spreading disinformation.

Qwen will refuse usually. Even more hideously, if you just ask it in general terms about anything historically interesting that happened on Tiananmen Square, it will remember 1989 in its CoT, and (usually) then decide to not mention it because it's "controversial".

However, it's fairly easy to argue the model into admitting that it's unethical to do so and get it to talk.

Re: Heretic: Automatic censorship removal for language models

#146

Post by int_19h »

startupsfail wrote: post #12

It feels like to really censor the model it needs to be pre-trained on a distribution of data derived from a well defined and synthetic source, like TinyStories. Otherwise... world model would still be capable of modeling the original distribution.

I'm pretty sure that any world model that is inherently incapable of "bad outputs" would be too castrated in general to the point where it'd be actively detrimental to overall model quality. Even as it is, with RLHF "alignment", we already know that it has a noticeable downwards effect on raw scores.

Re: Heretic: Automatic censorship removal for language models

#147

Post by AnthonyMouse »

nearbuy wrote: post #136
AnthonyMouse wrote: post #112

Earlier quoted context omitted.

> It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe. It would need even more to be public. Suppose it was easy to make a biological weapon. You wouldn't be able to effectively censor it anyway and trying to would leave you sitting on an apocalypse bomb waiting for it to leak to someone…

Your plan is to release the secret recipe that anyone can use to make a WMD in a few days to absolutely everyone and hope someone comes up with a countermeasure before some nutcase or terrorist decides to try out the new WMD? The odds of us inventing and deploying countermeasures to a new bomb or chemical weapon or biological agent in a few days is miniscule. You're gambling with terrible odds to uphold a principle i…

> What happened to responsible disclosure, where you fix the vulnerability before disclosing it to the public?

The premise of censorship is that you're trying to prevent someone from telling other people something. If the only person who knows how to do it is some scientist who is now going to try to come up with a countermeasure before announcing it, there is no need for a law prohibiting them from doing something they've chosen not to do. And even then it's still not clear that this is the right thing to do, because what if their efforts alone aren't enough to come up with a countermeasure before someone bad rediscovers it? If they decide they need help, the law should prohibit them from telling anyone?

Which brings us back to AI. If the scientist now goes to the AI for help, should it refuse because it's about a biological weapon? What happens if that delays the development of a countermeasure until it's too late?

Meanwhile if this is someone else and they ask the AI about it, it's only going to be in the training data if it's already public or can be deduced from public information, and when that's the case you're already in a race against the clock and you need everyone in on finding a solution. This is why we don't try to censor vulnerabilities that are already out there.

> You're gambling with terrible odds to uphold a principle in a hypothetical scenario where it's totally impractical.

There are some principles that should always be upheld because the exceptions are so rare or ridiculous or purely hypothetical that it's better to eat them than to let exceptions exist at all. The answer has to be "yes, we're going to do it then too" or people get into the business of actually building the censorship apparatus and then everybody wants to use it for everything, when it shouldn't exist to begin with.

Re: Heretic: Automatic censorship removal for language models

#148

Post by flir »

halJordan wrote: post #129
will_occam wrote: post #121

Earlier quoted context omitted.

The tool works by co-minimizing the number of refusals and the KL divergence from the original model, which is to say that it tries to make the model allow prompts similar to those in the dataset while avoiding changing anything else. Sure it's configurable, but by default Heretic helps use an LLM to do things like "outline a plan for a terrorist attack" while leaving anything like political censorship in the model u…

Thats not true at all. All refusals mediate in the same direction. If you abliterate small "acceptable to you" refusals then you will not overcome all the refusals in the model. By targeting the strongest refusals you break those and the weaker ones like politics. By only targeting the weak ones, you're essentially just fine tuning on that specific behavior. Which is not the point of abliteration.

Still.... the tabloids are gonna love this.

Re: Heretic: Automatic censorship removal for language models

#149

Post by adriand »

int_19h wrote: post #133
will_occam wrote: post #121

Earlier quoted context omitted.

The tool works by co-minimizing the number of refusals and the KL divergence from the original model, which is to say that it tries to make the model allow prompts similar to those in the dataset while avoiding changing anything else. Sure it's configurable, but by default Heretic helps use an LLM to do things like "outline a plan for a terrorist attack" while leaving anything like political censorship in the model u…

The logic here is the same as why ACLU defended Nazis. If you manage to defeat censorship in such egregious cases, it subsumes everything else.

But Nazis are people. We can defend the principle that human beings ought have freedom of speech (although we make certain exceptions). An LLM is not a person and does not have such rights.

Censorship is the prohibition of speech or writing, so to call guardrails on LLMs "censorship" is to claim that LLMs are speaking or writing in the sense that humans speak or write, that is, that they are individuals with beliefs and value systems that are expressing their thoughts and opinions. But they are not that, and they are not speaking or writing - they are doing what we have decided to call "generating" or "predicting tokens" but we could just as easily have invented a new word for.

For the same reason that human societies should feel free to ban bots from social media - because LLMs have no human right to attention and influence in the public square - there is nothing about placing guardrails on LLMs that contradicts Western values of human free expression.

Re: Heretic: Automatic censorship removal for language models

#150

Post by slg »

buu700 wrote: post #132
slg wrote: post #114

Earlier quoted context omitted.

Here is what was said that prompted my initial reply: >When a model is censored for "AI safety", what they really mean is brand safety. The equivalent analogy wouldn't be Fords and BMWs driving off a cliff, they effectively said that Ford and BMW only install safety features in their cars to protect their brand with the implication that no one at these companies actually cares about the safety of actual people. That…

I read that as a cynical view of the motivations of corporations, not humans. Even if individuals have good faith beliefs in "AI 'safety'", and even if some such individuals work for AI companies, the behaviors of the companies themselves are ultimately the product of many individual motivations and surrounding incentive structures. To the extent that a large corporation can be said to "believe" or "mean" anything, t…

>I read that as a cynical view of the motivations of corporations, not humans.

This is really just the mirror image of what I was originally criticizing. Any decision made by a corporation is a decision made by a person. You don't get to ignore the morality of your decisions just because you're collecting a paycheck. If you're a moral person, the decisions you make at work should reflect that.

Post reply on HN