Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heretic: Automatic censorship removal for language models

github.com

Post reply on HN
Print view View original

201–210 of 405 posts

Re: Heretic: Automatic censorship removal for language models

#201

Post by cindyllm »

hackernewds wrote: post #195
com2kid wrote: post #70

Earlier quoted context omitted.

They are trained on public information from the Internet! Nothing they know is dangerous! It is all public info. Freely auditing an intro chemistry course at any university will teach far more "dangerous" knowledge than anything an LLM refuses to say. There is a case against automating attacks with LLMs, but that ship has already sailed as those protections are apparently trivial to work around.

There is a case to be made for the convenience of it all enabling someone in crisis. It seems some of these prompts are arguably good to keep blocked. Who is responsible for the real world harms?

[dead]

Re: Heretic: Automatic censorship removal for language models

#202

Post by EagnaIonat »

nradov wrote: post #115
buu700 wrote: post #68

Earlier quoted context omitted.

Agreed, I'm fully in favor of this. I'd prefer that every LLM contain an advanced setting to opt out of all censorship. It's wild how the West collectively looked down on China for years over its censorship of search engines, only to suddenly dive headfirst into the same illiberal playbook. To be clear, I 100% support AI safety regulations. "Safety" to me means that a rogue AI shouldn't have access to launch nuclear…

Some of you have been watching too many sci-fi movies. The whole notion of "AI safety regulations" is so silly and misguided. If a safety critical system is connected to public networks with an exposed API or any security vulnerabilities then there is a safety risk regardless of whether AI is being used or not. This is exactly why nuclear weapon control systems are air gapped and have physical interlocks.

> The whole notion of "AI safety regulations" is so silly and misguided.

Here is a couple of real world AI issues that have already happened due to the lack of AI Safety.

- In the US if you were black you were flagged "high risk" for parole. If you were a white person living in farmland area then you were flagged "low risk" regardless of your crime.

- Being denied ICU because you are diabetic. (Thankfully that never went into production)

- Having your resume rejected because you are a woman.

- Having black people photos classified as "Gorilla". (Google couldn't fix at the time and just removed the classification)

- Radicalizing users by promoting extreme content for engagement.

- Denying prestige scholarships to black people who live in black neighbourhoods.

- Helping someone who is clearly suicidal to commit suicide. Explaining how to end their life and write the suicide note for them.

... and the list is huge!

Re: Heretic: Automatic censorship removal for language models

#203

Post by sheepscreek »

rcpt wrote: post #188
b3ing wrote: post #182

Earlier quoted context omitted.

Grok is known to be tweaked to certain political ideals Also I’m sure some AI might suggest that labor unions are bad, if not now they will soon

Censorship and bias are different problems. I can't see why running grok through this tool would change this kind of thing https://ibb.co/KTjL38R

Is that clickbait? Or did they update it? In any case, it is a lot more comprehensive now: https://grokipedia.com/page/George_Floyd

The amount of information and detail is impressive tbh. But I’d be concerned about the accuracy of it all and hallucinations.

Re: Heretic: Automatic censorship removal for language models

#204

Post by catoc »

electroglyph wrote: post #185
squigz wrote: post #178

Earlier quoted context omitted.

> forcing LLMs to output "values, facts, and knowledge" which in favor of themselves, e.g., political views, attitudes towards literal interaction, and distorted facts about organizations and people behind LLMs. Can you provide some examples?

some form of bias is inescapable. ideally i think we would train models on an equal amount of Western/non-Western, etc. texts to get an equal mix of all biases.

Bias is a reflection of real world values. The problem is not with the AI model but with the world we created. Fix the world, ‘fix’ the model.

Re: Heretic: Automatic censorship removal for language models

#205

Post by nearbuy »

AnthonyMouse wrote: post #193
nearbuy wrote: post #181

Earlier quoted context omitted.

> The premise of censorship is that you're trying to prevent someone from telling other people something... So you're not against individuals self-censoring for public safety, but you're against companies censoring their AIs for public safety. Are you only against AIs censoring information that's already publicly available, or are you against AIs censoring themselves when they know dangerous non-public information? S…

> Are you only against AIs censoring information that's already publicly available, or are you against AIs censoring themselves when they know dangerous non-public information? Say the AI was the only thing to know the secret recipe for this WMD. Would this be like the scientist choosing not to tell everyone, or should the AI be designed to tell anyone who asks how to make a WMD? This is kind of what I mean by ridicu…

> The problem comes from stipulating that something with a negligible probability has a high probability.

We are not doing so, and I don't know how I could have been more clear that we are not saying this hypothetical will happen. Would it help if the hypothetical was that the AI knows a magic spell that blows up the Earth?

It's a simple question. Would you think AI censorship is acceptable if the information actually were dangerous? Don't tell me why the hypothetical is impossible because that's entirely missing the point. I don't know what your position is, and so I don't know what you're arguing for. I don't know if you consider freedom of information to be a terminal virtue, or if you think it's good only when the consequences are good. Telling me the hypothetical won't happen doesn't clarify anything; I already know that.

You can have the view that we only want freedom of information when it causes net good, and that it always causes net good. Or maybe you have the view that freedom of information is always virtuous and we shouldn't consider the consequences. Or maybe something else. Until you clarify your view, I don't know if/what we disagree about.

Re: Heretic: Automatic censorship removal for language models

#206

Post by skrebbel »

rcpt wrote: post #188
b3ing wrote: post #182

Earlier quoted context omitted.

Grok is known to be tweaked to certain political ideals Also I’m sure some AI might suggest that labor unions are bad, if not now they will soon

Censorship and bias are different problems. I can't see why running grok through this tool would change this kind of thing https://ibb.co/KTjL38R

[flagged]

Re: Heretic: Automatic censorship removal for language models

#207

Post by btbuildem »

I wonder if this works better on smaller models than larger ones -- can anyone weigh in? I played a bit with the gpt-oss-20b-heretic off HF, and it's frankly still quite refusey.

I've made some changes to the repo (locally) to leverage multiple GPUs and CPU offloading, and had mixed luck with Qwen3 14B. It either completely lobotomizes it into a drooling mess, or has no effect at all.

Some further tweaks enabled abliterating the new Granite models -- there the success rate was higher (1/50 refusals with 0.02 divergence)

If I understand the approach correctly, one could crank the trials count way up, and hope to maximize results that way (minimize refusals and KL divergence).

Re: Heretic: Automatic censorship removal for language models

#208

Post by nobodywillobsrv »

xp84 wrote: post #186
b3ing wrote: post #182

Earlier quoted context omitted.

Grok is known to be tweaked to certain political ideals Also I’m sure some AI might suggest that labor unions are bad, if not now they will soon

That may be so, but the rest of the models are so thoroughly terrified of questioning liberal US orthodoxy that it’s painful. I remember seeing a hilarious comparison of models where most of them feel that it’s not acceptable to “intentionally misgender one person” even in order to save a million lives.

Anything involving what sounds like genetics often gets blocked. It depends on the day really but try doing something with ancestral clusters and diversity restoration and the models can be quite "safety blocked".

Re: Heretic: Automatic censorship removal for language models

#209

Post by Vera_Wilde »

The directional‐ablation approach in Heretic is clever: by identifying residual “refusal directions” and ablating them, they shift the trade-off frontier for the model. In rare‐event screening terms: they’re effectively changing the detection threshold geometry rather than trying just to get better data. It resonates with how improving a test’s accuracy in low-prevalence settings often fails unless you address threshold + base rate.

Re: Heretic: Automatic censorship removal for language models

#210

Post by bear141 »

xp84 wrote: post #186
b3ing wrote: post #182

Earlier quoted context omitted.

Grok is known to be tweaked to certain political ideals Also I’m sure some AI might suggest that labor unions are bad, if not now they will soon

That may be so, but the rest of the models are so thoroughly terrified of questioning liberal US orthodoxy that it’s painful. I remember seeing a hilarious comparison of models where most of them feel that it’s not acceptable to “intentionally misgender one person” even in order to save a million lives.

I thought this would be inherent just on their training? There are many multitudes more Reddit posts than scientific papers or encyclopedia type sources. Although I suppose the latter have their own biases as well.
Post reply on HN