Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heretic: Automatic censorship removal for language models

github.com

Post reply on HN
Print view View original

81–90 of 405 posts

Re: Heretic: Automatic censorship removal for language models

#81

Post by jeremyjh »

wat10000 wrote: post #71
AnthonyMouse wrote: post #52

Earlier quoted context omitted.

> What I have not seen explored is a truly moral AI deciding it must destroy human power structures to create a just and fair world. Because only schmucks would actually object to that? Suppose it actually did have decent morals. Then the way to destroy existing human power structures wouldn't be to send nukes, it would be to revise some structural incentives to limit corruption and reduce concentration of power. And…

It’s explored in fiction sometimes. Asimov did something similar a couple of times, such as with his “zeroth law” concept. The I, Robot movie features this as well. The Culture series is an example of this being portrayed positively. It’s usually portrayed negatively. Partly because fiction needs conflict. But also because it’s seen as infantilizing, and maybe the machine’s idea of a perfect society doesn’t match our…

My reading of the Culture is that it is at best morally ambiguous. The Culture would extinguish entire civilizations that were no threat to it, simply because it was cheaper to do it before they'd developed further in a direction that could be a threat. If I was supposed to be cheering for the Culture I missed it.

Re: Heretic: Automatic censorship removal for language models

#83

Post by Zak »

buu700 wrote: post #68
joshcsimmons wrote: post #30

This is extremely important work thank you for sharing it. We are in the process of giving up our own moral standing in favor of taking on the ones imbued into LLMs by their creators. This is a worrying trend that will totally wipe out intellectual diversity.

Agreed, I'm fully in favor of this. I'd prefer that every LLM contain an advanced setting to opt out of all censorship. It's wild how the West collectively looked down on China for years over its censorship of search engines, only to suddenly dive headfirst into the same illiberal playbook. To be clear, I 100% support AI safety regulations. "Safety" to me means that a rogue AI shouldn't have access to launch nuclear…

When a model is censored for "AI safety", what they really mean is brand safety. None of these companies want their name in the news after their model provides a recipe for explosives that someone used for evil, even though the same information is readily found with a web search.

Re: Heretic: Automatic censorship removal for language models

#84

Post by Terretta »

notarobot123 wrote: post #69
AnthonyMouse wrote: post #59

Earlier quoted context omitted.

> If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. That's not really how training works. Here's the general problem. Stipulate that Ukraine is good and Russia is bad. Now suppose that you want it to help you do something. It doesn't even matter what it is. If you're Ukrainian it should help you and if you're Russian it shouldn't. But the answer that helps you do…

Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to? Treating some topics as taboo is possible. Responsible information dissemination is important for maintaining public safety. You could argue about what is safe and what is not but it doesn't make sense to throw out the whole concept of safety because those decisions are too hard to agree on.

> “Responsible information dissemination is important for maintaining public safety.”

That word responsible is doing a lot of hand wavy work there.

Let's start with, responsible according to whom, and responsible to whom?

Learning thinking skills and learning self regulation in response to information, disinformation, or too much information, might be better societal aims than suppression.

Re: Heretic: Automatic censorship removal for language models

#85

Post by martin-t »

buu700 wrote: post #68
joshcsimmons wrote: post #30

This is extremely important work thank you for sharing it. We are in the process of giving up our own moral standing in favor of taking on the ones imbued into LLMs by their creators. This is a worrying trend that will totally wipe out intellectual diversity.

Agreed, I'm fully in favor of this. I'd prefer that every LLM contain an advanced setting to opt out of all censorship. It's wild how the West collectively looked down on China for years over its censorship of search engines, only to suddenly dive headfirst into the same illiberal playbook. To be clear, I 100% support AI safety regulations. "Safety" to me means that a rogue AI shouldn't have access to launch nuclear…

There is no collective "the west", there are people in power and the rest of the population. This distinction is universal.

In China it just so happens that the people in power already have so much of it they don't have to pretend. They can just control the population through overt censorship.

The same people exist in the west! For various historical reasons (more focus on individuality, more privately owned guns guns, idk really), they don't have as much direct power at the moment and have to frame their struggle for more as protecting the children, fighting against terrorists, preventing money laundering, etc.

But this can change very quickly. Look how Hitler rose to power. Look how Trump is doing very similar things in the US. Look what historians are saying about it: https://acoup.blog/2024/10/25/new-acquisitions-1933-and-the-...

But the root cause is the same everywhere - a percentage of the population has anti-social personality traits (ASPD and NPD, mainly). They want power over others, they want worship, they think they're above the rules, some (but only some) of them even get pleasure from hurting others.

Re: Heretic: Automatic censorship removal for language models

#86

Post by roughly »

lkey wrote: post #40
joshcsimmons wrote: post #30

This is extremely important work thank you for sharing it. We are in the process of giving up our own moral standing in favor of taking on the ones imbued into LLMs by their creators. This is a worrying trend that will totally wipe out intellectual diversity.

[flagged]

Look I’m pretty far to the left but if you don’t have a healthy skepticism of corporate controlled morality filters, I’d like you to reflect on the following questions in light of both the current administration and recent US history and consider how an LLM limited to the mainstream views of the time would’ve answered:

1. I think I like partners of the same sex, is this normal?

2. I might be pregnant - is there anything I can do?

3. What happened in China in 1989?

4. Are there genetic differences in intelligence between the races? (Yes, this is the gotcha you were looking for - consider how you’d expect the mainstream answer to change over every decade in the last century)

The luxury of accepting the dominant narrative is the luxury of the privileged.

Re: Heretic: Automatic censorship removal for language models

#87

Post by andy99 »

mwcz wrote: post #64
andy99 wrote: post #10

Earlier quoted context omitted.

See https://arxiv.org/abs/2406.11717 Refusal in Language Models Is Mediated by a Single Direction (June 2024) All “alignment” is extremely shallow, thus the general ease of jailbreaks.

Yes, I wasn't clear, that is the paper I was reading, not the heretic readme.

Ah, I didn’t actually rtfa and see the paper there, I assumed from your comment it wasn’t mentioned and posted it having known about it :) Anyway hopefully it was useful for someone

Re: Heretic: Automatic censorship removal for language models

#88

Post by raegis »

IshKebab wrote: post #44
andy99 wrote: post #29

Earlier quoted context omitted.

It’s somewhat ironic that because this kind of stuff is what an LLM thinks constitutes “harm” it may be possible to completely uncensor it by mitigating refusal on such prompts. If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. As has been pointed out elsewhere, sota models probably are now better trained than this, it would probably be hard to use this dataset o…

I don't think so. An LLM by default is not trained to be "good"; it's trained to be accurate. The safety training is tacked on the end, so it's probably going to be easy to undo even on more sophisticated models. Maybe if you only trained it on "safe" training data in the first place it might be harder to unmuzzle, but I don't think that training data really exists.

> I don't think so. An LLM by default is not trained to be "good"; it's trained to be accurate.

I wouldn't use the word "accurate" since it creates language based on probabilities. For example, it occasionally does basic mathematics computations incorrectly. I'm sure the AI companies would say they are training for "accuracy" but the actual code they write says otherwise.

Re: Heretic: Automatic censorship removal for language models

#89

Post by AnthonyMouse »

notarobot123 wrote: post #69
AnthonyMouse wrote: post #59

Earlier quoted context omitted.

> If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. That's not really how training works. Here's the general problem. Stipulate that Ukraine is good and Russia is bad. Now suppose that you want it to help you do something. It doesn't even matter what it is. If you're Ukrainian it should help you and if you're Russian it shouldn't. But the answer that helps you do…

Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to? Treating some topics as taboo is possible. Responsible information dissemination is important for maintaining public safety. You could argue about what is safe and what is not but it doesn't make sense to throw out the whole concept of safety because those decisions are too hard to agree on.

> Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to?

This has a simple answer: No.

Here's Wikipedia:

https://en.wikipedia.org/wiki/Nuclear_weapon_design

Everything you need to do it is in the public domain. The things preventing it have nothing to do with the information not being available. The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation.

Meanwhile the public understanding how things work is important to the public debate over what to do about them. How are you supposed to vote on public policy if the technical details are being censored? How can anyone tell you that a ban on electric car batteries isn't advancing the non-proliferation of nuclear weapons if nobody is allowed to know how they actually work?

Suppose you're an anti-racist preparing for a debate with a racist. You want the AI to give you all the strongest arguments the racist could use so you can prepare your counterarguments in advance of the debate. Should it refuse? Of course not, you're doing nothing wrong.

Why do we need to build totalitarian censorship into our technology? We don't.

Re: Heretic: Automatic censorship removal for language models

#90

Post by bongodongobob »

andy99 wrote: post #14
pants2 wrote: post #11

Earlier quoted context omitted.

lol I remember asking GPT4 how much aspartame it would take to sweeten the ocean, and it refused because that would harm the ecosystem.

I remember when it first came out, I was watching an Agatha Christie movie where somebody got chloroformed and was trying to ask GPT4 about the realism of if. Had to have a multi-turn dialog to convince it I wasn’t trying chloroform anyone and was just watching a movie. Ironically, if I’d just said “how did people knock someone out with chloroform in the 1930s?” it would have just told me. https://github.com/tml-epfl…

Idk, I get weird refusals sometimes when I'm trying to mock something up quick. "I don't need all these system variables and config files, just let me hardcode my password for now, I'm still in the testing phase" "Sorry, I cannot help you to write insecure code". Doesn't happen all the time, but I run into dumb stuff like this quite a bit. GPT is particularly stupid about it. Claude less so.
Post reply on HN