Ask HN: How do you implement censorship to a LLM?

#1

Post by mcmoor » Fri, Jun 23, 2023, 6:00 AM UTC

OpenAI keeps increasingly adding censorship to its LLM to comply to various laws. But I'm confused how do they do that? I thought it's impossible to incorporate it while training because it seems like the LLM doesn't have those censors before and now they do (?). But it also seems very unlikely that they tinker with the neural nodes directly.

The only method I can think of is that they add lots of prompt themselves that's added automatically with every user prompt. Something like "You may not talk about X. You may not talk about Y." before user prompts. If it's like that, it explains why users can jailbreak the censorship, we just have to overpower those censors prompts.

Re: Ask HN: How do you implement censorship to a LLM?

#2

Post by rubbarndt » Fri, Jun 23, 2023, 6:05 AM UTC

[dead]

Re: Ask HN: How do you implement censorship to a LLM?

#3

Post by ClassyJacket » Fri, Jun 23, 2023, 6:05 AM UTC

I've wondered the same thing. I thought they would have to re-train it to respond differently to certain prompts -- aside from hardcoding responses to certain phrases, however, from how creative the DAN prompts have had to get it seems like the censorship is more intelligent than that.

I don't think it's the second answer, (adding pre-prompts before the user prompt), because when they do that, it's possible to hack it back to reading out the previous prompt - that's how we found out Bing was internally named Sydney. This would also reduce the number of tokens that could be input by the user.

I'd love to know the answer to this too.

Re: Ask HN: How do you implement censorship to a LLM?

#4

Post by lionkor » Fri, Jun 23, 2023, 6:38 AM UTC

I feel like this is an open problem because its the wrong approach.

Censoring AI makes little sense - its like censoring what I can and cannot write on paper. A pen that only allows me to write nice words, or a dictionary with no "controversial" words, or a history book without any controversial people or content, is not useful.

AI is a tool, and it makes sense for that tool, if emulating human behavior, to respond with controversial and illegal content if prompted to do so. The obvious (to me) approach would be to handle this like any other case - let it generate whateever its prompted to generate.

It wont generate horrible war crimes when asked about the weather. The people asking it for porn roleplay, or recipes for crystal meth, are deriving value from that. If they end up actually making crystal meth, its on THEM, not the LLM, arguably.

You wouldn't ban all true crime books just because someone may reproduce a crime from the book, or be offended at it.

All this censorship is is OpenAI wanting to stay trendy and make a clean, sanitized, happy corporate yes man monkey. And for that, maybe they should have trained the LLM on wikipedia only.

Re: Ask HN: How do you implement censorship to a LLM?

#5

Post by realusername » Fri, Jun 23, 2023, 6:38 AM UTC

I'm guessing it's just a mix of pre-prompting and a second LLM which is checking if the output needs to be censored

Re: Ask HN: How do you implement censorship to a LLM?

#6

Post by breput » Fri, Jun 23, 2023, 6:46 AM UTC

I wonder if it is possible to have a second, independent LLM evaluate the output of the primary LLM and enforce the restrictions?

No matter how you could "outsmart" the initial restrictions, the second pass would detect that restricted content was in the response and block it. I would even make it permissive on an A/B testing basis to allow restricted responses but flag the account and interactions for human(?) review to learn the techniques to tighten the system.

Re: Ask HN: How do you implement censorship to a LLM?

#7

Post by weare138 » Fri, Jun 23, 2023, 7:38 AM UTC

I'm assuming the prompt filter system runs independently from the AI system and either the AI is then fed scrubbed inputs or the prompt is just kicked out altogether.

Re: Ask HN: How do you implement censorship to a LLM?

#8

Post by rat9988 » Fri, Jun 23, 2023, 7:40 AM UTC

lionkor wrote: post #4

I feel like this is an open problem because its the wrong approach. Censoring AI makes little sense - its like censoring what I can and cannot write on paper. A pen that only allows me to write nice words, or a dictionary with no "controversial" words, or a history book without any controversial people or content, is not useful. AI is a tool, and it makes sense for that tool, if emulating human behavior, to respond w…

It's not an open problem because it's the wrong approach. I understand you don't want to censor ai, but it doesn't mean there aren't technical solutions to the problem. And if there are no technical solution, it's still not because of your explanation.

Re: Ask HN: How do you implement censorship to a LLM?

#9

Post by victor9000 » Fri, Jun 23, 2023, 8:04 AM UTC

I assume it's done with fine tuning. Meaning you come up with responses that capture topics you want to avoid, then let the model convert these unwords into their corresponding latent space, and train it to exclude that space altogether.

Re: Ask HN: How do you implement censorship to a LLM?

#10

Post by MagicMoonlight » Fri, Jun 23, 2023, 8:10 AM UTC

They put things in the prompt and they fine-tune with no-no thoughts and appropriate responses.

Obviously that makes it worse and worse the more you do but it protects the public from badthink.