Ask HN: How do you implement censorship to a LLM?

They have a huge training set that is composed of two types of data. The first set of data is the raw data of text from the internet. The second set of data is questions and answers that it should learn from.

That second set can have a prompt "What do you think about this race?", and then the LLM answers. Using NLP, they judge the answer with a point based system to inhibit certain behavior or not. I think the dataset is very large. It includes some questions with the answer. They definitely have a training step that asks the question, sees how the model responds, and then judges the answer.

The LLM that they run is open source. The real secret sauce is the training data and process. They have revealed things here and there.

I'm not sure what exactly you mean by censorship here because nothing that I know of that they do is actually censorship. In particular they don't do the sort of prompt stuffing you're talking about, because that would just use up the context window of the LLM and you'd know because the context window is public and as a dev you can use the whole window. There is a "system" role that allows application developers to "whisper in the ear" of the LLM and provide prompts into the context that are different from the user prompts. That's how you build chat apps where you have a user providing some instruction but you have some overall rails around the conversation.

That being said, the question of how to account for and deal with bias in llms is an active area of research, and (like everything else in llms) comes down to vector math. See https://arxiv.org/pdf/2106.13219.pdf for example.

Content moderation in openAI is the subject of this paper https://arxiv.org/abs/2208.03274 and they have published their api here https://openai.com/blog/new-and-improved-content-moderation-... That also includes their reference dataset for training.

You could put an additional layer between the LLM output and the user, and filter than. Maybe an additional model trained to detect or even rewrite "offensive" content.

Strategically prompted LLM "layers" on top of the original LLM => High probability of successful censorship

You're expecting far too much from the generation. It doesn't even know what a centaur is. I don't think it's been trained exhaustively on fantastic creatures.