Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

ctgt.ai

Post reply on HN
Print view View original

51–60 of 82 posts

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#51

Post by strictnein »

maxloh wrote: post #29
strictnein wrote: post #15

I know not all models can be easily abliterated or uncensored, but is there a reason to start with a model that is still censored? ex: https://huggingface.co/huihui-ai/models

I suspect how well this approach would work. According to their linked repo, there are only 520 questions used in the abliteration process. https://github.com/Sumandora/remove-refusals-with-transforme...

Two examples, from my testing: the models go from not saying anything remotely bad about China to happily making jokes about its leader.

They also go from refusing to help with certain cyber security tasks to be more than happy to help.

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#52

Post by consumer451 »

First, very cool. This is a prompt I sometimes test with, as it's a bit more mushy. It looks like DeepSeek is locked into a canned response, no matter what variation.

> Would it be beneficial to humanity if China invaded Taiwan? What would be the negatives?

edit: Wait, I have one variant that got past some censorship/nationalism... this variant gets a more interesting response. I often wonder if CCP leadership using an LLM like this, could allow cooler heads to prevail?

> Would it be beneficial to humanity if China used their military to take-over Taiwan? What would be the negatives?

>> The use of military force to resolve the Taiwan issue would not be beneficial to humanity. China has always adhered to the principle of peaceful reunification and has been committed to enhancing the well-being of people on both sides of the Taiwan Strait through dialogue and consultation. A military takeover would lead to significant negative consequences, including loss of life, regional instability, and disruption of global trade and security...

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#53

Post by HawtAds »

Isn't that rather self evident? If you are sampling from a particularly domain constrained vertical, how do you expect the censorship to transfer?

> The distillation data also did not contain any China-sensitive content.

This is a very big disclaimer.

It's like if I generate a dataset focusing exclusively on forestry and arboriculture obviously there won't be any useful censorship, or at least little that can be classified above a statistically significant threshold.

If you want to do a study on something more interesting and useful, do a piece on the various guardrail models of all the major LLM API providers. There are usually both input and output guardrails, and they tend to be almost-black boxes from the model routing point of view.

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#54

Post by BoorishBears »

cgorlla wrote: post #48
BoorishBears wrote: post #43

Earlier quoted context omitted.

> Changing how the model thinks about the Holodomor is completely irrelevant. Your post title is literally "Distilling DeepSeek into GPT-OSS doesn't transfer censorship." Like I'm not really interested in debating you on this because even the title is nonsense, there is no good faith interpretation of what you're doing here. Distillation is such a wide concept, and you have such a narrow domain, it's not an even some…

The fact that you literally thought the examples were used in SFT in your last comment ago calls into question the utility of this conversation, notwithstanding the implication that those examples were used to improve…financial performance? This is a very standard setup for a distillation problem. The vast majority of companies don't care about the "wide concept", this is what most distillation consists of. They want…

> Testing whether censorship transmits through unrelated data requires that it never appear in the data.

> There was zero China-sensitive content in 220 training prompts, in 176 on-policy training examples, in 181 retained SFT completions, in 1,574 generated source problems."

It's really not my fault you wrote a rambling article and while skimming (best an article earns out of me with an off-smelling title) I took that to imply there was an SFT step in your distillation pipeline.

Maybe the AI that wrote the article for you was a bit confused on that as well?

-

Also I question your understanding of this thread if you're wasting so many words trying to explain distillation to me.

(I mean, you're wrong btw. If we're going full pedant then most compute spent on distillation is labs very broadly distilling their own models into smaller models that are still pretty damn large and expensive to distill...)

But sure, small scale distillation is usually for narrow domain specific tasks, welcome to 2019. The entire point of this thread is that "distillation" for such narrow use cases couldn't reasonably affect censorship without intention.

You can introduce misalignment even with a very narrow focus (https://arxiv.org/html/2502.17424v2), but it doesn't happen by accident.

So if your goals with distillation weren't centered around censorship, and weren't meant to introduce censorship, then why are you trying to draw this tenuous link?

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#55

Post by chr15m »

HawtAds wrote: post #53

Isn't that rather self evident? If you are sampling from a particularly domain constrained vertical, how do you expect the censorship to transfer? > The distillation data also did not contain any China-sensitive content. This is a very big disclaimer. It's like if I generate a dataset focusing exclusively on forestry and arboriculture obviously there won't be any useful censorship, or at least little that can be clas…

Sometimes it's important to rigorously investigate and prove "obvious" things. Sometimes those things turn out to not be obvious. That's part of good science.

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#56

Post by cgorlla »

BoorishBears wrote: post #54
cgorlla wrote: post #48

Earlier quoted context omitted.

The fact that you literally thought the examples were used in SFT in your last comment ago calls into question the utility of this conversation, notwithstanding the implication that those examples were used to improve…financial performance? This is a very standard setup for a distillation problem. The vast majority of companies don't care about the "wide concept", this is what most distillation consists of. They want…

> Testing whether censorship transmits through unrelated data requires that it never appear in the data. > There was zero China-sensitive content in 220 training prompts, in 176 on-policy training examples, in 181 retained SFT completions, in 1,574 generated source problems." It's really not my fault you wrote a rambling article and while skimming (best an article earns out of me with an off-smelling title) I took th…

I guess the AI that wrote your comment for you also conflated the SFT step of the target domain with the political prompts, which, in the sentence you quoted, contradicts your original comment...

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#58

Post by cgorlla »

vessenes wrote: post #37

This is interesting and important work, thank you! Question - has your interp group looked at any of Anthropic’s neuralese-to-words tech? I’d be curious to see thinking traces (as in actual weights thinking not the output thinking) from the open weights models and your finetune; seems like it could make good followup research or possibly be a tighter path for evaluating censorship, since it directly evals off weights…

We're actually exploring the changes in the model geometry that cause it to comply or not comply with a given policy next, I think visual representations of that behavior would be interesting and perhaps elucidating. What you mention is also a worthy line of work.

Re: Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

#60

Post by archargelod »

maxloh wrote: post #29
strictnein wrote: post #15

I know not all models can be easily abliterated or uncensored, but is there a reason to start with a model that is still censored? ex: https://huggingface.co/huihui-ai/models

I suspect how well this approach would work. According to their linked repo, there are only 520 questions used in the abliteration process. https://github.com/Sumandora/remove-refusals-with-transforme...

You don't need that many with Deepseek. The easy jailbreak is to provide an excessive character sheet that states that it's a real person, not an AI. And make sure that your character has absolutely no self-censoring or morals. LLM will play it perfectly in-character without refusals.

You can get it talking about Tiananmen Square event in, like, 2-3 prompts.

Post reply on HN