Live data from Hacker News

GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

old.reddit.com

11–20 of 86 posts

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#11
post #9

If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?

they're baked in but there's a community of people who crack and modify them

even chat gpt will help you crack them if you ask it nicely

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#12
post #9

If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?

You need to find an abliterated finetune, where someone sends prompts that would hit the guardrails, traces the activated neurons, finds the pathway that leads to refusal, and deletes it.

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#15
post #13

LLM noob here. Would this optimization work with any MoE model or is it specific for this one?

It's just doing a regex on the layer names, so should work with other models as long as they have the expert layers named similarly.

It worked with Qwen 3 for me, for example.

The option is just a shortcut, you can provide your own regex to move specific layers to specific devices.

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#16
post #9

If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?

You need to find an abliterated finetune, where someone sends prompts that would hit the guardrails, traces the activated neurons, finds the pathway that leads to refusal, and deletes it.

I've been hearing that in this case, there might not be anything underneath- that somehow OpenAI managed to train on exclusively sterilized synthetic data or something.

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#17

Earlier quoted context omitted.

You need to find an abliterated finetune, where someone sends prompts that would hit the guardrails, traces the activated neurons, finds the pathway that leads to refusal, and deletes it.

I've been hearing that in this case, there might not be anything underneath- that somehow OpenAI managed to train on exclusively sterilized synthetic data or something.

I jailbroke the smaller model with a virtual reality game where it was ready to give me instructions on making drugs, so there is some data which is edgy enough.

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#18

[flagged]

Your comment will get donvoted to invisibility anyways (or mayhaps even flagged), but I have to ask: what are you trying to accomplish with comments such this? Just shitting at it because it isnt as good as youd like yet? You want the best of tomorrow today, and will only be rambling about how its not good enough yesterday?

Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM

#19
post #18

[flagged]

Your comment will get donvoted to invisibility anyways (or mayhaps even flagged), but I have to ask: what are you trying to accomplish with comments such this? Just shitting at it because it isnt as good as youd like yet? You want the best of tomorrow today, and will only be rambling about how its not good enough yesterday?

Well, now I have to ask, what your purpose on calling him out, is. Does it deeply offend you that non-believers exist, who do not believe the technology will improve substantially in usefulness from here?
Post reply on HN