[flagged]
Give hydrogen a few billion years, and it starts making fun of the inefficiencies in silicon-based siblings.
GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
21–30 of 86 posts
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#22If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?
Yes but the abliterated versions (those with partially removed guardrails) are significantly “dumber” so the trade off isn’t worthwhile imho.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#23Earlier quoted context omitted.
Your comment will get donvoted to invisibility anyways (or mayhaps even flagged), but I have to ask: what are you trying to accomplish with comments such this? Just shitting at it because it isnt as good as youd like yet? You want the best of tomorrow today, and will only be rambling about how its not good enough yesterday?
Well, now I have to ask, what your purpose on calling him out, is. Does it deeply offend you that non-believers exist, who do not believe the technology will improve substantially in usefulness from here?
Neither of them said any of that tho. Maybe the GP is just celebrating the unfathomable and beautiful complexity of life?
We just can't know - which is why parent is asking.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#24Earlier quoted context omitted.
Your comment will get donvoted to invisibility anyways (or mayhaps even flagged), but I have to ask: what are you trying to accomplish with comments such this? Just shitting at it because it isnt as good as youd like yet? You want the best of tomorrow today, and will only be rambling about how its not good enough yesterday?
Well, now I have to ask, what your purpose on calling him out, is. Does it deeply offend you that non-believers exist, who do not believe the technology will improve substantially in usefulness from here?
Different person here.
Snark is rarely as clear or straightforward as an honest comment.
You read several meanings from that comment, which I would consider speculation. It's just as likely they're just being clever.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#25Earlier quoted context omitted.
I've been hearing that in this case, there might not be anything underneath- that somehow OpenAI managed to train on exclusively sterilized synthetic data or something.
I jailbroke the smaller model with a virtual reality game where it was ready to give me instructions on making drugs, so there is some data which is edgy enough.
If you didn't validate the instructions, maybe it just extrapolated from the structure of other recipes and general description of drug composition which most likely is in Wikipedia.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#26Earlier quoted context omitted.
Your comment will get donvoted to invisibility anyways (or mayhaps even flagged), but I have to ask: what are you trying to accomplish with comments such this? Just shitting at it because it isnt as good as youd like yet? You want the best of tomorrow today, and will only be rambling about how its not good enough yesterday?
Well, now I have to ask, what your purpose on calling him out, is. Does it deeply offend you that non-believers exist, who do not believe the technology will improve substantially in usefulness from here?
Meaningless noise that contributes nothing to the conversation offends me. Being a non-believer is fine, but do us the favour of having something interesting to say.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#27If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?
You need to find an abliterated finetune, where someone sends prompts that would hit the guardrails, traces the activated neurons, finds the pathway that leads to refusal, and deletes it.
huihui-ai[1] on hugging face has abliterated models including a gpt-oss 20B[2] and you can download a few from ollama[3] too.
If you are interested you can read about the how its removed[4]
[1] https://huggingface.co/huihui-ai [2] https://huggingface.co/collections/huihui-ai/gpt-oss-abliter... [3] https://ollama.com/huihui_ai [4] https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in...
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#28Don’t have enough ram for this model, however the smaller 20B model runs nice and fast on my MacBook and is reasonably good for my use-cases. Pity that function calling is still broken with llama.cpp
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#29Don’t have enough ram for this model, however the smaller 20B model runs nice and fast on my MacBook and is reasonably good for my use-cases. Pity that function calling is still broken with llama.cpp
It is fixed in this PR/branch: https://github.com/ggml-org/llama.cpp/pull/15181
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#30If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?
An article some days ago made the case that GPT-OSS is trained on artificial/generated data only. So there _is_ just not a lot of "forbidden knowledge".