GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
old.reddit.com
GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
1–10 of 86 posts
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#2[flagged]
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#3[deleted]
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#4I have a 5950x with 128 gb ram and a 12 gb 3060 gpu.
The speed of generating tokens is excellent, the killer is that when the context grows even a little processing of it is super slow.
Hopefully someone smart will optimize this, but as it is now I keep using other models like qwen, mistral and gemma.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#5I have a 5950x with 128 gb ram and a 12 gb 3060 gpu. The speed of generating tokens is excellent, the killer is that when the context grows even a little processing of it is super slow. Hopefully someone smart will optimize this, but as it is now I keep using other models like qwen, mistral and gemma.
I would so appreciate concrete data instead of subjectivities like "excellent" and "super slow".
How many tokens is excellent? How many is super slow? How many is non-filled context?
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#6I have a 5950x with 128 gb ram and a 12 gb 3060 gpu. The speed of generating tokens is excellent, the killer is that when the context grows even a little processing of it is super slow. Hopefully someone smart will optimize this, but as it is now I keep using other models like qwen, mistral and gemma.
I would so appreciate concrete data instead of subjectivities like "excellent" and "super slow". How many tokens is excellent? How many is super slow? How many is non-filled context?
People can read at a rate around 10 token/sec. So faster than that is pretty good, but it depends how wordy the response is (including chain of thought) and whether you'll be reading it all verbatim or just skimming.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#7I have a 5950x with 128 gb ram and a 12 gb 3060 gpu. The speed of generating tokens is excellent, the killer is that when the context grows even a little processing of it is super slow. Hopefully someone smart will optimize this, but as it is now I keep using other models like qwen, mistral and gemma.
I would so appreciate concrete data instead of subjectivities like "excellent" and "super slow". How many tokens is excellent? How many is super slow? How many is non-filled context?
I'm not really timing it as I just use these models via open webui, nvim and a few things I've made like a discord bot, everything going via ollama.
But for comparison, it is generating tokens about 1.5 times as fast as gemma 3 27B qat or mistral-small 2506 q4. Prompt processing/context however seems to be happening at about 1/4 of those models.
A bit more concrete of the "excellent", I can't really notice any difference between the speed of oss-120b once the context is processed and claude opus-4 via api.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#8I have a 5950x with 128 gb ram and a 12 gb 3060 gpu. The speed of generating tokens is excellent, the killer is that when the context grows even a little processing of it is super slow. Hopefully someone smart will optimize this, but as it is now I keep using other models like qwen, mistral and gemma.
I would so appreciate concrete data instead of subjectivities like "excellent" and "super slow". How many tokens is excellent? How many is super slow? How many is non-filled context?
Some numbers are posted in the comments:
> … you can expect the speed to half when going from 4k to 16k long prompt …
> … it did slow down somewhat (from 25T/s to 18T/s) for very long context …
Depends on the hardware configuration (size of VRAM, speed of CPU and system RAM) and llama.cpp parameter settings, a bigger context prompt slows the T/s number significantly but not order of magnitudes.
Facit: gpt-oss 120B on a small GPU is not the proper setup for chat use cases.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#9If you run these on your own hardware can you take the guard-rails off (ie "I'm afraid I can't assist with that"), or are they baked into the model?
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#10I have a 5950x with 128 gb ram and a 12 gb 3060 gpu. The speed of generating tokens is excellent, the killer is that when the context grows even a little processing of it is super slow. Hopefully someone smart will optimize this, but as it is now I keep using other models like qwen, mistral and gemma.
What are you aiming to do with these models that isn’t chat/text manipulation?