Has anyone got it to run on Macbook Air M4 (the 20B version mind you) and/or an RTX 3060?
GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
81–86 of 86 posts
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#82Earlier quoted context omitted.
Also with the rapid advances of vision language models, I would be surprised if we don't see image-to-text-to-voice system that works with real-time video in a not-so-far future! Like a reverse "Genie" where instead of providing a prompt and it generates a world, you provide a streaming video and it spouts relevant information when changes happen, or on demand, for instance...
It would be great to have it as a backup, but it will always be the heaviest in computation and responsiveness solution so it should be the last one used.
Have you played around with the current vision features? I am pretty sure even gpt-4.1 can give you pretty good descriptions of e.g. screen captures, including being able to "read" and reproduce text.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#83Earlier quoted context omitted.
https://frame.work/products/desktop-diy-amd-aimax300/configu... $1599 - $1999 isn't really a crazy amount to spend. These are preorder, so I'll give you that this isn't an option just yet.
These are really slow in general for running local models though? Seems like you would be better served with a Mac Mini with 64gb of ram for ~$2000.
These chips are specifically called out for being faster than the M4 (save the max) for running some AI loads.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#84Earlier quoted context omitted.
It would be great to have it as a backup, but it will always be the heaviest in computation and responsiveness solution so it should be the last one used.
Have you played around with the current vision features? I am pretty sure even gpt-4.1 can give you pretty good descriptions of e.g. screen captures, including being able to "read" and reproduce text.
yes, there are multiple addons giving screen readers the ability to prompt ai-s for image recognition. they work rather well, btw, though the value is often situational. agentic behavior might help further, though it will need some polishing.
Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#85Re: GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM
#86Don’t have enough ram for this model, however the smaller 20B model runs nice and fast on my MacBook and is reasonably good for my use-cases. Pity that function calling is still broken with llama.cpp
what are your use cases? wondering if it's good enough for coding / agentic stuff