101–110 of 179 posts
Earlier quoted context omitted.
Definitely do it. You get a lot of deep research, access to GPT5 Pro, Sora and the Codex limits are MUCH higher.
Curious why this is downvoted? Wrong information?
I hope the people downvoting get some minor joy out of it, I know you need it.
Earlier quoted context omitted.
I am currently using a local model qwen3:8b running on a 2020 (2018 intel chip) Mac mini for classifying news headlines and it's working decently well for my task. Each headline takes about 2-3 seconds but is pretty accurate. Uses about 5.3 gigs of ram.
Can you expand a bit on your software setup? I thought running local models was restricted to having expensive GPUs or latest Apple Silicon with unified memory. I have a Intel 11th gen home server which I would like to use to run some local model for tinkering if possible.
If you have 8GB of RAM you can even try running them directly in Chrome via WebAssembly. Here's a demo running a model that's less than 1GB to load, entirely in your browser (and it worked for me in mobile safari just now): https://huggingface.co/spaces/cfahlgren1/Qwen-2.5-WebLLM
Seems to completely ignore usage of local/free models as well as anything but Sonnet/ChatGPT. So my confidence in the good faith of the author is... heavily restricted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
Using Qwen3:32b on a 32GB M1 Pro may not be "close to cloud capabilities" but it is more than powerful enough for me, and most importantly, local and private.
As a bonus, running Asahi Linux feels like I own my Personal Computer once again.
Earlier quoted context omitted.
Today. But what about in 5 years? Would you bet we will be paying hundreds of billions to OpenAI yearly or buying consumer GPUs? I know what I will be doing.
But the progress goes both ways: In five years, you would still want to use whatever is running on the cloud supercenters. Just like today you could run gpt-2 locally as a coding agent, but we want the 100x-as-powerful shiny thing.
Earlier quoted context omitted.
Honestly though how many people reading this do you think have that setup vs. 85% of us being on a MBx? > The Qwen3-Coder-480B-A35B model could also be run on a 4090 or 5090 by splitting the active 35B parameters across VRAM and RAM. Reminds me of running Doom when I had to hack config.sys to forage 640KB of memory. Less than 0.1% of the people reading this are doing that. Me, I gave $20 to some cloud service and I c…
>I can do whatever the hell I want from this M1 MBA in a hotel room in Japan. As long as it's within terms and conditions of whatever agreement you made for that $20. I can run queries on my own inference setup from remote locations too
Even for non-developer use cases o3 is a much better model for me than GPT5 on any setting. 30 seconds-1 minute is just the time I am patient enough to wait as that's the time I am spending on writing a question. Faster models just make too many mistakes / don't understand the question.
GPT-$ is the money gpt in my opinion. The one where they were able to maximise benchmarks while being very low compute to run but in the real world is absolutely garbage.
For development use cases, I switched to Sonnet 4.5 and haven't looked back. I mean, sure, sometimes I also use GPT-5 (and mini) and Gemini 2.5 Pro (and Flash), and also Cerebras Code just switched to providing GLM 4.6 instead of the previous Qwen3 Coder so those as well, but in general the frontier models are pretty good for development and I wouldn't have much reason to use something like Sonnet 4 or 3.7 or whateve…
What tool are you using to enable switching between so many models?
I mean, this is technically false, right? They’re not running these models but calling the APIs? Not that it matters.
Earlier quoted context omitted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
Agreed, GPU is the expensive route, especially when I was looking at external GPU solutions. Using Qwen3:32b on a 32GB M1 Pro may not be "close to cloud capabilities" but it is more than powerful enough for me, and most importantly, local and private. As a bonus, running Asahi Linux feels like I own my Personal Computer once again.
Running smaller models on Apple Silicon is kinder on the environment/energy use and has privacy benefits for corporate use.
Using a hybrid approach makes sense for many use cases. Everyone gets to make their own decisions; for me, I like to factor in externalities like social benefit, environment, and wanting the economy to do as well as it can in our new post-mono polar world.