Ollama is now powered by MLX on Apple Silicon in preview

Post by brcmthrowaway » Tue, Mar 31, 2026, 5:28 AM UTC

Foobar8568 wrote: post #18

HDBaseT wrote: post #11

Earlier quoted context omitted.

You can run Qwen3.5-35B-A3B on 32GB of RAM sure, although to get 'Claude Code' performance, which I assume he means Sonnet or Opus level models in 2026, this will likely be a few years away before its runnable locally (with reasonable hardware).

I fully agree, I run that one with Q4 on my MBP, and the performance (including quality of response) is a let down. I am wondering how people rave so much about local "small devices" LLM vs what codex or Claude code are capable of. Sadly there are too much hype on local LLM, they look great for 5min tests and that's it.

Post by xiconfjs » Tue, Mar 31, 2026, 5:34 AM UTC

brcmthrowaway wrote: post #22

What is the difference between Ollama, llama.cpp, ggml and gguf?

Post by QuantumNomad_ » Tue, Mar 31, 2026, 5:38 AM UTC

AugSun wrote: post #15

aurareturn wrote: post #9

Earlier quoted context omitted.

It isn't going to replace cloud LLMs since cloud LLMs will always be faster in throughput and smarter. Cloud and local LLMs will grow together, not replace each other. I'm not convinced that local LLMs use less electricity either. Per token at the same level of intelligence, cloud LLMs should run circles around local LLMs in efficiency. If it doesn't, what are we paying hundreds of billions of dollars for? I think lo…

Looking at downvotes I feel good about SDE future in 3-5 years. We will have a swamp of "vibe-experts" who won't be able to pay 100K a month to CC. Meanwhile, people who still remember how to code in Vim will (slowly) get back to pre-COVID TC levels.

Post by Ericson2314 » Tue, Mar 31, 2026, 5:45 AM UTC

QuantumNomad_ wrote: post #25

AugSun wrote: post #15

Earlier quoted context omitted.

Looking at downvotes I feel good about SDE future in 3-5 years. We will have a swamp of "vibe-experts" who won't be able to pay 100K a month to CC. Meanwhile, people who still remember how to code in Vim will (slowly) get back to pre-COVID TC levels.

What is CC and TC? I have not heard these abbreviations (except for CC to mean credit card or carbon copy, neither of which is what I think you mean here).

Post by [deleted] » Tue, Mar 31, 2026, 5:47 AM UTC

QuantumNomad_ wrote: post #25

AugSun wrote: post #15

Earlier quoted context omitted.

Looking at downvotes I feel good about SDE future in 3-5 years. We will have a swamp of "vibe-experts" who won't be able to pay 100K a month to CC. Meanwhile, people who still remember how to code in Vim will (slowly) get back to pre-COVID TC levels.

What is CC and TC? I have not heard these abbreviations (except for CC to mean credit card or carbon copy, neither of which is what I think you mean here).