Ollama is now powered by MLX on Apple Silicon in preview • Hacker News Board

Re: Ollama is now powered by MLX on Apple Silicon in preview

#221

Post by beepbooptheory » Tue, Mar 31, 2026, 1:54 PM UTC

corndoge wrote: post #216

dgacmu wrote: post #210

Earlier quoted context omitted.

The pile of shell and sed is cleaning up the ai output and then running it in the shell. The instruction to the AI was to create _a_ shell command. So it's a random shell command generator (maybe).

that part is the system prompt, the script is a function that takes a prompt describing a shell command as an argument

But it's gotta be just a joke right? Which is why all the examples are just classic things you do with bash/unix utilities?

I'll just say, if not a joke, the bit is appreciated either way!

"AI change to the home directory. Make it snappy!"

Re: Ollama is now powered by MLX on Apple Silicon in preview

#222

Post by dudefeliciano » Tue, Mar 31, 2026, 1:56 PM UTC

pama wrote: post #191

dgb23 wrote: post #174

Earlier quoted context omitted.

> False, it creates consumer demand for inference chips, which will be badly utilised. I think the opposite is true. Local inference doesn't have to go over the wire and through a bunch of firewalls and what have you. The performance from just regular consumer hardware with local, smaller models is already decent. You're utilizing the hardware you already have. > The performance limitations are inherent to the limite…

Parallel inference on large compute scales in superlinear ways. There is no way to beat the reduction in memory transfers that a data-center inference model provides with hardware that fits at anything called a home. It is much more energy efficient to process huge batches of parallel requests compared to having one or a handful of queries running on an accelerator.

Aren't data centers extremely energy inneficient due to network latency, memory bottlenecks and so on? I mean the models that run on them are extremely powerful compared to what you can run on consumer hardware, but I wouldn't call them efficient...

Re: Ollama is now powered by MLX on Apple Silicon in preview

#223

Post by abu_ameena » Tue, Mar 31, 2026, 2:01 PM UTC

On-device models are the future. Users prefer them. No privacy issues. No dealing with connectivity, tokens, or changes to vendors implementations. I have an app using Foundation Model, and it works great. I only wish I could backport it to pre macOS 26 versions.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#224

Post by pram » Tue, Mar 31, 2026, 2:02 PM UTC

harrouet wrote: post #188

As being on the market for a new mac and comparing refub M4 Max vs M5 _Pro_, I am interested in how much faster the neural engines are -- compared to marketing claims.

M4 Max is going to be faster.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#225

Post by ranjeethacker » Tue, Mar 31, 2026, 2:04 PM UTC

I used today, working nicely.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#226

Post by podlp » Tue, Mar 31, 2026, 2:11 PM UTC

franze wrote: post #86

I created "apfel" https://github.com/Arthur-Ficial/apfel a CLI for the apple on-device local foundation model (Apple intelligence) yeah its super limited with its 4k context window and super common false positives guardrails (just ask it to describe a color) ... bit still ... using it in bash scripts that just work without calling home / out or incurring extra costs feels super powerful.

Neat! I’ve actually been building with AFM, including training some LoRA adapters to help steer the model. With the right feedback mechanisms and guardrails, you can even use it for code generation! Hopefully I’ll have a few apps and tools out soon using AFM. I think embedded AI is the future, and in the next few years more platforms will come around to AI as a local API call, not an authorized HTTP request. That said, AFM is still incredibly premature and I’m experimenting with newer models that perform much better.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#227

Post by xiconfjs » Tue, Mar 31, 2026, 2:11 PM UTC

yard2010 wrote: post #81

xiconfjs wrote: post #23

Earlier quoted context omitted.

Ollama on MacOS is a one-click solution with stable obe-click updates. Happy so far. But the mlx support was the only missing piece for me.

Can you please write about your hardware?

* macOS 26.x on MacBookPro M1 Max 32GB * Ollama on macOS, cursor to play around * Open WebUI [1] on my Homeserver via API to Ollama (also for remote „A.I.“ access) * running gpt-oss:20b, qwen3.5:9b with ease, qwen3.5:27b for more complex tasks

[1] https://github.com/open-webui/open-webui

Re: Ollama is now powered by MLX on Apple Silicon in preview

#228

Post by konschubert » Tue, Mar 31, 2026, 2:12 PM UTC

ekianjo wrote: post #164

konschubert wrote: post #152

Earlier quoted context omitted.

I disagree with every sentence of this. > solves the problem of too much demand for inference False, it creates consumer demand for inference chips, which will be badly utilised. > also would use less electricity What makes you think that? (MAYBE you can save power on cooling. But not if the data center is close to a natural heat sink) > It's just a matter of getting the performance good enough. The performance limit…

> What makes you think that? Looking at actual users of LLMs

While not everybody is a professional in YOUR domain, many people are professionals in SOME domain. And even outside of that, they deserve a smart conversation partner, for example on topics like health and politics.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#229

Post by eddieroger » Tue, Mar 31, 2026, 2:13 PM UTC

robotswantdata wrote: post #83

Why are people still using Ollama? Serious. Lemonade or even llama.cpp are much better optimised and arguably just as easy to use.

`ollama serve` and `ollama run`

The devex is great and familiar to folks who have used Docker. Reading through the Lemonade documentation, it seems like a natural migration, but we're talking about two steps for getting started versus just one. So I'd need a reason to make that much change when I'm happy enough with Ollama.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#230

Post by konschubert » Tue, Mar 31, 2026, 2:14 PM UTC

locknitpicker wrote: post #167

konschubert wrote: post #152

Earlier quoted context omitted.

I disagree with every sentence of this. > solves the problem of too much demand for inference False, it creates consumer demand for inference chips, which will be badly utilised. > also would use less electricity What makes you think that? (MAYBE you can save power on cooling. But not if the data center is close to a natural heat sink) > It's just a matter of getting the performance good enough. The performance limit…

> What makes you think that? The fact that today's and yesterday's models are quite capable of handling mundane tasks, and even companies behind frontier models are investing heavily in strategies to manage context instead of blindly plowing through problems with brute-force generalist models. But let's flip this around: what on earth even suggests to you that most users need frontier models?

Everybody has difficult decisions to make in their daily lives and in their work.

Having access to a model that is drawing from good sources and takes time to think instead of hallucinating a response is important in many domains of life.