Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama is now powered by MLX on Apple Silicon in preview

ollama.com

Post reply on HN
Print view View original

321–330 of 384 posts

Re: Ollama is now powered by MLX on Apple Silicon in preview

#321

Post by podlp »

adam_patarino wrote: post #250
podlp wrote: post #232

Earlier quoted context omitted.

Rig sounds cool, I just joined the waitlist! I’m building something similar although with a much narrower purpose. Excited to learn more

Tell me more! Thanks for the waitlist

Sent a LinkedIn request. I’m building a language-specific coding agent using Apple Intelligence with custom adapters. It’s more a proof-of-concept at this point, but basic functionality actually works! The 4K context window is brutal, but there’s a variety of techniques to work around it. Tighter feedback loops, linters, LSPs, and other tools to vet generated code. Plus mechanisms for on-device or web-based API discovery. My hypothesis is if all this can work “well enough” for one language/ runtime, it could be adapted for N languages/ runtimes.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#322

Post by KerrAvon »

charcircuit wrote: post #306
roadside_picnic wrote: post #285

Earlier quoted context omitted.

> Users don’t care about “privacy”. I worked for a research focused AI startup that had a strict "no external LLM" policy for code touching our core research. You're right that the average consumer doesn't care about privacy, but there are many, many users who do. The average consumer also don't have a desktop with GPU or high end Mac Studio, but that doesn't mean there aren't many people working with AI how do have…

Those users are addressed by being able to rent their own exclusive machines to run the model on. There will be some compromise that will be made to get access to the best intelligence available.

As one of those users: absolutely fucking not.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#323

Post by raw_anon_1111 »

nozzlegear wrote: post #304
raw_anon_1111 wrote: post #261

Earlier quoted context omitted.

People A) don’t have to use Meta and B) do have a choice between not using a mobile phone by an ad tech company.

People don't have a choice between Facebook and not-Facebook-but-still-has-all-of-your-friends-and-family. Abstinence isn't a choice here any more than shutting off your cell phone service is a choice; true in the literal sense, but only if you don't mind being unreachable to everyone who still has a phone.

And they do have a choice on proactively giving FB more information than just what it infers

Re: Ollama is now powered by MLX on Apple Silicon in preview

#324

Post by Casteil »

a-dub wrote: post #218

is local llm inference on modern macbook pros comfortable yet? when i played with it a year or so ago, it worked fairly ok but definitely produced uncomfortable levels of heat. (regarding mlx, there were toolkits built on mlx that supported qlora fine tuning and inference, but also produced a bunch of heat)

It's gotten significantly better with the advent of local/offline MoE models (e.g. qwen3.5:35b-a3b, qwen3:30b-a3b, gpt-oss:20b-3.6b), which offer a good balance of prompt response speed and output quality.

'Dense' models of yesteryear (e.g. llama:70b, gemma2/3:27b) tend to be significantly slower by comparison, therefore, your hardware spends a lot more time 'maxed out' for a given prompt.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#327

Post by Nevermark »

raw_anon_1111 wrote: post #319
Nevermark wrote: post #307

Earlier quoted context omitted.

Have you done A/B tests to see if consumers prefer Facebook with or without privacy? No? What? Oh, you can't? Neither can consumers. Most consumers are very aware of the lack of privacy, the manipulation, and have very cynical feelings about Facebook and similar companies. But it's where their friends and family are. For most people the web is a mine field maze where basic things they want are compromised everywhere.…

Consumers pro actively tell Facebook their age, sexual preference, race, relationship status, likes and dislikes, they check in to where they are and who they are there with… They are choosing to give Facebook info.

> They are choosing to give Facebook info.

Yes, they do. That's is exactly the phenomena my comment addressed.

But the way you wrote that implies an improbable motivation or choice framing.

Perhaps their real motive/choice is to share with other people on the site.

It is called a network effect.

If (1) Facebook had been the surveillance/manipulation capital of the world from inception, (2) an equally inviting privacy protecting site took off at the same time, and (3) everyone chose Facebook over E2EE anyway, then sure, we could throw up our hands! Those silly users!

The term I have for when people discuss choices involving many-dimensional criteria, as if the choice involved just one or two selected dimensions, is "dimension blindness". It happens in a lot of heated discussions about phone choices too.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#328

Post by sbassi »

mgaunard wrote: post #292
abu_ameena wrote: post #223

On-device models are the future. Users prefer them. No privacy issues. No dealing with connectivity, tokens, or changes to vendors implementations. I have an app using Foundation Model, and it works great. I only wish I could backport it to pre macOS 26 versions.

These local models are far behind the capabilities of latest Gemini Pro, Claude Opus or GPT. Why waste time with subpar AI?

It's a trade off.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#329

Post by victords »

bwfan123 wrote: post #252

What is the cheapest usable local rig for coding ? I dont want fancy agents and such, but something purpose built for coders, and fast-enough for my use, and open-source, so I can tweak it to my liking. Things are moving fast, and I am hesitant to put in 3-4K now in the hope that it would be cheaper if i wait.

As mentioned before, I think Apple hardware is the best alternative right now.

Mac Studio, Mac Mini, MacBook Pro, you can find even some used ones with enough RAM that will run models like Qwen reasonably well.

I'm using a M1 Max MacBook Pro and it runs Qwen 3.5 on Ollama (without MLX) at a decent speed.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#330

Post by fauigerzigerk »

mrinterweb wrote: post #290
abu_ameena wrote: post #223

On-device models are the future. Users prefer them. No privacy issues. No dealing with connectivity, tokens, or changes to vendors implementations. I have an app using Foundation Model, and it works great. I only wish I could backport it to pre macOS 26 versions.

I think two recent advances make your statement more true. The new Qwen 3.5 series has shown a relatively high intelligence density, and Google's new turboquant could result in dramatically smaller/efficient models without the normal quantization accuracy tradeoff. I would expect consumer inference ASIC chips will emerge when model developments start plateauing, and "baking" a highly capable and dense model to a chip…

Who will be funding state of the art local models going forward? AI models are never done or good enough. They will have to be trained on new data and eventually with new model architectures. It will remain an expensive exercise.

I could be wrong because I'm not following this too closely, but the open weights future of both Llama and Qwen looks tenuous to me. Yes, there are others, but I don't understand the business model.

Post reply on HN