Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama is now powered by MLX on Apple Silicon in preview

ollama.com

Post reply on HN
Print view View original

271–280 of 384 posts

Re: Ollama is now powered by MLX on Apple Silicon in preview

#271

Post by esseph »

raw_anon_1111 wrote: post #269
esseph wrote: post #256

Earlier quoted context omitted.

> The world is not moving back to on prem. Lol, you should tell my customers (that are moving back on prem) that! You should also tell Microsoft, who just yesterday said they are going back to focusing on local apps.

Your customers are an anecdote, now compare that to the publicly reported numbers from AWS, GCP and Azure where they all say the only thing keeping them from growing more is the chip shortage.

Oh I'm sure they'll continue to have some cloud services, no doubt. But look at VMware for example, even after the insane price increases. Nutanix also seems to be doing quite well. I'm seeing a fair amount of on-prem bare metal k8s too.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#272

Post by amelius »

barelysapient wrote: post #241
raw_anon_1111 wrote: post #234

Earlier quoted context omitted.

Users don’t care about “privacy”. If they did, Meta and Alphabet wouldn’t be worth $1T+. Users really don’t matter at all. The revenue for AI companies will be B2B where the user is not the customer - including coding agents. Most people don’t even use computers as their primary “computing device” and most people are buying crappy low end Android phones - no I’m not saying all Android phones are crappy. But that’s wh…

Different users. Many people care about privacy and aren’t using Meta products. And many businesses care about it too and have information policies to protect their IP.

> Different users. Many people care about privacy and aren’t using Meta products.

Yeah but if they can rake in 100x as much by making products for people who don't care about privacy, then why spend time developing stuff for people who care?

There is still a small market left, of course, but that market will not have the billions of R&D behind it.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#273

Post by _doctor_love »

franze wrote: post #86

I created "apfel" https://github.com/Arthur-Ficial/apfel a CLI for the apple on-device local foundation model (Apple intelligence) yeah its super limited with its 4k context window and super common false positives guardrails (just ask it to describe a color) ... bit still ... using it in bash scripts that just work without calling home / out or incurring extra costs feels super powerful.

Dieser apfel ist sehr lecker!

Re: Ollama is now powered by MLX on Apple Silicon in preview

#275

Post by seanmcdirmid »

Barbing wrote: post #66
seanmcdirmid wrote: post #41

Earlier quoted context omitted.

About 2.5 decades from the start of the JVs, but they did it. Semiconductors and jet turbines are really the last two tech trees that China has yet to master.

Which might they master first?

Both are hard nuts but China is throwing massive amounts of money at the problem. They can already get performance or economy from each, they just need to figure out how to get both at the same time.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#276

Post by raw_anon_1111 »

esseph wrote: post #271
raw_anon_1111 wrote: post #269

Earlier quoted context omitted.

Your customers are an anecdote, now compare that to the publicly reported numbers from AWS, GCP and Azure where they all say the only thing keeping them from growing more is the chip shortage.

Oh I'm sure they'll continue to have some cloud services, no doubt. But look at VMware for example, even after the insane price increases. Nutanix also seems to be doing quite well. I'm seeing a fair amount of on-prem bare metal k8s too.

Again - anecdotes is not data. We have data. That would be about as silly as me citing my own experience as proof that “everyone is moving to AWS” when I work for a company that is exclusively an AWS partner consulting company.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#278

Post by jesse23 »

abu_ameena wrote: post #223

On-device models are the future. Users prefer them. No privacy issues. No dealing with connectivity, tokens, or changes to vendors implementations. I have an app using Foundation Model, and it works great. I only wish I could backport it to pre macOS 26 versions.

Yes so far do we have a working practice that, with a given local mode, any infra we could use, that provide a good practice that can leverage it for local task?

Re: Ollama is now powered by MLX on Apple Silicon in preview

#279

Post by xiconfjs »

brcmthrowaway wrote: post #267
xiconfjs wrote: post #227

Earlier quoted context omitted.

* macOS 26.x on MacBookPro M1 Max 32GB * Ollama on macOS, cursor to play around * Open WebUI [1] on my Homeserver via API to Ollama (also for remote „A.I.“ access) * running gpt-oss:20b, qwen3.5:9b with ease, qwen3.5:27b for more complex tasks [1] https://github.com/open-webui/open-webui

Seems complicated. Switch to LMStudio

I tried man times but at least with its API active, LMStudio has some kind of memory leaks which will slow down the whole system (after ~1-2 days of uptime) even after unloading the model and stopping LMStudio up to a point where even playing a 1080p video results in frame drops. No such issues with Ollama.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#280

Post by svachalek »

testing22321 wrote: post #238
abu_ameena wrote: post #223

On-device models are the future. Users prefer them. No privacy issues. No dealing with connectivity, tokens, or changes to vendors implementations. I have an app using Foundation Model, and it works great. I only wish I could backport it to pre macOS 26 versions.

I see all these LLM posts about if a certain model can run locally on certain hardware and I don’t get it. What are you doing with these local models that run at x tokens/sec. Do you have the equivalent of ChatGPT running entirely locally? What do you do with it? Why? I honestly don’t understand the point or use case.

1. There are small local models that have the capabilities of frontier models a year ago

2. They aren't harvesting your data for government files or training purposes

3. They won't be altered overnight to push advertising or a political agenda

4. They won't have their pricing raised at will

5. They won't disappear as soon as their host wants you to switch

Post reply on HN