Ollama is now powered by MLX on Apple Silicon in preview

Post by roboror » Tue, Mar 31, 2026, 6:13 PM UTC

dec0dedab0de wrote: post #257

testing22321 wrote: post #238

Earlier quoted context omitted.

I see all these LLM posts about if a certain model can run locally on certain hardware and I don’t get it. What are you doing with these local models that run at x tokens/sec. Do you have the equivalent of ChatGPT running entirely locally? What do you do with it? Why? I honestly don’t understand the point or use case.

most of the llm tooling can handle different models. Ollama makes it easy to install and run different models locally. So you can configure aider or vscode or whatever you're using to connect to chatgpt to point to your local models instead. None of them are as good as the big hosted models, but you might be surprised at how capable they are. I like running things locally when I can, and I also like not worrying abou…

Post by whazor » Tue, Mar 31, 2026, 6:20 PM UTC

abu_ameena wrote: post #223

On-device models are the future. Users prefer them. No privacy issues. No dealing with connectivity, tokens, or changes to vendors implementations. I have an app using Foundation Model, and it works great. I only wish I could backport it to pre macOS 26 versions.

Post by victorbjorklund » Tue, Mar 31, 2026, 6:23 PM UTC

innagadadavida wrote: post #262

raw_anon_1111 wrote: post #246

Earlier quoted context omitted.

70% of the world’s population use at least one Meta property at least once per day. How many of the other 30% are too poor/young/computer illiterate to be part of an addressable market? Every company has dozens of SaaS products that store their business critical information. Amazon installs Office on each computer, Slack (they were moving away from Chime when I left), and the sales department uses SalesForce - SA’s a…

These are all great statistics, but how do you explain ClawdBot explosion. Even in lower income countries like China. So much demand that Apple can’t keep up production of Mac Minis. Why aren’t these folks going towards cloud solutions? Is it cost or is there some consideration for having more control over their data?

Post by aurareturn » Tue, Mar 31, 2026, 6:38 PM UTC

RALaBarge wrote: post #201

aurareturn wrote: post #62

Earlier quoted context omitted.

I highly doubt they can catch up in 3-5 years to Nvidia. Chips take about 3 years to design. Do you think China will have Feymann-level AI systems in 3 years? I think in 3 years, they'll have H200-equivalent at home.

You must have an inside line on information for 'China' -- those are bold predictions!

Post by raw_anon_1111 » Tue, Mar 31, 2026, 6:43 PM UTC

Nevermark wrote: post #307

raw_anon_1111 wrote: post #234

Earlier quoted context omitted.

Users don’t care about “privacy”. If they did, Meta and Alphabet wouldn’t be worth $1T+. Users really don’t matter at all. The revenue for AI companies will be B2B where the user is not the customer - including coding agents. Most people don’t even use computers as their primary “computing device” and most people are buying crappy low end Android phones - no I’m not saying all Android phones are crappy. But that’s wh…

Have you done A/B tests to see if consumers prefer Facebook with or without privacy? No? What? Oh, you can't? Neither can consumers. Most consumers are very aware of the lack of privacy, the manipulation, and have very cynical feelings about Facebook and similar companies. But it's where their friends and family are. For most people the web is a mine field maze where basic things they want are compromised everywhere.…