Local AI needs to be the norm

Post by PeterStuer » Tue, May 12, 2026, 5:30 AM UTC

IMTDb wrote: post #573

pbgcp2026 wrote: post #492

Earlier quoted context omitted.

I'm sorry to spoil it for you, but Perl script was able to do all of that like ... 10 years ago? The out-of-the-box Shotwell manages photos quite well without any intelligence. The problem, as people mentioned above, is SOTA models cognitive and tooling abilities. Also, have you noticed as top-end Mac Studios got downgraded recently? They don't want you to have access to frontier models. And you will not have it. See…

> They don't want you to have access to frontier models. And you will not have it. See Mythos as Exibit A. "They" fully well know that they current frontier model are maybe 6 month ahead of what people will have access to without their control. See Deepseek as Exibit B The reason you can't run these locally are more with the fact that those mythos sized models require extreme amount of memory and processing power to…

Post by otabdeveloper4 » Tue, May 12, 2026, 5:33 AM UTC

CamperBob2 wrote: post #673

otabdeveloper4 wrote: post #387

Earlier quoted context omitted.

> higher param count models will remain smarter for a looong time They're not smarter, they just know more stuff. You probably don't need knowledge about Pokemon or the Diamond Sutra in your enterprise coding LLM. The "smarts" comes from post-training, especially around tool use.

You probably don't need knowledge about Pokemon or the Diamond Sutra in your enterprise coding LLM. That's one of the biggest remaining head-scratchers in this whole business. You do need all that unrelated stuff to make a good coding model. Nobody knows why you can't build a coding model by training on nothing but code, CS texts, specifications, and case studies, but so far it appears that you can't.

Post by henry_kang » Tue, May 12, 2026, 5:57 AM UTC

jimbokun wrote: post #671

henry_kang wrote: post #600

Earlier quoted context omitted.

Actually you can do way more things than that. We have optimized it to process 2TB of high def videos on a M5 MBP in under 24 hours, including everything such as speech understanding, face recog, LLM and VLM. Super fun.

If Steve Jobs was alive Apple would have already demoed this as a new line of Macs with open weight models pre-installed with hooks into all of their existing content creation software. And he would have the audience believing all the demos were running through third party AI providers, until at the last moment explaining “actually all of that ran on device with no connection to any external services.”

Post by albuic » Tue, May 12, 2026, 7:03 AM UTC

DrewADesign wrote: post #667

achenet wrote: post #615

Earlier quoted context omitted.

mainframe industry vs personal computers. If their product is "access to a big model running on a really big computer" (if we can count 'multiple data-centers' as a single enormous distributed computer), then the product "small, accessible device that everyone has" risks killing their cash cow. Ironically enough, the first company to really focus on "an LLM in every phone" will have a good shot at actually being the…

As an aside, the mainframe industry was profitable for decades before PCs took over. It’s not like they spent a zillion dollars ramping up at the same time.

Post by fragmede » Tue, May 12, 2026, 7:55 AM UTC

codebje wrote: post #256

fragmede wrote: post #85

Earlier quoted context omitted.

The business model is the total lack of attention to Qwen and Kimi that would happen if their models weren't downloadable. Before releasing the weights, there was basically zero attention paid in the western hemisphere to them, for whatever reason. By releasing the weights, they're relevant in the western world. The business model is to get people in the West to pay to use their platform hosting their AI, that otherw…

Baidu have a lot of services I've never heard of, that are highly successful in China. The lack of interest in expanding into Western audiences doesn't seem to matter there - what's different about inference?