The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.
The M-series chips really changed the game here
Local AI is driving the biggest change in laptops in decades
181–190 of 273 posts
Re: Local AI is driving the biggest change in laptops in decades
#182The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.
This article is to sell more laptops.
Re: Local AI is driving the biggest change in laptops in decades
#183Earlier quoted context omitted.
But economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI. However, I agree with the article that people will run big LLMs on their laptop N years down the line. Especially if hardware outgrows best-in-class LLM model requirements. If a phone could run a 512GB LLM model fast, you would want it.
Are you sure the subscription will still be affordable after the venture capital flood ends and the dumping stops?
100% yes.
The amount of compute in the world is doubling over 2 years because of the ongoing investment in AI (!!)
In some scenario where new investment stops flowing and some AI companies go bankrupt all that compute will be looking for a market.
Inference providers are already profitable so with cheaper hardware it will mean even cheaper AI systems.
Re: Local AI is driving the biggest change in laptops in decades
#184I’ve been running LLMs on my laptop (M3 Max 64GB) for a year now and I think they are ready, especially with how good mid sized models are getting. I’m pretty sure unified memory and energy efficient GPUs will be more than just a thing on Apple laptops in the next few years.
You doing code completion and agentic stuff successfully with local models? Got any tips? I've been out of the game for [checks watch] a few months and am behind on the latest. Is Cline the move?
I haven't bothered doing code completion locally yet, though its something I want to try with the QWEN model. I'm mostly using it to generate/fix code CLI style.
Re: Local AI is driving the biggest change in laptops in decades
#185Earlier quoted context omitted.
> economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI Uber is economical, too; but folks prefer to own cars, sometimes multiple. And how there's market for all kinds of vanity cars, fast sportscars, expensive supercars... I imagine PCs & Laptops will have such a market, too: In probably less than a decade, may be a £20k laptop running a 671b+ LLM locally will be…
Paying $30-$70/day to commute is economical?
if you calculate depreciation and running costs on a new car in most places - I think it probably would be.
Re: Local AI is driving the biggest change in laptops in decades
#186I’ve been running LLMs on my laptop (M3 Max 64GB) for a year now and I think they are ready, especially with how good mid sized models are getting. I’m pretty sure unified memory and energy efficient GPUs will be more than just a thing on Apple laptops in the next few years.
Only because of Apples unified memory architecture. The groundwork is there, we just need memory to be cheaper so we can fit 512+GB now ;)
There's not in the end all that much point having more memory than you can compute on in a reasonable time. So I think probably the useful amount tops out in the 128GB range where you can still run a 70b model and get a useful token rate out of it.
Re: Local AI is driving the biggest change in laptops in decades
#187Earlier quoted context omitted.
> economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI Uber is economical, too; but folks prefer to own cars, sometimes multiple. And how there's market for all kinds of vanity cars, fast sportscars, expensive supercars... I imagine PCs & Laptops will have such a market, too: In probably less than a decade, may be a £20k laptop running a 671b+ LLM locally will be…
> Uber is economical, too One time I took an Uber to work because my car broke down and was in the shop and the Uber driver (somewhat pointedly) made a comment that I must be really rich to commute to work via Uber because Ubers are so expensive
Most people don't realise the amount of money they spend per year on cars.
Re: Local AI is driving the biggest change in laptops in decades
#188Earlier quoted context omitted.
You still need ridiculously high spec hardware, and at Apple’s prices, that isn’t cheap. Even if you can afford it (most won't), the local models you can run are still limited and they still underperform. It’s much cheaper to pay for a cloud solution and get significantly better result. In my opinion, the article is right. We need a better way to run LLMs locally.
You still need ridiculously high spec hardware, and at Apple’s prices, that isn’t cheap. You can easily run models like Mistral and Stable Diffusion in Ollama and Draw Things, and you can run newer models like Devstral (the MLX version) and Z Image Turbo with a little effort using LM Studio and Comfyui. It isn't as fast as using a good nVidia GPU or a cloud GPU but it's certainly good enough to play around with and l…
You can run newer models like Z Image Turbo or FLUX.2 [dev] using Draw Things with no effort too.
Re: Local AI is driving the biggest change in laptops in decades
#189Earlier quoted context omitted.
Apple has a 10-18% market share for laptops. That's significant but it certainly isn't "most". Most laptops can run at best a 7-14b model, even if you buy one with a high spec graphics chip. These are not useful models unless you're writing spam. Most desktops have a decent amount of system memory but that can't be used for running LLMs at a useful speed, especially since the stuff you could run in 32-64GB RAM would…
my laptop is 4 years old. I only have 6Gb VRam. I run, mostly, 4b and 8b models. They are extremely useful in a variety of situations. Just because you can't replicate what you do in chatgpt doesn't mean they don't have their use cases. It seems to me you know very little about what these models can do. Not to speak of trained models for specific use cases, or even smaller models like functiongemma or TTS/ASR models.…
A few examples of useful tasks would be appreciated. I do suffer from a sad lack of imagination.
Re: Local AI is driving the biggest change in laptops in decades
#190I spent a good 30 seconds trying to figure out what DDS was an acronym for in this context.
Care to share the answer?