Earlier quoted context omitted.
749 for an M4 air at Amazon right now
Try running anything interesting on these 8gb of ram. You need 96gb or 128gb to do non trivial things. That is not yet 749 usd
64gb is fine.
Local AI is driving the biggest change in laptops in decades
161–170 of 273 posts
Re: Local AI is driving the biggest change in laptops in decades
#162Earlier quoted context omitted.
Then idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory. > However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero. This stra…
So I'm hearing a lot of people running LLMs on Apple hardware. But is there actually anything useful you can run? Does it run at a usable speed? And is it worth the cost? Because the last time I checked the answer to all three questions appeared to be no. Though maybe it depends on what you're doing? (Although if you're doing something simple like embeddings, then you don't need the Apple hardware in the first place.…
I can definitely write code with a local model like Devstral small or a quantized granite, or a quantized deep-seek on an M1 Max w/ 64gb of ram.
Re: Local AI is driving the biggest change in laptops in decades
#163Earlier quoted context omitted.
> economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI Uber is economical, too; but folks prefer to own cars, sometimes multiple. And how there's market for all kinds of vanity cars, fast sportscars, expensive supercars... I imagine PCs & Laptops will have such a market, too: In probably less than a decade, may be a £20k laptop running a 671b+ LLM locally will be…
Paying $30-$70/day to commute is economical?
> Paying $30-$70/day to commute is economical?
When LLM use approaches this number, running one locally would be, yes. What you and other commentator seem to miss is, "Uber" is a stand-in for Cloud-based LLMs: Someone else builds and owns those servers, runs the LLMs, pays the electricity bills... while its users find it "economical" to rent it.
(btw, taxis are considered economical in parts of the world where owning cars is a luxury)
Re: Local AI is driving the biggest change in laptops in decades
#164Earlier quoted context omitted.
Then idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory. > However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero. This stra…
So I'm hearing a lot of people running LLMs on Apple hardware. But is there actually anything useful you can run? Does it run at a usable speed? And is it worth the cost? Because the last time I checked the answer to all three questions appeared to be no. Though maybe it depends on what you're doing? (Although if you're doing something simple like embeddings, then you don't need the Apple hardware in the first place.…
This paper shows a use case running on Apple silicon that’s theoretically valuable:
https://pmc.ncbi.nlm.nih.gov/articles/PMC12067846/
Who cares if result is right / wrong etc as it will all be different in a year … just interesting to see a test of desktop class hardware go ok.
Re: Local AI is driving the biggest change in laptops in decades
#165Earlier quoted context omitted.
Then idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory. > However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero. This stra…
Apple has a 10-18% market share for laptops. That's significant but it certainly isn't "most". Most laptops can run at best a 7-14b model, even if you buy one with a high spec graphics chip. These are not useful models unless you're writing spam. Most desktops have a decent amount of system memory but that can't be used for running LLMs at a useful speed, especially since the stuff you could run in 32-64GB RAM would…
my laptop is 4 years old. I only have 6Gb VRam. I run, mostly, 4b and 8b models. They are extremely useful in a variety of situations. Just because you can't replicate what you do in chatgpt doesn't mean they don't have their use cases. It seems to me you know very little about what these models can do. Not to speak of trained models for specific use cases, or even smaller models like functiongemma or TTS/ASR models. (btw, I've trained models using my 6Gb VRAM too)
Re: Local AI is driving the biggest change in laptops in decades
#166Earlier quoted context omitted.
A lazy easy cheap shot. But do you deny these aspects from the article are not coming? Or won't be still here in 5 years? - Addition of more—and faster—memory. - Consolidation of memory. - Combination of chips on the same silicon. All of these are also happening for non AI reasons. The move to SoC that really started with the M1 wasn't because of AI, but unified memory being the default is something we will see in 5…
We just had a series of articles and sysadmin outcry that major vendors were bringing 8gb laptops back to standard models because of the ram prices. In the short term, we're seeing a reduction.
In terms of demand, anecdotally-speaking I can certainly see this influencing some decisions when other circumstances permit. Many people I know are both excited for new and better games, and equally exited about running LLM/SD/etc models locally with Comfy, LM studio and the like
Re: Local AI is driving the biggest change in laptops in decades
#167The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.
You still need ridiculously high spec hardware, and at Apple’s prices, that isn’t cheap. Even if you can afford it (most won't), the local models you can run are still limited and they still underperform. It’s much cheaper to pay for a cloud solution and get significantly better result. In my opinion, the article is right. We need a better way to run LLMs locally.
I was pleasantly surprised at the speed and power of my second hand M1 Pro 32GB running Asahi & Qwen3:32B. It does all I need, and I dont mind the reading pace output, although I'd be tempted by M2 Ultra if the secondhand market hadn't also exploded with the recent RAM market manipulations.
Anyway, I'm on a mission to have no subscriptions in the New Year. Plus it feels wrong to be contributing towards my own irrelevance (GAI).
Re: Local AI is driving the biggest change in laptops in decades
#168Earlier quoted context omitted.
Try running anything interesting on these 8gb of ram. You need 96gb or 128gb to do non trivial things. That is not yet 749 usd
64gb is fine.
This subthread is about the Macbook Air, which tops out at 32 GB, and can't be upgraded further.
While browsing the Apple website, it looks like the cheapest Macbook with 64 GB of RAM is the Macbook Pro M4 Max with 40-core GPU, which starts at $3,899, a.k.a. more than five times more expensive than the price quoted above.
Re: Local AI is driving the biggest change in laptops in decades
#169The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.
Only if you want to take all the proprietary baggage and telemetry that comes with Apple platforms by default. A Lenovo T15g with a 16gb 3080 mobile doesn’t do too badly and will run more than just Windows.
I just got a Framework desktop with 128 GB of shared RAM just before the memory prices rocketed, and I can comfortably run many even bigger oss models locally. You can dedicate 112GB to the GPU and it runs Linux perfectly.
Re: Local AI is driving the biggest change in laptops in decades
#170Earlier quoted context omitted.
We’ve been through multiple cycles of scarcity/surplus DRAM cycles in the last couple of decades. Why do we think it will be different now?
> Why do we think it will be different now? Margins. AI usage can pay a lot more. Even if they sell less than can still be more profitable. In the past there wasn’t a high margin usage. Servers didn’t charge such a high premium.
Do you not think that some DRAM producer isn't going to see the high margins as a signal to create more capacity to get ahead of the other DRAM producers? This is how it always has worked before, but somehow it is different this time?