I spent a good 30 seconds trying to figure out what DDS was an acronym for in this context.
Care to share the answer?
Turns out the first word of the article was odds.
Local AI is driving the biggest change in laptops in decades
261–270 of 273 posts
Re: Local AI is driving the biggest change in laptops in decades
#262I have no desire to run an LLM on my laptop when I can run one on a computer the size of six football fields.
I've been playing around with my own home-built AI server for a couple months now. It is so much better than using a cloud provider. It is the difference between drag racing in your own car, and renting one from a dealership. You are going to learn far more doing things yourself. Your tools will be much more consistent and you will walk away with a far greater understanding of every process. A basic last-generation P…
Unless you normally use electric resistance heating (or some kind of fossil fuel with higher gCO2/kWh) then you don't get necessarily a free pass on the global warming thing!
Our whole home is heated with <500W on average: at this moment the heat pump is drawing 501W (H4 boundary) at close to freezing outside, and its demand is intermittent.
Re: Local AI is driving the biggest change in laptops in decades
#263Earlier quoted context omitted.
You can get any low spec laptop that has no soldered DIMMs and just replace them with the maximum supported capacity. You don't necessarily need to go the maxed up SKU.
Would that be unified memory? Where the gpu and cpu can share the memory? Which is key for performance.
No, it wouldn’t. You’d be limited to using the CPU and the lower bandwidth system memory.
Re: Local AI is driving the biggest change in laptops in decades
#264Earlier quoted context omitted.
Think of it like loading a moving truck where: - The house is the disk - You are the RAM - The truck is the VRAM There won't be a single time you can observe yourself carrying the weight of everything being moved out of the house because that's not what's happening. Instead you can observe yourself taking many tiny loads until everything is finally moved, at which point you yourself should not be loaded as a result o…
My confusion was on the shuffling process happening per token. If this was happening per token, it would be effectively the same as loading the model from disk every token.
[deleted]
Re: Local AI is driving the biggest change in laptops in decades
#265Earlier quoted context omitted.
You can get any low spec laptop that has no soldered DIMMs and just replace them with the maximum supported capacity. You don't necessarily need to go the maxed up SKU.
Would that be unified memory? Where the gpu and cpu can share the memory? Which is key for performance.
Right, no, it wouldn't, I appreciate that in this particular context my comment was entirely wrong.
Thanks for helping me see it!
Re: Local AI is driving the biggest change in laptops in decades
#266if you focus out of local LLMs (also served using dedicated apps), the title holds a lot of promise. case in point: WASM and WebGPU
the edge/on-device AI use cases on smartphones can also extend without user friction through web apps built on the above standards. perhaps one day there will be a "WebNPU" or just get supported through existing standards.
there are already some use cases on apps but it usually fallbacks on cpu. perhaps it could be the hw accelerated moment that we saw with video on the web.
Re: Local AI is driving the biggest change in laptops in decades
#267Earlier quoted context omitted.
> The move to SoC that really started with the M1 No it did not. There were numerous SoC that came before it and was inevitable in this space.
Which widely available laptops with comparable but prior to M1 SOCs are you thinking of?
x86 laptop chips have been SoCs for a lot more than 5 years
Re: Local AI is driving the biggest change in laptops in decades
#268Earlier quoted context omitted.
If Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price. And as far as I understand it, a…
it's astonishing how apple gouges on the memory and ssd upgrade prices (I'm on an M1 w/ 64Gb/4Tb). That said they have some elasticity when it comes to the DRAM shortage.
The M-series unified memory is built into the chip itself, not separate components. Of course Apple is going to maintain their margins, but it’s easy to see why with this design more memory is more expensive than drams. Well maybe not with the current market pricing which hopefully is temporary.
Re: Local AI is driving the biggest change in laptops in decades
#269I'm running GPT-OSS 120B on a MacBook Pro M3 Max w/128 GB. It is pretty good, not great, but better than nothing when the wifi on the plane basically doesn't work.
I'm running it on PC laptop with mobile 5090 and 64GB of ram. Start is a bit rough, but once it gets going it is perfectly servicable when I'm on a bad connection.
Re: Local AI is driving the biggest change in laptops in decades
#270Kind of ironic that it is a factor that 95% of regular users don’t care about or actively avoid.