I predict we will see compute-in-flash before we see cheap laptops with 128+ gigs of ram.
I can't tell if this is optimism for compute-in-flash or pessimism with how RAM has been going lately!
Local AI is driving the biggest change in laptops in decades
41–50 of 273 posts
Re: Local AI is driving the biggest change in laptops in decades
#42I’ve been running LLMs on my laptop (M3 Max 64GB) for a year now and I think they are ready, especially with how good mid sized models are getting. I’m pretty sure unified memory and energy efficient GPUs will be more than just a thing on Apple laptops in the next few years.
Only because of Apples unified memory architecture. The groundwork is there, we just need memory to be cheaper so we can fit 512+GB now ;)
Memory prices will rise short term and generally fall long term, even with the current supply hiccup the answer is to just build out more capacity (which will happen if there is healthy competition). I meant, I expect the other mobile chip providers to adopt unified architecture and beefy GPU cores on chip and lots of bandwidth to connect it to memory (at the max or ultra level, at least), I think AMD is already doing UM at least?
Re: Local AI is driving the biggest change in laptops in decades
#43Earlier quoted context omitted.
Laptop manufacturers are too desperate to cash on the AI craze. There's nothing special about an 'AI PC'. It's just a regular PC with Windows Copilot... which is a standard Windows feature anyway. >I don't want this garbage on my laptop, especially when its running of its battery! The one bit of good news is it's not going to impact your battery life because it doesn't do any on-device processing. It's just calling a…
Even collecting and sending all that data to the cloud is going to drain battery life. I'd really rather my devices only do what I ask them to than have AI running the background all the time trying to be helpful or just silently collecting data.
Copilot is just ChatGPT as an app.
If you don't use it, it will have no impact on your device. And it's not sending your data to the cloud except for anything you paste into it.
Re: Local AI is driving the biggest change in laptops in decades
#44Earlier quoted context omitted.
Laptop manufacturers are too desperate to cash on the AI craze. There's nothing special about an 'AI PC'. It's just a regular PC with Windows Copilot... which is a standard Windows feature anyway. >I don't want this garbage on my laptop, especially when its running of its battery! The one bit of good news is it's not going to impact your battery life because it doesn't do any on-device processing. It's just calling a…
Doesn't this lead to a lot of tension between the hardware makers and Microsoft? MS wants everyone to run Copilot on their shiny new data centre, so they can collect the data on the way. Laptop manufacturers are making laptops that can run an LLM locally, but there's no point in that unless there's a local LLM to run (and Windows won't have that because Copilot). Are they going to be pre-installing Llama on new lapto…
It isn't just copilot that these laptops come with; manufacturers are already putting their own AI chat apps as well.
For example, the LG gram I recently got came with just such an app named Chat, though the "ai button" on the keyboard (really just right alt or control, I forget which) defaults to copilot.
If there's any tension at all, it's just who gets to be the default app for the "ai button" on the keyboard that I assume almost nobody actually uses.
Re: Local AI is driving the biggest change in laptops in decades
#45Seems like wishful thinking. > How many TOPS do you need to run state-of-the-art models with hundreds of millions of parameters? No one knows exactly. Why not extrapolate from open-source AIs which are available? The most powerful open-source AI (which I know of) is Kimi K2 and >600gb. Running this at acceptable speed requires 600+gb GPU/NPU memory. Even $2000-3000 AI-focused PCs like the DGX spark or Strix Halo typi…
You may be correct, but I wonder if we'll see Mac Mini sized external AI boxes that do have the 1TB of RAM and other hardware for running local models. Maybe 100% of computer users wouldn't have one, but maybe 10-20% of power users would, including programmers who want to keep their personal code out of the training set, and so on. I would not be surprised though if some consumer application made it desirable for eac…
> but I wonder if we'll see Mac Mini sized external AI boxes that do have the 1TB of RAM
Isn't that the Mac Studio already? Ok, it seems to max at 512 GB.
Re: Local AI is driving the biggest change in laptops in decades
#46This must be referring mostly to windows, or non-Apple laptops
Re: Local AI is driving the biggest change in laptops in decades
#47I'm running GPT-OSS 120B on a MacBook Pro M3 Max w/128 GB. It is pretty good, not great, but better than nothing when the wifi on the plane basically doesn't work.
Re: Local AI is driving the biggest change in laptops in decades
#48Earlier quoted context omitted.
Doesn't this lead to a lot of tension between the hardware makers and Microsoft? MS wants everyone to run Copilot on their shiny new data centre, so they can collect the data on the way. Laptop manufacturers are making laptops that can run an LLM locally, but there's no point in that unless there's a local LLM to run (and Windows won't have that because Copilot). Are they going to be pre-installing Llama on new lapto…
It isn't just copilot that these laptops come with; manufacturers are already putting their own AI chat apps as well. For example, the LG gram I recently got came with just such an app named Chat, though the "ai button" on the keyboard (really just right alt or control, I forget which) defaults to copilot. If there's any tension at all, it's just who gets to be the default app for the "ai button" on the keyboard that…
Interesting. Yeah, that'll be the argument
Re: Local AI is driving the biggest change in laptops in decades
#49I predict we will see compute-in-flash before we see cheap laptops with 128+ gigs of ram.
You could get 128gb ram laptops from the time ddr4 came around: workstation class laptops with 4 ram slots would happily take 128gb of memory.
The fact that nowadays there are little to no laptops with 4 ran slots is entirely artificial.
Re: Local AI is driving the biggest change in laptops in decades
#50Earlier quoted context omitted.
Yeah especially since what is happening in the memory market
Feast and famine. In three years we will be swimming in more ram than we know what to do with.
That might not be the case. The kind of memory that will flood the second-hand market could not be the kind of memory we can stuff in laptops or even desktop systems.