Ask HN: MacBook vs. Dedicated GPU for LLM

If you don’t already have a MacBook, then there’s a bit of a sweet-spot for the AI experimenter right now, which is to buy a second-hand 16” MBP with an M1 Max chip and 64GB of shared ram. Because these are about 5 years old now, they have depreciated to the point where they can be had for around £1100 / €1300 / $1500 and make a phenomenal platform for learning because the 64Gb of shared memory means you can host models up to about 48GB in size, and then task them to do interesting things with coding without ever having to worry about token burn.

The downside is that they’re slow, and prone to having to be nudged to keep them on track, but that’s part of the fun too. The “latency” is atrocious granted - you ask something and the machine thinks for a few minutes before saying anything which is a different experience to using Claude. But… it does work. You can think of yourself more like a manager with a junior member of staff and set the machine running and leave it to do its thing for a couple of hours which can be actually useful work, but this approach will likely be shouted down by some commenters here who treat Claude like some kind of expensive and quick-fire dopamine pump. Can also use a Mac like this for running diffusion models for image generation and suchlike in ComfyUI, even though, again, results will be slow. Spending more money on a more recent MBP with as much RAM as you can afford will deliver the same results more expensively in a quicker and quicker time.

To get the same kind of size of model you’d have to combine a couple of Nvidia 3090 24GB cards in a decent workstation with the PCI capacity to handle them, or hack some kind of solution to hang GPUs off the back of a motherboard on ribbon cables with the GPUs running on their own PSU, which is what I’m building next… the difference is those cards have 24GB of vram and cost about $1000 each second-hand, but will operate much much faster than the M1 Max MBP, or even the most recent M5 because they have so much more bandwidth (because they’re burning 350 watts on GPU compute rather than 140 watts total which is what a super efficient MBP has for the cpu/gpu/screen/everything).

So say you had $6000 to spend today, you could buy a second hand workstation and craft a solution with external GPUs which would completely smoke any Mac in existence, even though macs have the edge in the size of model you’d can run (slowly) due to their shared memory. External GPUs and access to the Nvidia frameworks and general CUDA ecosystem wins out on the performance front though. A real sweet spot is to buy an M1 Max MBP and have that as your front end to a Linux workstation full of GPUs.

But any apple silicon MBP is a totally competent gateway drug to local agentic computing.

Google Gemini could give you an in-depth and useful discussion about this exact question.