[dead]
Ask HN: MacBook vs. Dedicated GPU for LLM
41–50 of 76 posts
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#42MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly. Dedicated GPUs have less video RAM so can run smaller less smart models quickly.
> MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly.
With the model using MLX the speed increase is night and day. Even non-MLX is good.
You also don't have the transfer costs related to moving CPU data into the GPU.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#43MacBooks have lots of RAM and no PCIe bottleneck, but ~10x fewer FLOP/s than a much cheaper Nvidia GPU. Test LLMs on rented GPUs on vast.ai or other similar services (beware storage etc). Don't spend thousands before trying and knowing exactly what you get.
Also beware local models tend to be slow. Also, the main optimization trick for LLM inference is running large batches (concurrent users) and you won't take advantage of this (batch=1).
IMHO using Macs for LLMs is a fad. An expensive fad.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#44Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#45If you want a massive MacBook anyway then it's great. They are decent for local LLMs, awesome for local image models and it's a MacBook so AppleCare+ has your back. IMO it's a no brainer if you wanted a MacBook anyway but it's a poor choice if your reason to buy it is to run LLMs.
I agree. To run an acceptable model (e.g. Qwen/Qwen3.6-27B or google/gemma-4-31B) with a good quantization (minimum Q5) with a good context size (min 64k) you could buy 2 or even 3 GTX 5060 16GiB VRAM for ~550$ each. Fyi the much faster MoE models were useless for my usecases - e.g not able to correctly identify me/I/you, endless thinking loops, etc. I'm currently running those models using an RTX 5070 12GiB + RTX 50…
How are you running these together, splitting the model somehow or did you mean different models on any one card at a time?
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#46Earlier quoted context omitted.
I agree. To run an acceptable model (e.g. Qwen/Qwen3.6-27B or google/gemma-4-31B) with a good quantization (minimum Q5) with a good context size (min 64k) you could buy 2 or even 3 GTX 5060 16GiB VRAM for ~550$ each. Fyi the much faster MoE models were useless for my usecases - e.g not able to correctly identify me/I/you, endless thinking loops, etc. I'm currently running those models using an RTX 5070 12GiB + RTX 50…
how many tokens per second do you get?
I bought two RTX3080s with 20GB during my holiday in china (set me back 700euros) I'm getting 800-1000 input tps and 60-100tps output with Qwen 3.6 27b Q8 (MTP, P2P, 200k context) this feels like opus4.5 level while coding (pi harness).
Also easy to just host your own openai compatible api from home this way and still use your MacBook as dev station.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#47With a dedicated GPU, the lag is in transferring data to the GPU. You don't have that lag in ARM.
But it really depends on what it is you want to do. An MLX optimised recent model will run fine and at decent speeds. Granite4.1 (a few months old) for example takes up 2GB of memory, insanely fast and results are good vs much bigger models like gpt-oss-120b (a year old). It even runs on an M1 mac with good speeds.
The models are only getting better.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#48With a M5 16c 48GB and Qwen 3.6 35B Q4 I get up to 1900 PP/s and 80 TG/s.
With an Nvidia 5090 I get 7800 PP/s and 280 TG/s.
Together with pi mono I wouldn't want to go back to Claude & Co. Speed, quality of the answers, short answer times at any time of day - once you have eaten from the fruit your definition of SOTA will change...
For reference, I do software development since 30 years, I am not vibe coding the umpteenth todo list.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#49macbooks (macs in general) are a good package for llms because they come with so much RAM
and for llms more RAM means access to better models
macbooks might not be as fast as a GPU with similar amount of RAM but more affordable and well integrated
last but not least: compared to a PC+GPU the macbook is either silent (air) or at least way less annoying when you care about noise
for ultimate flexibility and low noise: GPU in the cloud for when you need/want it is probably also most cost effective if you don't have workloads that need to run 24/7
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#50My opinion is that you should wait for 6-12 months before making a purchase either way. Open weight models are getting good. With GLM 5.2 now chasing Opus, I'm very excited to see a smaller model's distillation. Plus, the OLED MacBook Pro should be released by then.
The OLED touchscreen MacBook is rumoured to be called MacBook Ultra now, and it has been delayed quite a few times. I will probably cost the same than a decent bike.