For those who are using llms on macbook, Want to understand how macbook is different than dedicated GPU in running those models? and how to know how much a macbook is capable of running a model?
Ask HN: MacBook vs. Dedicated GPU for LLM
1–10 of 76 posts
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#2MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly.
Dedicated GPUs have less video RAM so can run smaller less smart models quickly.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#3Both are going to be super super slow and low payback.
You gotta really want it right now.
It's still early!
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#4MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly. Dedicated GPUs have less video RAM so can run smaller less smart models quickly.
Do Mac Pros provide more headroom? noob here, noob questions
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#5If you want a massive MacBook anyway then it's great. They are decent for local LLMs, awesome for local image models and it's a MacBook so AppleCare+ has your back. IMO it's a no brainer if you wanted a MacBook anyway but it's a poor choice if your reason to buy it is to run LLMs.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#6Dual 3090 >>> Any Apple product.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#7My opinion is that you should wait for 6-12 months before making a purchase either way.
Open weight models are getting good. With GLM 5.2 now chasing Opus, I'm very excited to see a smaller model's distillation.
Plus, the OLED MacBook Pro should be released by then.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#8MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly. Dedicated GPUs have less video RAM so can run smaller less smart models quickly.
Do Mac Pros provide more headroom? noob here, noob questions
Idk why you’re being downvoted for asking a question. Pending specs they _could_ provide more headroom for a larger model but they would still be limited by the CPU and it’s associated bus speeds.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#9I asked a few of my friends that are ML engineers this question and all of them said to run the LLMs in the cloud with their infrastructure because it was going to be way faster.
If you just want to tinker around I would look at @JSR_FDD's comment.
Re: Ask HN: MacBook vs. Dedicated GPU for LLM
#10Dual 3090 >>> Any Apple product.
> Dual 3090 >>> Any Apple product.
Dual 3090s are terrible airpods