No post body was provided.
Ask HN: What laptop would you recommend for running LLM models locally?
1–6 of 6 posts
Re: Ask HN: What laptop would you recommend for running LLM models locally?
#2I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.
Re: Ask HN: What laptop would you recommend for running LLM models locally?
#3I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.
[deleted]
Re: Ask HN: What laptop would you recommend for running LLM models locally?
#4I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.
Right now I'm trying Vicuna 13b on a MacbookAir M1, 16GB and it's extremely slow. 7b is ok.
Re: Ask HN: What laptop would you recommend for running LLM models locally?
#5I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.
Right now I'm trying Vicuna 13b on a MacbookAir M1, 16GB and it's extremely slow. 7b is ok.
I have the Air too. It's thermal throttled. Try someone's M1 Pro, or the Mac Mini. Also make sure you're using the GPU (I'm not sure but AFAIK these models can run on CPU too?).
People at my employer use maxxed out Macbook Pro M2 for their ML research needs and say you can't get better performance for that price (in a laptop).
Re: Ask HN: What laptop would you recommend for running LLM models locally?
#6I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.
[deleted]