Its interesting to notice that here https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com... we default to measuring LLM coding performance as how long[~5h] a human task a model can complete with 50% success-rate (with 80% fall back for the second chart [~.5h]), while here it seems that for actual coding we really care about the last 90-100% of the costly model's performance.
A guide to local coding models
151–160 of 363 posts
Re: A guide to local coding models
#152Buying a maxed out MacBook Pro seems like the most expensive way to go about getting the necessary compute. Apple is notorious for overcharging for hardware, especially on ram.
I bet you could build a stationary tower for half the price with comparable hardware specs. And unless I'm missing something you should be able to run these things on Linux.
Getting a maxed out non-apple laptop will also be cheaper for comparable hardware, if portability is important to you.
Re: A guide to local coding models
#153I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Somewhat comically, the author seems to have made it about 2 days. Out of 1,825. I think the real story is the folly of fixating your eyes on shiny new hardware and searching for justifications. I'm too ashamed to admit how many times I've done that dance... Local models are purely fo…
> I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Well, the hardware remains the same but local models get better and more efficient, so I don't think there is much difference between paying 5k for online models over 5 years vs getting a laptop (and well, you'll need a laptop anyway, so why not just get a good enough one to run loc…
Even if intelligence scaling stays equal, you'll lose out on speed. A sota model pumping 200 tk/s is going to be impossible to ignore with a 4 year old laptop choking itself at 3 tk/s.
Even still, right now is when the first gen of pure LLM focused design chipsets are getting into data centers.
Re: A guide to local coding models
#154Earlier quoted context omitted.
LMStudio is not open source though, ollama is but people should use llama.cpp instead
I suspect Ollama is at least partly moving away open source as they look to raise capitol, when they released their replacement desktop app they did so as closed source. You're absolutely right that people should be using llama.cpp - not only is it truly open source but it's significantly faster, has better model support, many more features, better maintained and the development community is far more active.
Desktop app is open-source now.
Re: A guide to local coding models
#155Earlier quoted context omitted.
> I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Well, the hardware remains the same but local models get better and more efficient, so I don't think there is much difference between paying 5k for online models over 5 years vs getting a laptop (and well, you'll need a laptop anyway, so why not just get a good enough one to run loc…
If you have inference running on this new 128GB RAM Mac, wouldn't you still need another separate machine to do the manual work (like running IDE, browsers, toolchains, builders/bundlers etc.)? I can not imagine you will have any meaningful RAM available after LLM models are running.
No? First of all you can limit how much of the unified RAM goes into VRAM, and second, many applications don't need that much RAM. Even if you put 108 GB to VRAM and 16 to applications, you'll be fine.
Re: A guide to local coding models
#156Earlier quoted context omitted.
I think you should mention that LM Studio isn't open source. I mean, what's the point of using local models if you can't trust the app itself?
> I mean, what's the point of using local models if you can't trust the app itself? and you think ollama doesn't do telemetry/etc. just because it's open source?
You're welcome to go through the source: https://github.com/ollama/ollama/
Re: A guide to local coding models
#157Buying a maxed out MacBook Pro seems like the most expensive way to go about getting the necessary compute. Apple is notorious for overcharging for hardware, especially on ram. I bet you could build a stationary tower for half the price with comparable hardware specs. And unless I'm missing something you should be able to run these things on Linux. Getting a maxed out non-apple laptop will also be cheaper for compara…
Can any x86 based system actually comes with that much unified memory?
Not an Apple fanboy, but I was under the impression that having access to up to 512GB usable GPU memory was the main feature in favour of the mac.
And now with Exo, you can even break the 512GB barrier.
Re: A guide to local coding models
#158Earlier quoted context omitted.
> I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Well, the hardware remains the same but local models get better and more efficient, so I don't think there is much difference between paying 5k for online models over 5 years vs getting a laptop (and well, you'll need a laptop anyway, so why not just get a good enough one to run loc…
Even if intelligence scaling stays equal, you'll lose out on speed. A sota model pumping 200 tk/s is going to be impossible to ignore with a 4 year old laptop choking itself at 3 tk/s. Even still, right now is when the first gen of pure LLM focused design chipsets are getting into data centers.
At a certain point, tokens per second stop mattering because the time to review stays constant. Whether it shits out 200 tokens a second versus 20, it doesn't much matter if you need to review the code that does come out.
Re: A guide to local coding models
#159Earlier quoted context omitted.
> I recently bought the $200/mo OpenAI subscription but could barely use 10% of it This entire comment is confusing. Why are you buying the $200/month plan if you’re only using 10% of it? I rotate providers. My comment above applies to all of them. It really depends on the work you’re doing and the codebase. There are tasks where I can get decent results and barely make the usage bar move. There are other tasks where…
Not the same poster, but apparently they tried the $200/mo subscription, but after seeing they don't need it, they "subscribed to all three at their lowest subscriptions (for $60/mo)" instead.
> but apparently they tried the $200/mo subscription, but after seeing they don't need it
This is why it’s confusing, though. Why start with the highest plan as the starting point when it’s so easy to upgrade?
Re: A guide to local coding models
#160Earlier quoted context omitted.
I'm talking about the general trend, not the exceptions. How much of the code do you manually write with the 100 dollar subscription? Vibe coding is a descriptive, not a prescriptive, label.
"How much of the code do you manually write" I review all of it, but hand write little of it. It's bizarre how I've ended up here, but yep. That said, I wouldn't / don't trust it with something from scratch, I only trust it to do that because I built -- by hand -- a decent foundation for it to start from.
Sure, you're like me, you're not a vibe coder by the actual definition then. Still, the general trend I see is that a lot of actual vibe coders do try to get their product working, code quality be damned. Personally, same as you, I stopped vibe coding and actually started writing a lot of architecture and code myself first then allowing the LLM to fill in the features so to speak.