Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guide to local coding models

aiforswes.com

Post reply on HN
Print view View original

141–150 of 363 posts

Re: A guide to local coding models

#141

Post by selcuka »

Aurornis wrote: post #139
sheepscreek wrote: post #131

Earlier quoted context omitted.

Not for Codex. Not even for Gemini/Antigravity! I am truly shocked by how much mileage I can get out of them. I recently bought the $200/mo OpenAI subscription but could barely use 10% of it. Now for over a month, I use codex for at least 2 hrs every day and have yet to reach the quota. With Gemini/Antigravity, there’s the added benefit of switching to Claude Code Opus 4.5 once you hit your Gemini quota, and Google i…

> I recently bought the $200/mo OpenAI subscription but could barely use 10% of it This entire comment is confusing. Why are you buying the $200/month plan if you’re only using 10% of it? I rotate providers. My comment above applies to all of them. It really depends on the work you’re doing and the codebase. There are tasks where I can get decent results and barely make the usage bar move. There are other tasks where…

Not the same poster, but apparently they tried the $200/mo subscription, but after seeing they don't need it, they "subscribed to all three at their lowest subscriptions (for $60/mo)" instead.

Re: A guide to local coding models

#142

Post by nl »

sheepscreek wrote: post #131
Aurornis wrote: post #123

Earlier quoted context omitted.

The limits for the $20/month plan can be reached in 10-20 minutes when having it explore large codebases with directed. It’s also easy to blow right through the quota if you’re not managing content well (waiting until it fills up and then auto-compacting, or even using /compact frequently instead of /clear or the equivalent in different tools). For most of my work I only need the LLM to perform a structured search of…

Not for Codex. Not even for Gemini/Antigravity! I am truly shocked by how much mileage I can get out of them. I recently bought the $200/mo OpenAI subscription but could barely use 10% of it. Now for over a month, I use codex for at least 2 hrs every day and have yet to reach the quota. With Gemini/Antigravity, there’s the added benefit of switching to Claude Code Opus 4.5 once you hit your Gemini quota, and Google i…

I do the same and agree this works well.

It's worth noting that the Claude subscription seems notably less than the others.

Also there are good free options for code review.

Re: A guide to local coding models

#143

Post by selcuka »

ekianjo wrote: post #126
simonw wrote: post #5

This story talks about MLX and Ollama but doesn't mention LM Studio - https://lmstudio.ai/ LM Studio can run both MLX and GGUF models but does so from an Ollama style (but more full-featured) macOS GUI. They also have a very actively maintained model catalog at https://lmstudio.ai/models

Lmstudio runs llama.cpp under the hood.

They also run the Apple MLX engine on macOS.

Re: A guide to local coding models

#144

Post by jszymborski »

I just got a RTX 5090, so I thought I'd see what all the fuss was about these AI coding tools. I've previously copy pasted back and forth from Claude but never used the instruct models.

So I fired up Cline with gpt-oss-120b, asked it to tell me what a specific function does, and proceeded to watch it run `cat README.md` over and over again.

I'm sure it's better with other the Qwen Coder models, but it was a pretty funny first look.

Re: A guide to local coding models

#145

Post by SamDc73 »

eblanshey wrote: post #82
SamDc73 wrote: post #61

If privacy is your top priority, then sure spend a few grand on hardware and run everything locally. Personally, I run a few local models (around 30B params is the ceiling on my hardware at 8k context), and I still keep a $200 ChatGPT subscription cause I'm not spending $5-6k just to run models like K2 or GLM-4.6 (they’re usable, but clearly behind OpenAI, Claude, or Gemini for my workflow) I was got excited about ae…

What hardware can you buy for $5k to be able to run K2? That's a huge model.

This older HN thread shows R1 running on a ~$2k box using ~512 GB of system RAM, no GPU, at ~3.5-4.25 TPS: https://news.ycombinator.com/item?id=42897205

If you scale that setup and add a couple of used RTX 3090s with heavy memory offloading, you can technically run something in the K2 class.

Re: A guide to local coding models

#146

Post by cmrdporcupine »

wyldfire wrote: post #113
Workaccount2 wrote: post #12

I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Somewhat comically, the author seems to have made it about 2 days. Out of 1,825. I think the real story is the folly of fixating your eyes on shiny new hardware and searching for justifications. I'm too ashamed to admit how many times I've done that dance... Local models are purely fo…

Is that really the case? This summer there was "Frontier AI performance becomes accessible on consumer hardware within a year" [1] which makes me think it's a mistake to discount the open weights models. [1] https://epoch.ai/data-insights/consumer-gpu-model-gap

With RAM prices spiking, there's no way consumers are going to have access to frontier quality models on local hardware any time soon, simply because they won't fit.

That's not the same as discounting the open weight models though. I use DeepSeek 3.2 heavily, and was impressed by the Devstral launch recently. (I tried Kimi K2 and was less impressed). I don't use them for coding so much as for other purposes... but the key thing about them is that they're cheap on API providers. I put $15 into my deepseek platform account two months ago, use it all the time, and still have $8 left.

I think the open weight models are 8 months behind the frontier models, and that's awesome. Especially when you consider you can fine tune them for a given problem domain...

Re: A guide to local coding models

#147

Post by selcuka »

artursapek wrote: post #128

Imagine buying hardware that will be obsolete in 2 years instead of paying Anthropic $200 for $1000+ worth of tokens per month

> Imagine buying hardware that will be obsolete in 2 years

Unless the PC you buy is more than $4,800 (24 x $200) it is still a good deal. For reference, a MacBook M4 Max with 128GB of unified RAM is $4,699. You need a computer for development anyway, so the extra you pay for inference is more like $2-3K.

Besides, it will still run the same model(s) at the same speed after that period, or even maybe faster with future optimisations in inference.

Re: A guide to local coding models

#148

Post by Myrmornis »

Can anyone give any tips for getting something that runs fairly fast under ollama? It doesn't have to be very intelligent.

When I tried gpt-oss and qwen using ollama on an M2 Mac the main problem was that they were extremely slow. But I did have a need for a free local model.

Re: A guide to local coding models

#149

Post by brainless »

I do not spend $100/month. I spend for 1 Claude Pro subscription and then a (much cheaper) z.ai Coding Plan, which is like one fifth the cost.

I use Claude for all my planning, create task documents and hand over to GLM 4.6. It has been my workhorse as a bootstrapped founder (building nocodo, think Lovable for AI agents).

Re: A guide to local coding models

#150

Post by am17an »

Myrmornis wrote: post #148

Can anyone give any tips for getting something that runs fairly fast under ollama? It doesn't have to be very intelligent. When I tried gpt-oss and qwen using ollama on an M2 Mac the main problem was that they were extremely slow. But I did have a need for a free local model.

Use llama.cpp? I get 250 toks/sec on gpt-oss using a 4090, not sure about the mac speeds
Post reply on HN