Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guide to local coding models

aiforswes.com

Post reply on HN
Print view View original

261–270 of 363 posts

Re: A guide to local coding models

#261

Post by alfonsodev »

kristopolous wrote: post #91
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

I use local models + openrouter free ones. My monthly spend on ai models is I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually I'll basically do $ man tool | or even $ cat source | Things I used to do intensively I now do lazily. I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentatio…

I use llm from command line too, time to time, is just easier to do

llm 'output a .gitignore file for typical python project that I can pipe into the actual file ' > .gitignore

Re: A guide to local coding models

#262

Post by bubbi »

mudkipdev wrote: post #66
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

Claude's $20 plan should be renamed to "trial". Try Opus and you will reach your limit in 10 minutes. With Sonnet, if you aren't clearing the context very often, you'll hit it within a few hours. I'm sympathetic to developers who are using this as their only AI subscription because while I was working on a challenging bug yesterday I reached the limit before it had even diagnosed the problem and had to switch to anot…

[dead]

Re: A guide to local coding models

#263

Post by kergonath »

john_minsk wrote: post #223
baq wrote: post #191

Earlier quoted context omitted.

Yes, it’s worth it, if only because that Mac will be worth $20k in 3 months…

Do you think prices will go up for mac?

That comment was a joke, but still. Resale prices for Macs are quite high. I didn’t run the calculation but it is entirely plausible the TCO including resale over a couple of years is much less than $200/month, if that’s the alternative.

Re: A guide to local coding models

#265

Post by menaerus »

smcleod wrote: post #43
Workaccount2 wrote: post #12

I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Somewhat comically, the author seems to have made it about 2 days. Out of 1,825. I think the real story is the folly of fixating your eyes on shiny new hardware and searching for justifications. I'm too ashamed to admit how many times I've done that dance... Local models are purely fo…

My 2023 Macbook Pro (M2 Max) is coming up to 3 years old and I can run models locally that are arguably "better" than what was considered SOTA about 1.5 years ago. This is of course not an exact comparison but it's close enough to give some perspective.

OpenAI released GPT-4o in May 2024, and Anthropic released Claude 3.5 Sonnet in June 2024.

I haven't tried the local models as much but I'd find it difficult to believe that they would outperform the 2024 models from OpenAI or Anthropic.

The only major algorithmic shift was done towards the RLVR and I believe it was already being applied during the 2023-2024.

Re: A guide to local coding models

#266

Post by menaerus »

hu3 wrote: post #135
wyldfire wrote: post #113

Earlier quoted context omitted.

Is that really the case? This summer there was "Frontier AI performance becomes accessible on consumer hardware within a year" [1] which makes me think it's a mistake to discount the open weights models. [1] https://epoch.ai/data-insights/consumer-gpu-model-gap

Open weight models are neat. But for SOTA performance you need specialized hardware. Even for Open Weight models. 40k in consumer hardware is never going to compete with 40k of AI specialized GPUs/servers. Your link starts with: > "Using a single top-of-the-line gaming GPU like NVIDIA’s RTX 5090 (under $2500), anyone can locally run models matching the absolute frontier of LLM performance from just 6 to 12 months ago…

> 40k in consumer hardware is never going to compete with 40k of AI specialized GPUs/servers.

For general purpose LLM probably yes. For something very domain-specialized not necessarily.

Re: A guide to local coding models

#267

Post by kristopolous »

nl wrote: post #138
kristopolous wrote: post #91

Earlier quoted context omitted.

I use local models + openrouter free ones. My monthly spend on ai models is I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually I'll basically do $ man tool | or even $ cat source | Things I used to do intensively I now do lazily. I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentatio…

This is a completely different thing to AI coding models. If you aren't using coding models you aren't ahead of the curve. There are free coding models. I use them heavily. They are ok but only partial substitutes for frontier models.

I'm extremely familiar with them.

Some people, with some tasks, get great results

But me, with my tasks, I need to maintain provenance and accountability over the code. I can't just have AI fly by the seat of its pants.

I can get into lots of detail on this. If you have seen tools and setups I have done you'd realize why it doesn't work for me.

I've spent money, the results for me, with my tasks, have not been the right decision.

Re: A guide to local coding models

#268

Post by sheepscreek »

wahnfrieden wrote: post #196
sheepscreek wrote: post #186

Earlier quoted context omitted.

I got it to try Atlas, their agentic browser, before it was open to Plus users. I convinced myself that I could use the additional capacity to multi-task and push through hard core problems without worrying about quota limits. For context, this was a few months ago when GPT 5 was new and I was used to constantly hitting o3 limits. It was an experiment to see if the higher plan could pay for itself. It most certainly…

To use up the Pro tier plan you must close the loop so to speak - so that Codex knows how to test the quality of its output and incrementally inch toward its goals. This can be harder or easier depending on your project. You should also queue up many "continue ur work" type messages.

I’m actively doing that for a fun side project - systematically rewriting SQLite in Rust. The goal is to preserve 100% compatibility, quirks and all. First I got it to run the native test harness, and now it’s basically doing TDD by itself. Have to say, with regular check-ins, it works quite well.

Note: I’m using the $20 plan for this! With codex-5.2-medium most of the time (previously codex-5.1-max-medium). For my work projects, Gemini 3 and Antigravity Claude Opus 4.5 are doing the heavy lifting at the moment, which frees up codex :) I usually have it running constantly in a second tab.

The only way I can now justify Pro is if I am developing multiple parallel projects with codex alone. But that isn’t the case for me. I am happier having a mix of agents to work with.

Re: A guide to local coding models

#269

Post by kasey_junk »

satvikpendem wrote: post #160
cmrdporcupine wrote: post #111

Earlier quoted context omitted.

"How much of the code do you manually write" I review all of it, but hand write little of it. It's bizarre how I've ended up here, but yep. That said, I wouldn't / don't trust it with something from scratch, I only trust it to do that because I built -- by hand -- a decent foundation for it to start from.

Sure, you're like me, you're not a vibe coder by the actual definition then. Still, the general trend I see is that a lot of actual vibe coders do try to get their product working, code quality be damned. Personally, same as you, I stopped vibe coding and actually started writing a lot of architecture and code myself first then allowing the LLM to fill in the features so to speak.

The issue is that your claim was that if you are using up tokens you are probably vibe coding.

But I’ve not found that to be true at all. My actually engineered processes where I care the most is where I push tokens the hardest. Mostly because I’m using llms in many places in the sdlc.

When I’m vibing it’s just a single agent sort of puttering along. It uses much fewer tokens.

Re: A guide to local coding models

#270

Post by bearjaws »

I am sorry but anyone who actually has tried this knows it is horrifically slow, significantly slower than you just typing for any model worth its weight.

That 128gb of RAM is nice but the time to first token is so long on any context over 32k, and the results are not even close to a Codex or Sonnet.

Post reply on HN