Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guide to local coding models

aiforswes.com

Post reply on HN
Print view View original

311–320 of 363 posts

Re: A guide to local coding models

#311

Post by ryandrake »

slicktux wrote: post #51
ekjhgkejhgk wrote: post #17

Earlier quoted context omitted.

I agree with everything you said, and yet I cannot help but respect a person who wants to do it himself. It reminds me of the hacker culture of the 80s and 90s.

Agreed, Everyone seems to shun the DIY hacker now a days; saying things like “I’ll just pay for it”. It’s not about just NOT paying for it but doing it yourself and learning how to do it so that you can pass the knowledge on and someone else can do it.

And, it's not just about "pay for it" vs. "don't pay for it". It's about needing to pay for it monthly or it goes away. I hate subscriptions. They sneak their way into your life, little by little. $4.99/mo here. $9.99/mo there. $24.99/yr elsewhere. And then at some point, in a moment of clarity, you wake up and look at your monthly expenses and notice you're paying a fortune just to exist in your life as you are existing.

I'm not going to pay monthly for X service when similar Y thing can be purchased once (or ideally open source downloaded), self-hosted, and it's your setup forever.

Re: A guide to local coding models

#313

Post by ekjhgkejhgk »

ryandrake wrote: post #311
slicktux wrote: post #51

Earlier quoted context omitted.

Agreed, Everyone seems to shun the DIY hacker now a days; saying things like “I’ll just pay for it”. It’s not about just NOT paying for it but doing it yourself and learning how to do it so that you can pass the knowledge on and someone else can do it.

And, it's not just about "pay for it" vs. "don't pay for it". It's about needing to pay for it monthly or it goes away. I hate subscriptions. They sneak their way into your life, little by little. $4.99/mo here. $9.99/mo there. $24.99/yr elsewhere. And then at some point, in a moment of clarity, you wake up and look at your monthly expenses and notice you're paying a fortune just to exist in your life as you are exis…

> or ideally open source downloaded

Ideally Free software downloaded. Even more ideally copyleft Free software downloaded.

Re: A guide to local coding models

#314

Post by jgalt212 »

bilater wrote: post #305
Aurornis wrote: post #297

Earlier quoted context omitted.

> I'd be very excited if Jane Street or DE Shaw were running their trading models through Claude. Then I'd have access to billions of dollars of secrets. Using Claude for inference does not mean the codebase gets pulled into their training set. This is a tired myth that muddies up every conversation about LLMs

lol yeah its weird to me why even ppl on HN can't wrap their heads around stateless calls.

unless you control both the client and the server, you cannot prove a call is stateless.

Re: A guide to local coding models

#315

Post by larodi »

jwr wrote: post #232
larodi wrote: post #217

Earlier quoted context omitted.

Claude Code is a lot about prompting and orchestration of the conversation. The LLM is just a tool in these agentic frameworks. Whats truly ingenious is how context is engineered/managed, how is the code-RAG approached, and them LLM memory that is used. So my guess would be - we need open conversation or something along the line of "useful linguistic-AI approaches for combing and grooming code"

Agreed. I've been trying to use opencode and crush, and none of them do anything useful for me. In contrast, claude code "just works" and does genuinely useful work. And it's not just because of the specific LLM used, it's the overall engineering of the tool, the prompt behind the scenes, etc. But the bottom line is that I still can't find a way to use either local LLMs and/or opencode and crush for coding.

Which is very sad and perhaps she should be aiming to introduce some very smart linguists into the whole ML:LLM thing that can learn and explore how to best to interact with the funny archive that models are.

Re: A guide to local coding models

#316

Post by pcl »

bilater wrote: post #304
pcl wrote: post #293

Earlier quoted context omitted.

Or if you want to do development work while offline.

Good to have fallbacks but in reality most ppl ( at least in the west) will have internet 99% of the time.

Sure, but I am not one of them. I find myself wanting to code on trains and planes pretty often, and so local toolchains are always attractive for me.

Re: A guide to local coding models

#318

Post by beeboop0 »

bilater wrote: post #289

If you are using local models for coding you are midwiting this. Your code should be worth more than a subscription. The only legit use case for local models is privacy. I don't know why anyone would want to code with an intern level model when they can get a senior engineer level model for a couple of bucks more. It DOESN'T MATTER if you're writing a simple hello world function or building out a complex feature. Jus…

[dead]

Re: A guide to local coding models

#319

Post by stronglikedan »

simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

> The OpenAI one in particular is a great deal, because Codex is charged a whole lot lower than Claude.

From what my team tells me, it's not a great deal since it's so far behind Claude in capabilities and IDE integration.

Re: A guide to local coding models

#320

Post by techwizrd »

kristopolous wrote: post #91
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

I use local models + openrouter free ones. My monthly spend on ai models is I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually I'll basically do $ man tool | or even $ cat source | Things I used to do intensively I now do lazily. I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentatio…

Have you looked at tldr/tealdeer[0]? It may do much of what you're looking for, albeit without LLM assistance.

0: https://tealdeer-rs.github.io/tealdeer/

Post reply on HN