Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guide to local coding models

aiforswes.com

Post reply on HN
Print view View original

271–280 of 363 posts

Re: A guide to local coding models

#271

Post by throwthrowuknow »

deaux wrote: post #122
throwthrowuknow wrote: post #121

Earlier quoted context omitted.

Session limit that resets after 5 hours timed from the first message you sent. Most people I’ve seen report between 1 to 2 hours of dev time using Opus 4.5 on the Pro plan before hitting it unless you’re feeding in huge files and doing a bad job of managing your context.

Okay, that sounds pretty reasonable for a $20 subscription.

Yeah it’s really not too bad but it does get frustrating when you hit the session limit in the middle of something. I also add $20 of extra usage so I can finish up the work in progress cleanly and have Opus create some notes so we can resume when the session renews. Gotta be careful with extra usage though because you can easily use it up if the context is getting full so it’s best to try to work in small independent chunks and clear the context after each. It’s more work but helps both with usage and Opus performs better when you aren’t pushing the context window to the max.

Re: A guide to local coding models

#272

Post by throwthrowuknow »

lodovic wrote: post #183
throwthrowuknow wrote: post #118

Earlier quoted context omitted.

I half agree, but it should be called “Hobbiest” since that’s what it’s good for. 10 minutes is hyperbolic, I average 1h30m even when using plan mode first and front loading the context with dev diaries, git history, milestone documents and important excerpts from previous conversations. Something tells me your modules might be too big and need refactoring. That said, it’s a pain having to wait hours between sessions…

I actually get more mileage out of Claude using a Github Copilot subscription. The regular Claude Pro will give me an hour or up to 90 minutes max, before it reaches the cap. The Github version has a monthly limit for the Claude requests (100 "premium requests") which I find much easier to manage. I was about to switch to the max plan but this setup (both Claude pro and Github Copilot, costing 30 a month together) wa…

Good to hear that’s working. When I was using copilot before Opus 4.5 came out I found it didn’t perform as well as Claude Code but maybe it works better now with 4.5 and the latest improvements to VSCode. I’ll have to try it again.

Re: A guide to local coding models

#273

Post by KronisLV »

cmrdporcupine wrote: post #76
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

Codex $20 is a good deal but they have nothing inbetween $20 and $200. The $20 Anthropic plan is only enough to wet my appetite, I can't finish anything. I pay for $100 Anthropic plan, and keep a $20 Codex plan in my back pocket for getting it to do additional review and analysis overtop of what Opus cooks up. And I have a few small $ of misc credits in DeepSeek and Kimi K2 AI services mainly to try them out, and for…

Idk why the gap is so big, surely a bunch of people would also pay 50$ a month across multiple vendors for medium amount of tokens.

Re: A guide to local coding models

#274

Post by qayxc »

BoredPositron wrote: post #253
qayxc wrote: post #235

Earlier quoted context omitted.

Hard disagree. The difference in performance is not something you'll notice if you actually use these cards. In AI benchmarks, the RTX 3090 beats the RTX 4080 SUPER, despite the latter having native BF16 support. 736GiB/s (4080) memory bandwidth vs 936 GiB/s (3090) plays a major role. Additionally, the 3090 is not only the last NVIDIA consumer card to support SLI. It's also unbeatable in price to performance as the n…

If you run fp32 maybe but no sane person does that. The tensor performance of the 3090 is also abysmal. If you run bf16 or fp8 stay away from obsolete cards. Its barely usable for llms and borderline garbage tier on video and image gen.

Actual benchmarks show otherwise.

> The tensor performance of the 3090 is also abysmal.

I for one compared my 50-series card's performance to my 3090 and didn't see "abysmal performance" on the older card at all. In fact, in actual real-world use (quantised models only, no one runs big fp32 models locally), the difference in performance isn't very noticeable at all. But I'm sure you'll be able to provide actual numbers (TTFT, TPS) to prove me wrong. I don't use diffusion models, so there might be a substantial difference there (I doubt it, though), but for LLMs I can tell you for a fact that you're just wrong.

Re: A guide to local coding models

#275

Post by cmrdporcupine »

KronisLV wrote: post #273
cmrdporcupine wrote: post #76

Earlier quoted context omitted.

Codex $20 is a good deal but they have nothing inbetween $20 and $200. The $20 Anthropic plan is only enough to wet my appetite, I can't finish anything. I pay for $100 Anthropic plan, and keep a $20 Codex plan in my back pocket for getting it to do additional review and analysis overtop of what Opus cooks up. And I have a few small $ of misc credits in DeepSeek and Kimi K2 AI services mainly to try them out, and for…

Idk why the gap is so big, surely a bunch of people would also pay 50$ a month across multiple vendors for medium amount of tokens.

Indeed I would consider switching to Codex completely if a) they had a $100 or $50 membership b) they really worked on improving the CLI tool a lot more. It's about 4-6 months behind Claude Code

Re: A guide to local coding models

#276

Post by cdelsolar »

throwthrowuknow wrote: post #118
mudkipdev wrote: post #66

Earlier quoted context omitted.

Claude's $20 plan should be renamed to "trial". Try Opus and you will reach your limit in 10 minutes. With Sonnet, if you aren't clearing the context very often, you'll hit it within a few hours. I'm sympathetic to developers who are using this as their only AI subscription because while I was working on a challenging bug yesterday I reached the limit before it had even diagnosed the problem and had to switch to anot…

I half agree, but it should be called “Hobbiest” since that’s what it’s good for. 10 minutes is hyperbolic, I average 1h30m even when using plan mode first and front loading the context with dev diaries, git history, milestone documents and important excerpts from previous conversations. Something tells me your modules might be too big and need refactoring. That said, it’s a pain having to wait hours between sessions…

The word is hobbyist btw, not that you're the source for this typo, it seems to have percolated downwards from the blog post through these comments.

Re: A guide to local coding models

#277

Post by fragmede »

kristopolous wrote: post #91
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

I use local models + openrouter free ones. My monthly spend on ai models is I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually I'll basically do $ man tool | or even $ cat source | Things I used to do intensively I now do lazily. I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentatio…

> My monthly spend on ai models is > I'm not cheap

You're cheap. It's okay. We're all developers here. It's a safe space.

Re: A guide to local coding models

#278

Post by KronisLV »

zarzavat wrote: post #198
nl wrote: post #176

Earlier quoted context omitted.

Is 4 TPS actually useful for anything? That's around 350,000 tokens in a day. I don't track my Claude/Codex usage, but Kilocode with the free Grok model does and I'm using between 3.3M and 50M tokens in a day (plus additional usage in Claude + Codex + Mistral Vibe + Amp Coder) I'm trying to imagine a use case where I'd want this. Maybe running some small coding task overnight? But it just doesn't seem very useful.

3.5-50M tokens a day? What are you doing with all those tokens? Yesterday I asked Claude to write one function. I didn't ask it to do anything else because it wouldn't have been helpful.

Here’s my own stats, for comparison: https://news.ycombinator.com/item?id=46216192

Essentially migrating codebases, implementing features, as well as all of the referencing of existing code and writing tests and various automation scripts that are needed to ensure that the code changes are okay. Over 95% of those tokens are reads, since often there’s a need for a lot of consistency and iteration.

It works pretty well if you’re not limited by a tight budget.

Re: A guide to local coding models

#279

Post by mathgeek »

fragmede wrote: post #277
kristopolous wrote: post #91

Earlier quoted context omitted.

I use local models + openrouter free ones. My monthly spend on ai models is I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually I'll basically do $ man tool | or even $ cat source | Things I used to do intensively I now do lazily. I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentatio…

> My monthly spend on ai models is > I'm not cheap You're cheap. It's okay. We're all developers here. It's a safe space.

While I say this somewhat in jest, frugal is just cheap but with better value.

Re: A guide to local coding models

#280

Post by JamesSwift »

Aurornis wrote: post #123
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

The limits for the $20/month plan can be reached in 10-20 minutes when having it explore large codebases with directed. It’s also easy to blow right through the quota if you’re not managing content well (waiting until it fills up and then auto-compacting, or even using /compact frequently instead of /clear or the equivalent in different tools). For most of my work I only need the LLM to perform a structured search of…

That has not been my experience with sonnet, and even so it is largely remedied by having better AI docs caching the results of that investigation for future use.
Post reply on HN