Live data from Hacker News

A guide to local coding models

aiforswes.com

121–130 of 363 posts

Re: A guide to local coding models

#121
post #114
post #88

Earlier quoted context omitted.

> Try Opus and you will reach your limit in 10 minutes. That hasn't been true with Opus 4.5. I usually hit my limit after an hour of intense sessions.

Daily limit? Weekly limit? Hitting a weekly limit after an hour still doesn't seem very productive.

Session limit that resets after 5 hours timed from the first message you sent. Most people I’ve seen report between 1 to 2 hours of dev time using Opus 4.5 on the Pro plan before hitting it unless you’re feeding in huge files and doing a bad job of managing your context.

Re: A guide to local coding models

#122
post #114

Earlier quoted context omitted.

Daily limit? Weekly limit? Hitting a weekly limit after an hour still doesn't seem very productive.

Session limit that resets after 5 hours timed from the first message you sent. Most people I’ve seen report between 1 to 2 hours of dev time using Opus 4.5 on the Pro plan before hitting it unless you’re feeding in huge files and doing a bad job of managing your context.

Okay, that sounds pretty reasonable for a $20 subscription.

Re: A guide to local coding models

#123
post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

The limits for the $20/month plan can be reached in 10-20 minutes when having it explore large codebases with directed. It’s also easy to blow right through the quota if you’re not managing content well (waiting until it fills up and then auto-compacting, or even using /compact frequently instead of /clear or the equivalent in different tools).

For most of my work I only need the LLM to perform a structured search of the codebase or to refactor something faster than I can type, so the $20/month plan is fine for me.

But for someone trying to get the LLM to write code for them, I could see the $20/month plans being exhausted very quickly. My experience with trying “vibecoding” style app development, even with highly detailed design documents and even providing test case expected output, has felt like lighting tokens on fire at a phenomenal rate. If I don’t interrupt every couple of commands and point out some mistake or wrong direction it can spin seemingly for hours trying to deal with one little problem after another. This is less obvious when doing something basic like a simple React app, but becomes extremely obvious once you deviate from material that’s represented a lot in training materials.

Re: A guide to local coding models

#124

Earlier quoted context omitted.

LMStudio is so much better than Ollama it's silly it's not more popular.

LMStudio is not open source though, ollama is but people should use llama.cpp instead

Besides optimizations specific to running locally lands in lamma.cpp first.

Re: A guide to local coding models

#125

I'm curious what the mental calculus was that a $5k laptop would competitively benchmark against SOTA models for the next 5 years was. Somewhat comically, the author seems to have made it about 2 days. Out of 1,825. I think the real story is the folly of fixating your eyes on shiny new hardware and searching for justifications. I'm too ashamed to admit how many times I've done that dance... Local models are purely fo…

That's the kind of attitude that removes power from the end user. If everything becomes SAAS you don't control anything anymore.

Re: A guide to local coding models

#126
post #5

This story talks about MLX and Ollama but doesn't mention LM Studio - https://lmstudio.ai/ LM Studio can run both MLX and GGUF models but does so from an Ollama style (but more full-featured) macOS GUI. They also have a very actively maintained model catalog at https://lmstudio.ai/models

Lmstudio runs llama.cpp under the hood.

Re: A guide to local coding models

#127

Earlier quoted context omitted.

LMStudio is so much better than Ollama it's silly it's not more popular.

LMStudio is not open source though, ollama is but people should use llama.cpp instead

Ollama did not open source their GUI.

Re: A guide to local coding models

#129

Earlier quoted context omitted.

To me, it doesn’t matter how cheap open AI codex is because that tool just burns up tokens, trying to switch to the wrong version of node using NVM on my machine. It spirals in a loop and never makes progress, for me, no matter how explicitly or verbosely i prompt. On the other hand, Claude has been nothing but productive for me. I’m also confused why you don’t assume people have the intelligence to only upgrade when…

Why is an LLM trying to switch node versions?

Because somewhere inside its little non-deterministic brain, the phrase "switch to node version xxx" was the most probable response to the previous context.

Re: A guide to local coding models

#130

Earlier quoted context omitted.

I've been a software developer for 25 years, and 30ish years in the industry, and have been programming my whole life. I worked at Google for 10 of those years. I work in C++ and Rust. I know how to write code. I don't pay $100 to "vibe code" and "learn to program" or "avoid learning to program." I pay $100 so I can get my personal (open source) projects done faster and more completely without having to hire people w…

Why would you ever hire someone to help with a personal open source project?

Depends on if the goal is to solve a problem (by writing code) or the goal is to write code (maybe solving a problem)
Post reply on HN