Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guide to local coding models

aiforswes.com

Post reply on HN
Print view View original

351–360 of 363 posts

Re: A guide to local coding models

#351

Post by littlestymaar »

ashirviskas wrote: post #340
littlestymaar wrote: post #250

Earlier quoted context omitted.

> Local models are purely for fun, hobby, and extreme privacy paranoia I always find it funny when the same people who were adamant that GPT-4 was game-changer level of intelligence are now dismissing local models that are both way more competent and much faster than GPT-4 was.

Moon lander computers were also game changers. Does not mean I should be impressed by the compute of a 30 year old calcualator that is 100x more powerful/efficient in 2025 when we have stuff a few orders of magnitude better. For simple compute, its usefulness curve is a log scale. 10x faster may only be 2x more useful. For LLMs (and human intelligence) its more quadratic, if not inverse log (140IQ human can do maths…

30-years old calculators are still good enough for basic arithmetic and in fact even in 2025 people have one emulated on their phone that isn't more powerful than the original, and people still use them routinely.

If Claude 3 Sonnet was good enough to be your daily driver last year, surely something that is as powerful is good enough to be your daily driver today. It's not like the amount of work you must do to get paid doubled over the past year or anything.

Some people just feel the need to live always on the edge for no particular reason.

Re: A guide to local coding models

#352

Post by rester324 »

onion2k wrote: post #213
throw-12-16 wrote: post #210

I never see devs containerize their coding agents. It seems so obvious to me, but I guess people are happy with claude living in their home directory and slurping up secrets.

The devs I work with don't put secrets in their home directories. ;)

How do you know? Do you snoop on their work machines?

Re: A guide to local coding models

#353

Post by rester324 »

bilater wrote: post #289

If you are using local models for coding you are midwiting this. Your code should be worth more than a subscription. The only legit use case for local models is privacy. I don't know why anyone would want to code with an intern level model when they can get a senior engineer level model for a couple of bucks more. It DOESN'T MATTER if you're writing a simple hello world function or building out a complex feature. Jus…

Is this some kind of mental problem that you want to tell people what they do and how they spend their money? Pretty jerk attitude IMO

Re: A guide to local coding models

#354

Post by pseudony »

raw_anon_1111 wrote: post #53

I don’t think I’ve ever read an article where the reason I knew the author was completely wrong about all of their assumptions was that they admitted it themselves and left the bad assumptions in the article. The above paragraph is meant to be a compliment. But justifying it based on keeping his Mac for five years is crazy. At the rate things are moving, coding models are going to get so much better in a year, the ga…

He actually addressed your point by pointing out that, in his view, the models this machine can run today are the worst it’ll ever be - he thinks, and my own experience over the last year, is that local models improve and at a pace which is CLOSING the gap to commercial models.

Re: A guide to local coding models

#355

Post by fenykep »

ashirviskas wrote: post #341
fenykep wrote: post #48

Earlier quoted context omitted.

No, they have quite a broad list of models: https://ollama.com/search [edit] Oh and apparently you can also directly run some models directly from HuggingFace: https://huggingface.co/docs/hub/ollama

Just use llama.cpp. Ollama tried to force their custom API (not the openai standard), they obscure the downloaded models making them a pain to use with other implementations, blatantly used llama.cpp as a thin wrapper without communicating it properly and now has to differentiate somehow to start making money. If you've ever used a terminal, use llama.cpp. You can also directly run models from llama.cpp afaik.

Yes, I wanted to try it already but setting up an environment with an MI50 was a bit tricky so I wanted to try something I knew first. Now that I have ollama running I will give llama.cpp a shot.

Re: A guide to local coding models

#356

Post by ashirviskas »

fenykep wrote: post #355
ashirviskas wrote: post #341

Earlier quoted context omitted.

Just use llama.cpp. Ollama tried to force their custom API (not the openai standard), they obscure the downloaded models making them a pain to use with other implementations, blatantly used llama.cpp as a thin wrapper without communicating it properly and now has to differentiate somehow to start making money. If you've ever used a terminal, use llama.cpp. You can also directly run models from llama.cpp afaik.

Yes, I wanted to try it already but setting up an environment with an MI50 was a bit tricky so I wanted to try something I knew first. Now that I have ollama running I will give llama.cpp a shot.

Ooh, I have experience with it. If you're on linux, just use Vulkan. If you face any other issues, just google my username + "MI50 32GB vbios reddit". It depends on which vBIOS you have, but that post on reddit has most of the info you may need. Good luck!

Re: A guide to local coding models

#357

Post by ashirviskas »

littlestymaar wrote: post #351
ashirviskas wrote: post #340

Earlier quoted context omitted.

Moon lander computers were also game changers. Does not mean I should be impressed by the compute of a 30 year old calcualator that is 100x more powerful/efficient in 2025 when we have stuff a few orders of magnitude better. For simple compute, its usefulness curve is a log scale. 10x faster may only be 2x more useful. For LLMs (and human intelligence) its more quadratic, if not inverse log (140IQ human can do maths…

30-years old calculators are still good enough for basic arithmetic and in fact even in 2025 people have one emulated on their phone that isn't more powerful than the original, and people still use them routinely. If Claude 3 Sonnet was good enough to be your daily driver last year, surely something that is as powerful is good enough to be your daily driver today. It's not like the amount of work you must do to get p…

Claude 3 Sonnet was good enough for many things, but not as universal as 4.5 Opus. It is immeasureably more useful to me.

I agree with all of your other points though.

Re: A guide to local coding models

#358

Post by someguyiguess »

simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

Maybe for very light work. But on the $20 subscription level I’d hit access limits every 3-4 hours.

Re: A guide to local coding models

#359

Post by someguyiguess »

uneekname wrote: post #63
simonw wrote: post #4

> I realized I looked at this more from the angle of a hobbiest paying for these coding tools. Someone doing little side projects—not someone in a production setting. I did this because I see a lot of people signing up for $100/mo or $200/mo coding subscriptions for personal projects when they likely don’t need to. Are people really doing that? If that's you, know that you can get a LONG way on the $20/month plans fr…

Yes, we are doing that. These tools help make my personal projects come to life, and the money is well worth it. I can hit Claude Code limits within an hour, and there's no way I'm giving OpenAI my money.

My thoughts exactly. The $100 Claude subscription is the sweet spot for me. I signed up for the $20 at first and got irritated constantly hitting access limits. Then I bought the $200 subscription but never even hit 1/4 of my allocation. So the $100 would be perfect.

And this is for hobby / portfolio projects.

Re: A guide to local coding models

#360

Post by someguyiguess »

satvikpendem wrote: post #95
cmrdporcupine wrote: post #77

Earlier quoted context omitted.

I've been a software developer for 25 years, and 30ish years in the industry, and have been programming my whole life. I worked at Google for 10 of those years. I work in C++ and Rust. I know how to write code. I don't pay $100 to "vibe code" and "learn to program" or "avoid learning to program." I pay $100 so I can get my personal (open source) projects done faster and more completely without having to hire people w…

I'm talking about the general trend, not the exceptions. How much of the code do you manually write with the 100 dollar subscription? Vibe coding is a descriptive, not a prescriptive, label.

How much assembly do you manually write?

Programming has always been about levels of abstraction, and the people who see LLM-generated code as “cheating” are the same people who argued that you can’t write good code with a compiler. Luddites, who will time-and-time again be proven wrong by the passage of time.

Post reply on HN