571–580 of 651 posts
I love that local LLMs are being discussed more often on HN recently. But for the post, I find it strange that the author claimed they were working with local models from day 1, but wrote a post that still links to Qwen2.5 and Qwen3 in mid June 2026.
Earlier quoted context omitted.
It can still mean Zero Data Retention - i just comes down to whether you trust the company to actually do what they promise. The fact that they've trained models on data that wasn't theirs does not make me trust them a lot when they make this claim.
When discussing this, may I ask (I know you are probably bored of the actual arguments), what does "trained models on data that wasn't theirs" actually mean in practice? Again, I know these arguments have been done to death, but every human who reads source code that wasn't written by them, or views art that wasn't created by them, and practices against this art, is training their brain on data "that wasn't theirs".…
It hinges somewhat on the concept of how much you believe things are being learned and how much is just pattern matching and borrowing a solution from memory. Certainly in the early days of Copilot it was possible to get it to output chunks of open source code near verbatim.
I think, generally, people are probably closer to believing that there is some kind of reasoning being carried out by these models than in those early days but it would also be easy to strip all of the immediately identifiable comments etc from the training materials to make it harder to detect.
I don't know about good, I use a lot of local models and they're still pretty painful to run locally You have dense models (qwen 27b, gemma 31b) who are pretty smart, but pretty slow You have MoE models (gemma 26b, qwen 35b, north mini code 30b) who are pretty fast, but make a lot of mistakes You need a lot of memory to run these well, quantization makes tool calling weaker, so most run at 4 bit quants and are wonder…
This is basically my experience as well. I have a moderately recent but high spec desktop (Radeon 6900 XT with 16 GB VRAM, Ryzen 9 7900X 12-core, 64 GB system RAM), and I tried out some recommended models with ollama a month or two ago. Anything not geared specifically towards coding seemed to struggled with actually making tool calls instead of just stating the actions they would take without making them (and trying…
They now offer DeepSeek V4 Flash for free and it def feels like a step up.
Earlier quoted context omitted.
99% of the population don’t code using models, local or remote. So that’s a useless metric. What % of developers could afford an older MacBook model, second hand? Far, far more than 1%.
But why would developers _step down_ to a Mac?
Or are you just being glib because you’ve got a misplaced superiority complex?
I don't know about good, I use a lot of local models and they're still pretty painful to run locally You have dense models (qwen 27b, gemma 31b) who are pretty smart, but pretty slow You have MoE models (gemma 26b, qwen 35b, north mini code 30b) who are pretty fast, but make a lot of mistakes You need a lot of memory to run these well, quantization makes tool calling weaker, so most run at 4 bit quants and are wonder…
This is the kind of thing that Anthropic et al should be worried about. As it becomes easier and easier to run local models, the ceiling of what they'll be able to charge will get lower and lower. Not that nobody will be willing to pay $$$$$ per month, but a lot of people are going to multiply the per-month charge by 12 or 24 and say "Could I set up a local model for less than that, and have it pay for itself within…
The general consensus is that local models will continue to improve drastically, but hosted models will as well. There will _always_ be a pretty big gulf of capability between what you can do with a desk full of hardware at home vs a few racks of hardware in a datacenter. That seems to be the real "moat" of hosted models at this point in time: access to capital. What's interesting/exciting is that local models are _a…
True, but this difference _should_ shrink with the hardware craze and we're able to buy memory (the biggest bottleneck at the moment) again, shouldn't it?
I subscribe to this guy on youtube for local model stuff if anyone is interested https://www.youtube.com/@AZisk . I'm not affiliated and I'm not even a paying subscriber. But I like all stuff local.
I clicked and immediately subscribed, but then checked out his latest videos and was so put off by the stereotypical clickbait stuff (stupid faces on thumbnails, "I tried (...) and then THIS happened" etc) that I unsubscribed. I understand that it must be what one needs to do to maximize views and brown nose the recommendation algorithm but I just find it incredibly off putting