91–100 of 153 posts
Earlier quoted context omitted.
ok, but government is how you do that. and as should be evident, its easy to year down and corrupt
Companies aren't really that better. Actually, sometimes it's even worse, because they are not even formally required to serve the public.
Earlier quoted context omitted.
There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course). So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).
Especially considering the millions of 2026-class data center GPUs that massively overinvested companies are currently buying, which will be obsolete in a few years.
Earlier quoted context omitted.
This seems wildly optimistic, do you have anything to support it?
History? This isn't the first RAM shortage. When one happens, producers build more fabs. The fabs come online, the availability of memory shoots up, and the shortage goes away, usually replaced by a glut. If you want to argue that this is different from all previous RAM shortages, you can, but the burden of proof is on you to show the difference.
this time demand doesn't stop. there is an exponential demand for tokens.
Earlier quoted context omitted.
the comparison misses that local LLM usage covers tasks you'd never send to an API — private code, offline work, medical notes. the baseline is 'local vs not-doing-it', not 'local vs cloud'
I have bad news about my private code and medical notes...
Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?
Earlier quoted context omitted.
I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run big models that serve many different people at the same time. My hunch is that the energy/water usage of the data centers is a whole lot more efficient than everyone running at home, but I'd be interested in seeing real data on that.
Water usage goes up with data centers because more cooling is needed when you run the hardware harder. So: if you're running the models on your own machine, presumably you're not running them as often, and air cooling is sufficient. But, at the same time, this is less efficient in terms of hardware use; the data centers need water cooling specifically because they're getting more bang from their buck from their hardw…
Over the long term, it seems like open models must win out. This feels like it rhymes with the story of operating systems. Despite the enormous financial contributions of Microsoft and Apple, linux still won because control matters over the long term. I predict that mech interp and things like Neuronpedia will matter more and more over time, and the frontier providers are disincentivized from providing those tools
Earlier quoted context omitted.
Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…
> You need a system with either 32+ GB VRAM I do hope you're right that it will get cheaper over time (it should), but right now 32GB of VRAM is not affordable to a lot of people. You're talking ~$4500 just for the GPU, or $800 ish used if you can find one.
Earlier quoted context omitted.
A drop in the bucket compared to the value of the collective human work that was stolen to train it. edit: come to think about it I think the ratio of one drop to one bucket is vastly over estimating the ratio of the trainer's effort.
No human work was stolen, it was read.
This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case.
Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated mediator between me and the information. I don't want a summary that was never intended by the author. I don't want the rough edges shaved off every blog until they all sound like they were written with an identical voice. Maybe some people would like that some of the time, but all?