Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

31–40 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#31

Post by schmuhblaster »

wizee wrote: post #10
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

Indeed, and with some tinkering around the harness it can even punch way above its weight.

Re: For most of the world, open-source AI is the only way forward

#32

Post by paxys »

OsrsNeedsf2P wrote: post #29
paxys wrote: post #26

We aren’t going to have Open Source AI without Open Source hardware specs and Open Source manufacturing. Software has been solo driving open computing for far too long, and with AI now the bottlenecks are finally moving up the stack.

What..? We have open weights AI already, and even if we didn't, I don't see how Open Source hardware is a pre-requisite

Weights are meaningless if you can’t run the model from a computer under your desk.

Re: For most of the world, open-source AI is the only way forward

#33

Post by fellowmartian »

simonw wrote: post #19
mbgerring wrote: post #15

Earlier quoted context omitted.

I’d bet that there won’t be a RAM shortage for very long.

The best article I've seen about that is this one by David Oks (ignore the headline, the content is much better): https://davidoks.blog/p/ai-is-killing-the-cheap-smartphone > It was only in 2025, as memory prices began an unprecedented surge, that the memory makers started to build new fabs targeted at HBM, all slated to start producing chips in 2027 or 2028.

It still won’t help unless the AI bubble pops. Even old fabs will continue pumping out HBM instead of DRAM as long as hyperscalers gobble it up.

Re: For most of the world, open-source AI is the only way forward

#34

Post by wmf »

paxys wrote: post #26

We aren’t going to have Open Source AI without Open Source hardware specs and Open Source manufacturing. Software has been solo driving open computing for far too long, and with AI now the bottlenecks are finally moving up the stack.

I don't see any evidence of this. Open source software thrived on proprietary hardware for decades.

Re: For most of the world, open-source AI is the only way forward

#35

Post by verdverm »

blakesterz wrote: post #6

There's a video of the entire session here: https://webtv.un.org/en/asset/k14/k14ej1ucqu?kalturaStartTim... (if that link doesn't work, it starts about 12 minutes into the start)

had to click the play button, but it keyed to the 12m mark

Re: For most of the world, open-source AI is the only way forward

#36

Post by wrs »

jochem9 wrote: post #21
wizee wrote: post #10

Earlier quoted context omitted.

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course). So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).

Especially considering the millions of 2026-class data center GPUs that massively overinvested companies are currently buying, which will be obsolete in a few years.

Re: For most of the world, open-source AI is the only way forward

#37

Post by verdverm »

simonw wrote: post #16
ktallett wrote: post #7

Earlier quoted context omitted.

We need to improve the waster and energy usage and this method doesn't. Most are not reinventing the wheel, a shared AI repository, communicated between online local computers would save a lot of need for these large models.

I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run big models that serve many different people at the same time. My hunch is that the energy/water usage of the data centers is a whole lot more efficient than everyone running at home, but I'd be interested in seeing real data on that.

With hardware like the Spark and Strix, the water usage is known to be zero, yea?

On the energy front, I assume less efficient, but I also think there is a tradeoff in efficiency versus freedom, that's why I have my own hardware.

Re: For most of the world, open-source AI is the only way forward

#38

Post by logancbrown »

mbgerring wrote: post #14

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

The effort of humans who had to toil through training models belongs to everyone? Do they no longer have any ownership over their hard work?

Re: For most of the world, open-source AI is the only way forward

#39

Post by wrs »

paxys wrote: post #32
OsrsNeedsf2P wrote: post #29

Earlier quoted context omitted.

What..? We have open weights AI already, and even if we didn't, I don't see how Open Source hardware is a pre-requisite

Weights are meaningless if you can’t run the model from a computer under your desk.

How does that follow? Plenty of open source software runs in a commodity datacenter. This is about the API bottleneck, not the physical location of the GPU.

Re: For most of the world, open-source AI is the only way forward

#40

Post by cyanydeez »

mbgerring wrote: post #14

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

ok, but government is how you do that. and as should be evident, its easy to year down and corrupt
Post reply on HN