Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

61–70 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#61

Post by thewebguyd »

wizee wrote: post #10
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

> You need a system with either 32+ GB VRAM

I do hope you're right that it will get cheaper over time (it should), but right now 32GB of VRAM is not affordable to a lot of people. You're talking ~$4500 just for the GPU, or $800 ish used if you can find one.

Re: For most of the world, open-source AI is the only way forward

#62

Post by vitally3643 »

thewebguyd wrote: post #58
tencentshill wrote: post #44

Earlier quoted context omitted.

Some consider copyright to be unethical. "Information wants to be free".

Information also wants to be expensive, and that tension will never go away. "Information wants to be free" is only one side of the context of that quote. > On the one hand you have—the point you’re making Woz—is that information sort of wants to be expensive because it is so valuable—the right information in the right place just changes your life. On the other hand, information almost wants to be free because the co…

No, greedy people want information to be expensive.

It is only because of rampant greed and capitalism that information is not free. There is nothing inherent about the collective knowledge of mankind that lends itself to being proprietary and expensive. Otherwise human society literally could not have evolved.

Re: For most of the world, open-source AI is the only way forward

#63

Post by Windchaser »

abetusk wrote: post #45
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Moore's law or one of its generalizations still holds, so it will only be a short matter of time before a $1k computer will be able to train and run a powerful enough model.

I thought Moore's Law came to an end in the last decade?

Certainly the transistors/chip or transistors/$ or flops/$ have not been progressing at the same exponential rate as during 1970-2010. There is still progress, but it's rather slower.

Re: For most of the world, open-source AI is the only way forward

#64

Post by peterlk »

Over the long term, it seems like open models must win out. This feels like it rhymes with the story of operating systems. Despite the enormous financial contributions of Microsoft and Apple, linux still won because control matters over the long term.

I predict that mech interp and things like Neuronpedia will matter more and more over time, and the frontier providers are disincentivized from providing those tools

Re: For most of the world, open-source AI is the only way forward

#65

Post by daan-k »

thewebguyd wrote: post #61
wizee wrote: post #10

Earlier quoted context omitted.

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

> You need a system with either 32+ GB VRAM I do hope you're right that it will get cheaper over time (it should), but right now 32GB of VRAM is not affordable to a lot of people. You're talking ~$4500 just for the GPU, or $800 ish used if you can find one.

For inference you can split the 32GB between two 16GB cards. Two new 5060tis for ~€1000 in total is more than fine.

It's a tad less efficient and a bit more of a hassle, but still a good experience for only a fraction of the price.

Re: For most of the world, open-source AI is the only way forward

#66

Post by crazycracker »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Historically the cost of compute has also gone down. Like just look at it as compared to a year ago. We have amazing open source models that can run on consumer hardware and if we go away from our obsession of using opus 4.8 or mythos for everything then it actually is super amazing to see what these open source models could do. I use qwen3.6:27b as a daily driver and I am heavily impressed with it.

Re: For most of the world, open-source AI is the only way forward

#67

Post by fluoridation »

verdverm wrote: post #37
simonw wrote: post #16

Earlier quoted context omitted.

I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run big models that serve many different people at the same time. My hunch is that the energy/water usage of the data centers is a whole lot more efficient than everyone running at home, but I'd be interested in seeing real data on that.

With hardware like the Spark and Strix, the water usage is known to be zero, yea? On the energy front, I assume less efficient, but I also think there is a tradeoff in efficiency versus freedom, that's why I have my own hardware.

All consumer hardware (not counting XOC) uses either air cooling or closed-loop liquid cooling, so the water usage is zero, always. Power is a little trickier. I'd assume it's less efficient, but also the total usage is less, because the user sometimes turns the machine off, and the hardware idles to a deeper sleep state than server hardware.

Re: For most of the world, open-source AI is the only way forward

#68

Post by failbuffer »

logancbrown wrote: post #38
mbgerring wrote: post #14

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

The effort of humans who had to toil through training models belongs to everyone? Do they no longer have any ownership over their hard work?

Are the going to pay for the societal harms they cause?

Re: For most of the world, open-source AI is the only way forward

#69

Post by logancbrown »

cgannett wrote: post #51
logancbrown wrote: post #38

Earlier quoted context omitted.

The effort of humans who had to toil through training models belongs to everyone? Do they no longer have any ownership over their hard work?

A drop in the bucket compared to the value of the collective human work that was stolen to train it. edit: come to think about it I think the ratio of one drop to one bucket is vastly over estimating the ratio of the trainer's effort.

No human work was stolen, it was read.

Re: For most of the world, open-source AI is the only way forward

#70

Post by nxm »

mbgerring wrote: post #14

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

It belongs to not us, but the copyright holders of that work
Post reply on HN