Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

21–30 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#21

Post by jochem9 »

wizee wrote: post #10
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course).

So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).

Re: For most of the world, open-source AI is the only way forward

#22

Post by majormajor »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

> Historically a good Windows computer cost $1000 and it was all it took to start programming.

Gotta remember inflation here.

$1K in 1995 was roughly equivalent to $2K now and wouldn't have been a particularly "good" machine then.

In 1982 the Commodore 64 started at about $600 bucks, also roughly around $2K today.

If you outgrew that, beefier machines back then were A LOT. It was easy to find $2k+ towers and (especially) laptops even into the 2000s, and a lot of those would be $5K+ equivalent today.

Re: For most of the world, open-source AI is the only way forward

#23

Post by Avicebron »

mbgerring wrote: post #15
simonw wrote: post #13

Earlier quoted context omitted.

... or rising, at least as long as there's a RAM shortage.

I’d bet that there won’t be a RAM shortage for very long.

This seems wildly optimistic, do you have anything to support it?

Re: For most of the world, open-source AI is the only way forward

#25

Post by cold_harbor »

simonw wrote: post #16
ktallett wrote: post #7

Earlier quoted context omitted.

We need to improve the waster and energy usage and this method doesn't. Most are not reinventing the wheel, a shared AI repository, communicated between online local computers would save a lot of need for these large models.

I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run big models that serve many different people at the same time. My hunch is that the energy/water usage of the data centers is a whole lot more efficient than everyone running at home, but I'd be interested in seeing real data on that.

the comparison misses that local LLM usage covers tasks you'd never send to an API — private code, offline work, medical notes. the baseline is 'local vs not-doing-it', not 'local vs cloud'

Re: For most of the world, open-source AI is the only way forward

#27

Post by rayiner »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Isn’t this just a bet that I’ll have an AI data center in my iPhone within 10 years? Why is that a bad bet?

Re: For most of the world, open-source AI is the only way forward

#28

Post by oceanplexian »

robwwilliams wrote: post #17

Yann is on the mark. Almost amusing to see the EU along with its many former “subjects” realize they are at great risk of joint Chinese-American hegemony in AI. We should all be more terrified of a few nation states defining the agendas and policies of AI use than current Ai variants that a inherently without purpose or autonomy. Great analogy to the fear of the printing press being really bad news in that it enabled…

AI is the canary in the coal mine. They don't have an AI problem they have an everything problem. Inability to maintain energy security, declines in manufacturing, their social programs are no longer sustainable (Pension age rises and reforms), German car industry is in decline, increased spending demands for defense, and so on.

All that's needed is another sovereign debt crisis to spark what is essentially dry tinder and I think the EU is a lot closer to collapsing than anyone even remotely realizes.

Re: For most of the world, open-source AI is the only way forward

#29

Post by OsrsNeedsf2P »

paxys wrote: post #26

We aren’t going to have Open Source AI without Open Source hardware specs and Open Source manufacturing. Software has been solo driving open computing for far too long, and with AI now the bottlenecks are finally moving up the stack.

What..? We have open weights AI already, and even if we didn't, I don't see how Open Source hardware is a pre-requisite
Post reply on HN