Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

141–150 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#141

Post by Thraway198 »

mossTechnician wrote: post #100

> LeCun framed AI as an infrastructure-level platform that will soon mediate "all of our interaction with the digital world with information more generally" This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case. Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated…

I'm pretty sure they mean that AI will become the infrastructure you view the digital world though. Like, instead of using a browser, just tell the LLM what you want to go to and it will bring you there, or give you the link, or even just the info.

Re: For most of the world, open-source AI is the only way forward

#142

Post by Thraway198 »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

With the free tiers available now you can make your own document processors and other such office software, in one pass.

Re: For most of the world, open-source AI is the only way forward

#143

Post by FloatArtifact »

thewebguyd wrote: post #61
wizee wrote: post #10

Earlier quoted context omitted.

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

> You need a system with either 32+ GB VRAM I do hope you're right that it will get cheaper over time (it should), but right now 32GB of VRAM is not affordable to a lot of people. You're talking ~$4500 just for the GPU, or $800 ish used if you can find one.

Intel Arc Pro B70 32gb for $999ish https://www.newegg.com/intel-arc-pro-b70-32gb-graphics-card/...

Re: For most of the world, open-source AI is the only way forward

#144

Post by orbital-decay »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

How far back are we talking? Historically there were just a few computers in the world, personal computing revolution happened much later. And a private inference rig costs roughly the same in 2026 as a personal computer did in the early 80's, if you want the performance comparable with a personal workstation of that time (vs a mainframe/supercomputer).

Re: For most of the world, open-source AI is the only way forward

#145

Post by ktallett »

robwwilliams wrote: post #131
echelon wrote: post #12

Earlier quoted context omitted.

NO! This is the wrong approach that will turn us into serfs. We need big honking models that do what the leading foundation hyperscaler models do to within a few percentage points of measured performance. The small-scale models are not productive, and the duct tape solutions built on top of them are hobbyist-tier "year of Linux on desktop" toys. I imagine fedora-wearing, crypto-shilling, coupon-cutting boffins every…

I initial thought this was a great tongue-in-cheek comment. I still think it is a joke.

Oh no, he is just so into focusing on profit and "REAL WORK" that this is serious

Re: For most of the world, open-source AI is the only way forward

#146

Post by shark1 »

mossTechnician wrote: post #100

> LeCun framed AI as an infrastructure-level platform that will soon mediate "all of our interaction with the digital world with information more generally" This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case. Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated…

Yes. Very precise way to put that.

It's the same way the algorithm telling us what to see when we just want to see the raw feed.

In all cases, we could and can have both.

Re: For most of the world, open-source AI is the only way forward

#147

Post by onel »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Even if you go with an open source AI model, it will still make a lot more financial sense to pay a provider then actually invest in the hardware to self-host it.

It's definitely worth investing in self-hosting the agent infrastructure around the model though: all the documents, knowledge base, all the connectors, the agent itself to run on your hardware

Re: For most of the world, open-source AI is the only way forward

#148

Post by echelon »

xtracto wrote: post #138
echelon wrote: post #4

We don't need rinky-dink RTX models that budget VRAM. We need large scale open weights models just as capable as what's at the frontier. And we need the ability to rent compute and spin up the weights easily. One-click, easy enough for anyone. Easier than nerd tools like ComfyUI, Claw, and node graph garbage. Freedom is owning very large scale weights. Anything less is subsistence.

we need this: https://news.ycombinator.com/item?id=48516751 > distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments. Also The only way to prevent that one entity…

Just rent an H200.

You rent your fiber optic internet. You're doing just fine. The world isn't collapsing because you don't own the hardware racks, routers, and fiber lines.

This crazy zany P2P communal infra is Arch Linux coded - too much work, aimed at the 0.001% of users, and the juice isn't worth the squeeze.

It sounds the exact same as being mad at your ISP, so you want to build a mesh internet protocol over microwave dishes and share with everyone in your neighborhood. That was a thing in the 00's. And it predictably got nowhere and delivered no value.

Nobody's got time for this kind of stuff except for hobbyists. It's not solving a real problem. Nor does it attract business investment to grow into a healthy offering.

Just build OpenRunPod instead.

The problem is that the world doesn't have access to frontier open weights. That's the only problem to focus on and solve.

The quickest way to get there is to use large scale open weights instead of little rinky dink RTX distillations. And the easiest way to run them is by renting H100s or using a managed service offering. There's nothing wrong with either of those options.

The infrastructure is the easy part anyway.

Re: For most of the world, open-source AI is the only way forward

#149

Post by giancarlostoro »

dgellow wrote: post #127
giancarlostoro wrote: post #20

Earlier quoted context omitted.

Before the AI "crisis" it used to take about $3500 to get a prebuilt with a 5090 which can run good enough LLMs. I run reasonable LLMs on just 16GB of VRAM on my Mac, and the 5090 has double that.

What model (and quant) do you run with 16GB? I assume you have a 24GB model, and dedicate 16GB to the model?

Quite the opposite, you want something below your max VRAM and you use up to your max VRAM for the context window, so it can run longer.

https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...

Re: For most of the world, open-source AI is the only way forward

#150

Post by civet_java »

logancbrown wrote: post #109
Upvoter33 wrote: post #99

Earlier quoted context omitted.

Come on. It reads all the code in the world; then, you ask it to produce code; then, it gladly writes out something "new" which is a synthesis of what is out there, often with direct copying. It's more than reading.

Sounds like exactly what I do

So what? You're a human (I'm assuming you are) and an LLM is not. The law ought to treat humans differently.
Post reply on HN