Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

111–120 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#111

Post by Tuna-Fish »

treis wrote: post #57
wrs wrote: post #36

Earlier quoted context omitted.

Especially considering the millions of 2026-class data center GPUs that massively overinvested companies are currently buying, which will be obsolete in a few years.

I think those are going to be run until they die. The capex vs opex is too high to obsolete them in a few years. They'll keep serving current gen LLMs for as long as they keep running.

It won't make sense to run them after two years. The vendors will be limited on datacenter space, power and cooling, and there will be new hardware available that will run the same models at a fraction of the power.

A100 -> H100 was >3x tokens per joule, H100 -> B200 >10x. There are significant low-hanging fruit still available in architectural efficiency, and the vendors are chasing them.

This is the big risk for AI companies that I feel is not being sufficiently priced in. Almost none of the investments they are making are durable, the depreciation schedules for everything but the real estate should be less than 24 months. Until the hardware is stable enough that you only get double-digit % improvements per generation, it should almost be counted as opex.

Re: For most of the world, open-source AI is the only way forward

#112

Post by Tuna-Fish »

Chu4eeno wrote: post #11
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Open weights/source doesn't necessarily mean running on local hardware, though. I imagine having multiple providers competing will drive down hosted versions of open weight models drastically.

You are describing OpenRouter. And yes, it does.

Re: For most of the world, open-source AI is the only way forward

#113

Post by grim_io »

mossTechnician wrote: post #100

> LeCun framed AI as an infrastructure-level platform that will soon mediate "all of our interaction with the digital world with information more generally" This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case. Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated…

This is already the reality for a large portion of the population.

They(we) don't consume directly from the firehose of the internet, but from whatever "The Algorithm" has curated for them(us).

Re: For most of the world, open-source AI is the only way forward

#114

Post by robwwilliams »

wrs wrote: post #36
jochem9 wrote: post #21

Earlier quoted context omitted.

There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course). So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).

Especially considering the millions of 2026-class data center GPUs that massively overinvested companies are currently buying, which will be obsolete in a few years.

I think you are right when you factor in the much more efficient newer high end GPU. That is what may make the current GPu investments obsolete in 2-4 years.

Re: For most of the world, open-source AI is the only way forward

#115

Post by robwwilliams »

Chu4eeno wrote: post #83
treis wrote: post #57

Earlier quoted context omitted.

I think those are going to be run until they die. The capex vs opex is too high to obsolete them in a few years. They'll keep serving current gen LLMs for as long as they keep running.

They can also be used for other things than running the main frontier whatever model as well. E.g. grok isn't truly multi-modal, it has a callable tool that is a separate VLM it invokes on image URLs or files (for a long time it was grok-1.5v, but I think they have upgraded now, it was pretty bad). And then you have the small summarizer models for the CoT/thought traces, the guidable summarizer models for the standar…

Yes, sure, but not efficiently. Even Pops will not want to run four hair dyer GPUs 24-7 in the garage.

Re: For most of the world, open-source AI is the only way forward

#116

Post by stackedinserter »

mbgerring wrote: post #14

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

It's closer to "that output belongs to everyone, everyone's free to train your own model"

Otherwise, everything you do belongs to all of us, because you learned on total intellectual and artistic output of human kind too.

Re: For most of the world, open-source AI is the only way forward

#117

Post by blitzar »

oceanplexian wrote: post #28
robwwilliams wrote: post #17

Yann is on the mark. Almost amusing to see the EU along with its many former “subjects” realize they are at great risk of joint Chinese-American hegemony in AI. We should all be more terrified of a few nation states defining the agendas and policies of AI use than current Ai variants that a inherently without purpose or autonomy. Great analogy to the fear of the printing press being really bad news in that it enabled…

AI is the canary in the coal mine. They don't have an AI problem they have an everything problem. Inability to maintain energy security, declines in manufacturing, their social programs are no longer sustainable (Pension age rises and reforms), German car industry is in decline, increased spending demands for defense, and so on. All that's needed is another sovereign debt crisis to spark what is essentially dry tinde…

> EU is a lot closer to collapsing than anyone even remotely realizes

I have been hearing that will happen next year for the last 20 years. No this time it isn't different.

Re: For most of the world, open-source AI is the only way forward

#118

Post by yowlingcat »

wizee wrote: post #10
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry…

I've seen folks make it work with a 3090 on 4 bit quant using turboquant for KV cache. That's key because 3090s remain the most cost effective gpu metal for enthusiasts (albeit 24g) and the jump to 5090 (32g) is quite expensive and not always worth the LLM specific performance; sadly, good 32g metal is somewhat lacking in the price point at or above the 3090.

Re: For most of the world, open-source AI is the only way forward

#119

Post by swiftcoder »

nok22kon wrote: post #93
AnimalMuppet wrote: post #43

Earlier quoted context omitted.

History? This isn't the first RAM shortage. When one happens, producers build more fabs. The fabs come online, the availability of memory shoots up, and the shortage goes away, usually replaced by a glut. If you want to argue that this is different from all previous RAM shortages, you can, but the burden of proof is on you to show the difference.

there is a glut if demand stops. this time demand doesn't stop. there is an exponential demand for tokens.

> there is an exponential demand for tokens

[citation needed]

There is certainly economic pressure to create an exponential demand for tokens, but we've already seen a pullback from the costly "token maxing" companies were pushing last year.

It's also pretty unclear to what degree the RAM shortage is driven by inference (versus by training). We're rapidly approaching the point where frontier models are "good enough" for everyday use, are at some point we're going to hit diminishing returns on training new trillion-parameter models...

Re: For most of the world, open-source AI is the only way forward

#120

Post by Supermancho »

skydhash wrote: post #82
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

> Historically a good Windows computer cost $1000 and it was all it took to start programming Started with computers around 2009 and later bought an oldish computer (a pentium 4 PC) for the equivalent of 50 usd. Codeblocks and Python Idle were free at the time (C and Python were the first languages I learned). The barrier to programming has always been low as the only thing you needed was books (the internet made thi…

In 1998 I had a second PC for the first time in my life. It was an old i486DX2-66 that my dad gave to me. He said it cost him $50 secondhand and was bought so I could put Linux on it.
Post reply on HN