Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

121–130 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#121

Post by seanclayton »

petcat wrote: post #74
peterlk wrote: post #64

Over the long term, it seems like open models must win out. This feels like it rhymes with the story of operating systems. Despite the enormous financial contributions of Microsoft and Apple, linux still won because control matters over the long term. I predict that mech interp and things like Neuronpedia will matter more and more over time, and the frontier providers are disincentivized from providing those tools

> linux still won because control matters over the long term. what has Linux won? Servers? sure

Laptops with Chromebooks. Mobile with Android phones. Game consoles in Steam decks and now Steam Machines. The things linux appears on are growing. The things proprietary OSes appear on are shrinking. Losers and victors in a war are often defined by who loses and who gains the most.

Re: For most of the world, open-source AI is the only way forward

#122

Post by abetusk »

Windchaser wrote: post #63
abetusk wrote: post #45

Earlier quoted context omitted.

Moore's law or one of its generalizations still holds, so it will only be a short matter of time before a $1k computer will be able to train and run a powerful enough model.

I thought Moore's Law came to an end in the last decade? Certainly the transistors/chip or transistors/$ or flops/$ have not been progressing at the same exponential rate as during 1970-2010. There is still progress, but it's rather slower.

I was careful to say "Moore's law-like". Moore's law is stated as "transistor count per area doubles about every 2 years." [0] As stated, then yes, this might be true but, while important, that's not really the quantity we care about.

As you point out it's really cost per transistor or cost per flop that we mostly care about. I'm finding it hard to find a succinct and clear plot, but I believe one is provided by Our World In Data on "GPU computational performance per dollar" [1] which, to my eyes, clearly shows exponential growth in computational power per dollar.

The picture for storage is a little more muddied but if you squint just right you might still be able to recover an uninterrupted exponential growth [2].

In my view, it's pretty clear that advances in AI have progressed so quickly because GPUs have been keeping up with the exponential growth of computational power (per unit cost).

Exponential growth in this area is usually characterized by "S-curves", where one technology gets saturated but the exponential increase in power or decrease in cost is picked up by another, adjacent, technology, that allows the growth to continue. For compute it's CPUs to GPUs. For storage it's platter drives that are now being overtaken by SSDs.

The more general phenomena is called Wright's law, or experience curve effects [3].

[0] https://en.wikipedia.org/wiki/Moore%27s_law

[1] https://ourworldindata.org/grapher/gpu-price-performance?ySc...

[2] https://ourworldindata.org/grapher/historical-cost-of-comput...

[3] https://en.wikipedia.org/wiki/Experience_curve_effect

Re: For most of the world, open-source AI is the only way forward

#123

Post by Chu4eeno »

anigbrowl wrote: post #108
Chu4eeno wrote: post #91

Earlier quoted context omitted.

A big reason for that is the dumb decision from Dodge v. Ford Motor Co., giving us this quasi-capitalism which optimizes for the wrong thing.

LEgislation could make that obsolete but it seems like most/all of our legislative institutions are captured by business interests.

Yeah, it's hard to get someone to understand a problem when their salary (or campaign contributions) depends on them not understanding it, or however the quote goes.

Re: For most of the world, open-source AI is the only way forward

#124

Post by Chu4eeno »

Tuna-Fish wrote: post #112
Chu4eeno wrote: post #11

Earlier quoted context omitted.

Open weights/source doesn't necessarily mean running on local hardware, though. I imagine having multiple providers competing will drive down hosted versions of open weight models drastically.

You are describing OpenRouter. And yes, it does.

Well, open router is a bit more general, since it just proxies requests to both proprietary and open weights models.

I was thinking more of the providers of inference on open weight models that openrouter proxies to.

Re: For most of the world, open-source AI is the only way forward

#126

Post by nicoburns »

pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

I think AI is still in it's "mainframe era". Once upon a time, computers that ran useful workloads filled a whole room. And now that's true of computers that run (state of the art) AI models. My guess would be that this won't always be the case, although I certainly don't claim to have a clear idea of how things will play out.

Re: For most of the world, open-source AI is the only way forward

#127

Post by dgellow »

giancarlostoro wrote: post #20
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Before the AI "crisis" it used to take about $3500 to get a prebuilt with a 5090 which can run good enough LLMs. I run reasonable LLMs on just 16GB of VRAM on my Mac, and the 5090 has double that.

What model (and quant) do you run with 16GB? I assume you have a 24GB model, and dedicate 16GB to the model?

Re: For most of the world, open-source AI is the only way forward

#128

Post by dgellow »

snootypoot wrote: post #125

not only is he correct, but this path will result in openai going the way of enron. i hope in the case of openai and their collaborators in nvidia we see more people going to prison.

Would need changes in the US government before that can happen

Re: For most of the world, open-source AI is the only way forward

#129

Post by tensegrist »

wrs wrote: post #36
jochem9 wrote: post #21

Earlier quoted context omitted.

There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course). So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).

Especially considering the millions of 2026-class data center GPUs that massively overinvested companies are currently buying, which will be obsolete in a few years.

aiui nvidia has buyback agreements covering these to prevent them from being sold off

Re: For most of the world, open-source AI is the only way forward

#130

Post by robwwilliams »

oceanplexian wrote: post #28
robwwilliams wrote: post #17

Yann is on the mark. Almost amusing to see the EU along with its many former “subjects” realize they are at great risk of joint Chinese-American hegemony in AI. We should all be more terrified of a few nation states defining the agendas and policies of AI use than current Ai variants that a inherently without purpose or autonomy. Great analogy to the fear of the printing press being really bad news in that it enabled…

AI is the canary in the coal mine. They don't have an AI problem they have an everything problem. Inability to maintain energy security, declines in manufacturing, their social programs are no longer sustainable (Pension age rises and reforms), German car industry is in decline, increased spending demands for defense, and so on. All that's needed is another sovereign debt crisis to spark what is essentially dry tinde…

Not how I see it. And I have seen Europe close up for 60 years. It is wealthier and happier than it was even 15 years ago. Visit Spain, Portugal, Greece, and of course the other larger northern states; no comparison. Immigration has served them well in wealth and health. Compare Europe to the hollowed out core of the USA and you will shudder. Visit rural America.

The European “problem” is they live to have a life whereas in the USA we seem to live to work.

Post reply on HN