Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For most of the world, open-source AI is the only way forward

techstrong.ai

Post reply on HN
Print view View original

131–140 of 153 posts

Re: For most of the world, open-source AI is the only way forward

#131

Post by robwwilliams »

echelon wrote: post #12
ktallett wrote: post #7

Earlier quoted context omitted.

We need to improve the waster and energy usage and this method doesn't. Most are not reinventing the wheel, a shared AI repository, communicated between online local computers would save a lot of need for these large models.

NO! This is the wrong approach that will turn us into serfs. We need big honking models that do what the leading foundation hyperscaler models do to within a few percentage points of measured performance. The small-scale models are not productive, and the duct tape solutions built on top of them are hobbyist-tier "year of Linux on desktop" toys. I imagine fedora-wearing, crypto-shilling, coupon-cutting boffins every…

I initial thought this was a great tongue-in-cheek comment. I still think it is a joke.

Re: For most of the world, open-source AI is the only way forward

#132

Post by oceanplexian »

robwwilliams wrote: post #130
oceanplexian wrote: post #28

Earlier quoted context omitted.

AI is the canary in the coal mine. They don't have an AI problem they have an everything problem. Inability to maintain energy security, declines in manufacturing, their social programs are no longer sustainable (Pension age rises and reforms), German car industry is in decline, increased spending demands for defense, and so on. All that's needed is another sovereign debt crisis to spark what is essentially dry tinde…

Not how I see it. And I have seen Europe close up for 60 years. It is wealthier and happier than it was even 15 years ago. Visit Spain, Portugal, Greece, and of course the other larger northern states; no comparison. Immigration has served them well in wealth and health. Compare Europe to the hollowed out core of the USA and you will shudder. Visit rural America. The European “problem” is they live to have a life whe…

I was in Italy 6 months ago, more than one person told me they are at the point of desperation due to the incredibly low wages (<3000 EUR/month).

Re: For most of the world, open-source AI is the only way forward

#133

Post by treis »

Tuna-Fish wrote: post #111
treis wrote: post #57

Earlier quoted context omitted.

I think those are going to be run until they die. The capex vs opex is too high to obsolete them in a few years. They'll keep serving current gen LLMs for as long as they keep running.

It won't make sense to run them after two years. The vendors will be limited on datacenter space, power and cooling, and there will be new hardware available that will run the same models at a fraction of the power. A100 -> H100 was >3x tokens per joule, H100 -> B200 >10x. There are significant low-hanging fruit still available in architectural efficiency, and the vendors are chasing them. This is the big risk for AI…

The annual operating cost of these is As it stands there's way more demand than supply. The new GPUs are going to run frontier models while the older ones serve smaller ones.

That said some of these are running in tents hooked up to mobile turbines. I can see some of those going away but generally I think you'll see them used until they start to fail in 5-10 years.

Re: For most of the world, open-source AI is the only way forward

#134

Post by jazzyjackson »

stackedinserter wrote: post #116
mbgerring wrote: post #14

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

It's closer to "that output belongs to everyone, everyone's free to train your own model" Otherwise, everything you do belongs to all of us, because you learned on total intellectual and artistic output of human kind too.

No I did not illegally download every book ever published. anyone who went to college paid good money for their textbooks.

Re: For most of the world, open-source AI is the only way forward

#135

Post by 627467 »

cesarvarela wrote: post #104
mossTechnician wrote: post #100

> LeCun framed AI as an infrastructure-level platform that will soon mediate "all of our interaction with the digital world with information more generally" This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case. Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated…

As AI amplifies everyone’s output, you’ll need AI to amplify your input capacity—or you’ll fall behind.

I thought this line of argument ended with the fall of the iron wall. Just because you can put every citizen digging holes to increase gdp it doesnt mean you should

Re: For most of the world, open-source AI is the only way forward

#136

Post by jazzyjackson »

logancbrown wrote: post #69
cgannett wrote: post #51

Earlier quoted context omitted.

A drop in the bucket compared to the value of the collective human work that was stolen to train it. edit: come to think about it I think the ratio of one drop to one bucket is vastly over estimating the ratio of the trainer's effort.

No human work was stolen, it was read.

It was downloaded without a license to read it

Re: For most of the world, open-source AI is the only way forward

#137

Post by jazzyjackson »

nok22kon wrote: post #96
Windchaser wrote: post #73

Earlier quoted context omitted.

Water usage goes up with data centers because more cooling is needed when you run the hardware harder. So: if you're running the models on your own machine, presumably you're not running them as often, and air cooling is sufficient. But, at the same time, this is less efficient in terms of hardware use; the data centers need water cooling specifically because they're getting more bang from their buck from their hardw…

that water is recirculated. nobody sends it down the drain after one loop

Evaporative cooling is cheaper

> [Google’s] largest data center, located in Council Bluffs, IA, withdrew an average of 3.9 million gallons of water and consumed 2.8 million gallons per day.

https://mostpolicyinitiative.org/science-note/data-center-wa...

Re: For most of the world, open-source AI is the only way forward

#138

Post by xtracto »

echelon wrote: post #4

We don't need rinky-dink RTX models that budget VRAM. We need large scale open weights models just as capable as what's at the frontier. And we need the ability to rent compute and spin up the weights easily. One-click, easy enough for anyone. Easier than nerd tools like ComfyUI, Claw, and node graph garbage. Freedom is owning very large scale weights. Anything less is subsistence.

we need this: https://news.ycombinator.com/item?id=48516751

> distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments.

Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.

Re: For most of the world, open-source AI is the only way forward

#139

Post by wbl »

grim_io wrote: post #113
mossTechnician wrote: post #100

> LeCun framed AI as an infrastructure-level platform that will soon mediate "all of our interaction with the digital world with information more generally" This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case. Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated…

This is already the reality for a large portion of the population. They(we) don't consume directly from the firehose of the internet, but from whatever "The Algorithm" has curated for them(us).

Ever since stumble upon.

Re: For most of the world, open-source AI is the only way forward

#140

Post by mncharity »

Chu4eeno wrote: post #11
pmontra wrote: post #2

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

Open weights/source doesn't necessarily mean running on local hardware, though. I imagine having multiple providers competing will drive down hosted versions of open weight models drastically.

> Open weights/source doesn't necessarily mean running on local hardware, though.

And we've barely started to scratch the surface on helping open-weight models "be the best they can be", with cloud burst parallel sampling and prompt mutation. Looking for best probabilistic results for a prompt, and looking for best prompt variants for a task. Adaptively scaling computes at generation, not just training.

And speculatively, if agentic coding is naturally a multiplicity, what UX might enable human devs to dance with that quantum superposition? Rather than quickly collapsing to one monkey and its keyboard.

Post reply on HN