Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI must win

opensourceaimustwin.com

Post reply on HN
Print view View original

481–490 of 538 posts

Re: Open source AI must win

#481

Post by boutell »

iugtmkbdfil834 wrote: post #477
sho wrote: post #197

Earlier quoted context omitted.

As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…

Dunno, in a sense, torrents came among similar restrictions. Everything at consumer level was just plain awful and at dial up level, mebbe ISDN if you were very lucky, with fiber only available to ridiculously rich people and corps. But with restrictions, came approaches on how to mitigate them.

If weights can't be looked at almost instantly in bulk, it just doesn't work. It's a different problem from distributing file downloads.

Re: Open source AI must win

#482

Post by aspenmartin »

c7b wrote: post #383
sho wrote: post #197

Earlier quoted context omitted.

As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…

Could you put some numbers and examples behind the efficiency gap between data center and consumer-grade AI hardware? Did you include examples like the RTX Spark on the consumer side? I was always amazed at the low power consumption of unified memory style architectures. In absolute terms and even more so compared to consumer-grade GPUs. I'd be genuinely interested in a comparison with data-center-grade hardware.

It's more than the raw hardware, it's the interconnect and communication between the hardware at scale. These models are trained on hundreds of thousands of GPUs today. You _will_ start to see cross-datacenter training runs but this needs to efficiently decide when and how to communicate across datacenter, which bears a very high cost compared to intra-datacenter communication.

Re: Open source AI must win

#483

Post by brandensilva »

bombcar wrote: post #466
rurban wrote: post #336

Earlier quoted context omitted.

You got it wrong. Inference can use crap GPU's. Training needs the 100x more expensive big guns. Our training machine is 100x more expensive than our inference machine.

How is the result of training stored? How big is that? It seems reasonable to assume we’ll eventually plateau and all we’ll need is relatively infrequent training.

I agree, leave the training to open source federations that roll out like operating systems. Minimal training over time.

Then have inference go down to the next layer to use those models as a P2P decentralized network.

Maybe like open router could tap federation networks.

Re: Open source AI must win

#484

Post by boutell »

Everybody who understands the technical problems is proposing a government fix. There is another option, foundations / NGOs could do this. Of course, openai has shown how quickly that can pivot into something completely different.

Re: Open source AI must win

#485

Post by ThejaCH »

yogthos wrote: post #327
ThejaCH wrote: post #320

I am really curious how long will it take for the open source models to hit current fable/mythos capabilities, KIMI 2.7 was launched recently and its quiet good for open source models its as good as Opus 4.6 maybe in practical applications not benchmarks so like 6 months to an year behind, after which the next step will be to wait for the day when we will be able to run mythos level intelligence on local hardware, Re…

There are two parts to this too. One is the raw model capability and the other is how well the harness guides the model and meets its expectations. I really think for stuff like agentic coding, this has to be treated as a package. This is my favorite example of how much difference a harness can make even for a tiny model https://github.com/itigges22/ATLAS And you're bang on with the storage comparison, we're basicall…

The repo is so cool!

Re: Open source AI must win

#486

Post by aspenmartin »

root-parent wrote: post #411
CuriouslyC wrote: post #392

Earlier quoted context omitted.

> It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do. 100% agree. The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization…

>> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it Currently AI has generated no profit. And as it sits, is a non viable business. I refuse to include the sellers of shovels as AI revenue. If the companies buying the shovels are still losing money, then the tool supplier fortunes have nothing to do with the economics of the AI application layer, who is losing mo…

It's the most naive opinion that keeps getting shoveled around. You have a product that is viewed as essential by businesses, with revenue growing by 10x a year and geopolitical ramifications that have continued to rear their heads and your opinion is "this is all an unprofitable shill". It is extraordinary to me that people really believe this. Whether or not labs run at a loss today is absolutely irrelevant. There is of course steady state economics that make sense, and its currently not well known what the profitability picture is right now, so to say "Currently AI has generated no profit" is also just speculation and not a very insightful one at that.

Re: Open source AI must win

#487

Post by aspenmartin »

CuriouslyC wrote: post #392
sho wrote: post #197

Earlier quoted context omitted.

As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…

> It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do. 100% agree. The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization…

> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization/global realignment is going to push us into a death spiral, and even if AI is a smash hit, the ~19% federal capture of corporate revenue isn't nearly enough to pull us out of it.

Any actual numbers to back this up? I don't see how nationalizing a very cutting edge technology outside of wartime is going to go super well. The leverage that these companies have is the same leverage that TSMC has: you can't just take over and expect things to rocket at the pace its going

Re: Open source AI must win

#488

Post by mythern »

xtracto wrote: post #391

This, and distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments. Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.

I propose the TokenTorrent protocol. Hoist yer pirate, erhm, clanker flags!

Re: Open source AI must win

#489

Post by dorfsmay »

jpalomaki wrote: post #301

Isn't training material the biggest problem for truly open source LLMs (such that could compete with top tier models)? The computation part can be solved with money, but compiling a comprehensive training set that could be freely shared and free of copyright issues is pretty much impossible.

Didn't the courts decide that if it's just for learning everything is fair game?

Re: Open source AI must win

#490

Post by nchmy »

boutell wrote: post #484

Everybody who understands the technical problems is proposing a government fix. There is another option, foundations / NGOs could do this. Of course, openai has shown how quickly that can pivot into something completely different.

IIRC, there once was a foundation created to create AI that is open for all. What happened to it?
Post reply on HN