Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI must win

opensourceaimustwin.com

Post reply on HN
Print view View original

411–420 of 538 posts

Re: Open source AI must win

#411

Post by root-parent »

CuriouslyC wrote: post #392
sho wrote: post #197

Earlier quoted context omitted.

As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…

> It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do. 100% agree. The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization…

>> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it

Currently AI has generated no profit. And as it sits, is a non viable business.

I refuse to include the sellers of shovels as AI revenue.

If the companies buying the shovels are still losing money, then the tool supplier fortunes have nothing to do with the economics of the AI application layer, who is losing money on every prompt.

Re: Open source AI must win

#412

Post by gordonhart »

CuriouslyC wrote: post #404
Folcon wrote: post #333

A question I've got which I've been wondering about, not sure if anyone else has been thinking about it, what actually made Fable so effective? From what I could tell from the very little time that I had to interact with it, it's instruction following seemed more consistent The other thing that comes to mind is a lot of people commented on how driven it was, so I'm wondering whether figuring out how to keep existing…

Fable had mostly the same pre-training data as Opus, and it's likely they're distilled from the same source. The difference is that it's a larger model with more post training on "dangerous" stuff they didn't want in the core model, and "long" task RL.

> it's likely they're distilled from the same source

Any credible references for this? The implication that Anthropic has an even bigger and better model that they haven't released is hard to believe.

Re: Open source AI must win

#413

Post by [deleted] »

WithinReason wrote: post #265
palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

The gradient info can be compressed 10000x with the right tricks, I think it is achievable. Nous claims they did it already: https://github.com/NousResearch/DisTrO There are other gradient compression papers from the past reporting large compression rates

[deleted]

Re: Open source AI must win

#414

Post by CuriouslyC »

root-parent wrote: post #411
CuriouslyC wrote: post #392

Earlier quoted context omitted.

> It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do. 100% agree. The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization…

>> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it Currently AI has generated no profit. And as it sits, is a non viable business. I refuse to include the sellers of shovels as AI revenue. If the companies buying the shovels are still losing money, then the tool supplier fortunes have nothing to do with the economics of the AI application layer, who is losing mo…

I've heard that the API calls by themselves are ~60% profit if you ignore capital expenditures. The labs haven't generated profit because they're constantly sinking money into the next generation of larger models to stay relevant. Dario has talked about the economics of this a lot, and I do believe him there.

There's clearly also a lot of pent up demand in the corporate world for inference, the problem is that it's currently expensive enough that enterprises are balking at the cost before they've had a chance to refine processes and see projects through to fruition. That's a tractable problem to solve though.

Re: Open source AI must win

#416

Post by addandsubtract »

raincole wrote: post #350
sanbor wrote: post #349

I would be totally willing to pay $50 per month to support an open source AI lab, rather to get open source models as byproducts of corporations.

I think OpenAI ruined the trust. How do you know this 'open source AI lab' won't somehow spin off a for-profit?

Unpopular opinion: a government funded and run lab, available to the public. The EU could make it happen.

Re: Open source AI must win

#417

Post by fullshark »

raincole wrote: post #350
sanbor wrote: post #349

I would be totally willing to pay $50 per month to support an open source AI lab, rather to get open source models as byproducts of corporations.

I think OpenAI ruined the trust. How do you know this 'open source AI lab' won't somehow spin off a for-profit?

Because they are supported by the Chinese government for the purpose of undermining US capital investments.

Re: Open source AI must win

#419

Post by CuriouslyC »

gordonhart wrote: post #412
CuriouslyC wrote: post #404

Earlier quoted context omitted.

Fable had mostly the same pre-training data as Opus, and it's likely they're distilled from the same source. The difference is that it's a larger model with more post training on "dangerous" stuff they didn't want in the core model, and "long" task RL.

> it's likely they're distilled from the same source Any credible references for this? The implication that Anthropic has an even bigger and better model that they haven't released is hard to believe.

Lab folks keep cards close to their chests here, but it's likely Mythos was an earlier teacher model for Opus that got additional cybersec post-training. Whether they have a bigger tier than that is hard to say, labs have been cautiously scaling parameters since the failure of GPT4.1. They 100% have a bigger/better model they haven't released, but that's probably more down to it not being done cooking yet. Once it's done, the single larger model lets them drop new Opus and Mythos iterations in rapid succession.

Googlers have hinted that Gemini 3 came in at 10T, which seems hard to operationalize, Google's flash and pro releases are staggered in a way that doesn't make sense if flash is a pro distill, and there are enough cases where Gemini flash outperforms pro on the same task that I think it's unlikely it's just being distilled from an "in progress" version of pro.

Re: Open source AI must win

#420

Post by pizzao »

xtracto wrote: post #391

This, and distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments. Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.

I wonder if there is way local small LLMs can complement each other in away that the sum-total yields a much more performant LLM
Post reply on HN