i think to create or make opensource ai need competition power and alot of investment to create and use or use it local you need spec pc to run and tune it at minimum 27b model to act good on context and agent work
Open source AI must win
461–470 of 538 posts
Re: Open source AI must win
#462This, and distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments. Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.
[dead]
Re: Open source AI must win
#463I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…
> It would be better for governments to buy and own their own datacenters,
I mean thats good, but they'd have to also build thier own dataset. Which involves either paying people, or breaking the law.
Plus if they do manage to make it work, they will not get any tax revenue from it, as it'll remove the need for labour, which is where a huge amount of tax revenues come from.
its a deeply hard problem with lots of second/third order effects.
Re: Open source AI must win
#464This, and distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments. Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.
> The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it There is a middle way; the policy space also includes government regulating both access and monopoly. I’m opposed to monopolies of this tech, but I hope the risks of giving everyone jailbroken AGI/ASI are clear. As a toy example you could imagine a Universal Basic AI where government subcontracts to (n_quorum) labs, everyone g…
We have nothing anywhere near AGI/ASI so you're good for another 25 years, my friend
Re: Open source AI must win
#465I feel like this is similar to saying "open source cloud platforms must win". I'm not really sure what the concrete argument/proposal/strategy is here. Would open source AI be nice? Sure! Will the incentives of our capitalist economy change for this one specific product? Probably not!
Re: Open source AI must win
#466Earlier quoted context omitted.
AI hardware is for inference, not training. Training uses normal HPC crap. Superpods aren't really power efficient, it's kind of a meme, and it stems from limiting the power draw of other components by having less of them. It's more of a rounding error. > you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your ow…
You got it wrong. Inference can use crap GPU's. Training needs the 100x more expensive big guns. Our training machine is 100x more expensive than our inference machine.
How is the result of training stored? How big is that? It seems reasonable to assume we’ll eventually plateau and all we’ll need is relatively infrequent training.
Re: Open source AI must win
#467Earlier quoted context omitted.
>> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it Currently AI has generated no profit. And as it sits, is a non viable business. I refuse to include the sellers of shovels as AI revenue. If the companies buying the shovels are still losing money, then the tool supplier fortunes have nothing to do with the economics of the AI application layer, who is losing mo…
I've heard that the API calls by themselves are ~60% profit if you ignore capital expenditures. The labs haven't generated profit because they're constantly sinking money into the next generation of larger models to stay relevant. Dario has talked about the economics of this a lot, and I do believe him there. There's clearly also a lot of pent up demand in the corporate world for inference, the problem is that it's c…
The number of capital-heavy businesses that are wildly profitable “if you ignore capital expenses” is too many to list.
Airlines, for example, which are so profitable they continually go bankrupt.
Re: Open source AI must win
#468Earlier quoted context omitted.
Lab folks keep cards close to their chests here, but it's likely Mythos was an earlier teacher model for Opus that got additional cybersec post-training. Whether they have a bigger tier than that is hard to say, labs have been cautiously scaling parameters since the failure of GPT4.1. They 100% have a bigger/better model they haven't released, but that's probably more down to it not being done cooking yet. Once it's…
Appreciate the long answer. Why is it more likely that Gemini 3 Pro/Flash/Lite are distillations of the same parent model than that they’re different training runs on the same dataset, with minor version bumps being different post-training setups?
The biggest tell is the fact that labs are staggering smaller model releases so much with big models. If the small models (flash, sonnet/haiku) were being distilled from pro models, you'd consistently see them be released fairly soon after new pro releases to maximize their competitiveness (and this was the case early on for Anthropic). Instead it seems like releases are timed to build/maintain hype.
A thing to keep in mind is that if they release a smaller model halfway between well spaced big model releases, why wait so long on the next big model release if it's sufficiently ready to distill to a smaller model? The ability to demonstrate AI superiority is worth a ton, there's no reason to hold back.
Re: Open source AI must win
#469For open source AI, AI infrastructure must be public just like roads, rail, ...
Re: Open source AI must win
#470Earlier quoted context omitted.
Tbh, there really needs to be some legal precedent set that makes model distillation a legal activity. If the model makers can rip everyone else's work and launder information as if it's their own without giving credit back to the original creators, I don't see why it should be illegal to distill the models. It's the same thing the frontier model makers are doing to IP everywhere else.
I agree. But this won't happen in the US because Anthropic / OpenAI is a big ol economic recession risk because we levered ourselves to the tits and put our chips on them.
Explain how an AI bust would tank the economy? They don't employ enough to feel the hit from that.
They're not even IPOed so how do they tank the market? GPU and ram prices will go down but that will actually help most tech companies.
I don't think the rest of the economy is inflated on the fantasy gains of AI.
We could actually go back to feeling like we can invest in products and content without FOMO.