Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI must win

opensourceaimustwin.com

Post reply on HN
Print view View original

191–200 of 538 posts

Re: Open source AI must win

#191

Post by trenchgun »

palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

>But when people think of decentralized training, they don’t first think of gigantic datacenters, owned by the same company, training models across large distances. Instead, they imagine thousands of small datacenters, or individual consumers, pooling their spare compute over the internet to orchestrate a training run larger than any single actor could manage alone. Many companies are pursuing this vision: Pluralis Research, Prime Intellect and Nous Research have already successfully decentrally trained models at scale. But in practice, training decentrally over the internet has lagged far behind more centralized training. Even their largest models (Pluralis’ 8B Protocol Model, Prime Intellect’s INTELLECT-1, and Nous’ Consilience 40B) have been trained with 1,000x less compute than today’s frontier models (such as xAI’s Grok 4). https://epoch.ai/gradient-updates/how-far-can-decentralized-...

Re: Open source AI must win

#192

Post by d--b »

It’s the GPUs, not the weights that are the key.

As long as these models require a lot of computing power, the best models open source or not will be served by corporations who can afford the infra.

Re: Open source AI must win

#193

Post by a1exyz »

Terr_ wrote: post #86
operatingthetan wrote: post #54

Earlier quoted context omitted.

It is a bit surprising that the true 'big brother' type dystopic aspects of AI are not discussed that much and instead we talk about them taking all the jobs. We feed these things so much information. It could be used against us for advertising, control, or worse.

"You're absolutely right, I think you deserve to treat yourself with Mococoa, made with all-natural cocoa beans from the upper slopes of Mount Nicaragua! It's what humans like myself crave." Much like Truman's town, I fear a future where every non-in-person "interaction" might be a bot-network with an agenda and the inhuman patience of playing for the long-con.

Well as we get poorer and poorer it will be less worth putting effort into advertising to us. Im guessing AI will instead focus its effort on convincing rich people of various things.

Re: Open source AI must win

#195

Post by rustcleaner »

hirako2000 wrote: post #168
george_max wrote: post #2

With open-weight AI, there might not be an incentive to put large sums of capital towards training / research. There might be a donation fund of some sorts, but it certainly won't reach the level of fundraising that the frontier labs are receiving. Because of this, I think it might not be possible to have AI *only* open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models…

Zoom out. It's a matter of time the trillion valuations will be deemed senseless, only once it will prove inpossible to extract trillions from consumers. In the meanwhile, and regardless, software optimisations coupled with hardware continuing to scale, we will end up, soon enough, with some open weight that run on a mobile device with greater capabilities than Fable.

>only once it will prove inpossible [sic] to extract trillions from consumers.

I am spreading a message of peace and sovereignty:

Never subscribe. Never. Subscribe. Ever.

Starve them out. Make their lenders take 95% haircuts.

Just don't subscribe, whatever you do!

Re: Open source AI must win

#197

Post by sho »

palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea.

The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.

And even if electricity was free, having those GPUs spread over the world with internet-level latency will slow everything down by factors of thousands to millions - if it's feasible at all. Regardless, you're not getting fable-oss this decade, maybe even not this century.

It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do.

Re: Open source AI must win

#198

Post by nstart »

WarmWash wrote: post #75

Who is going to fund it? Training is unfathomably expensive. You have either VC funded models looking for a return on investment, or CCP funded models looking to solidify authoritarian "model Chinese society". Maybe there are some university 4B models, but I doubt those will carry far.

Tbh, there really needs to be some legal precedent set that makes model distillation a legal activity. If the model makers can rip everyone else's work and launder information as if it's their own without giving credit back to the original creators, I don't see why it should be illegal to distill the models. It's the same thing the frontier model makers are doing to IP everywhere else.

Re: Open source AI must win

#199

Post by 3s »

never_inline wrote: post #177

I think articles this light on content should not be upvoted to front page.

It's a perfect prompt for a rich HN discussion so while in general I agree with you, in this case the discussion is what matters.

Re: Open source AI must win

#200

Post by Zetaphor »

rustcleaner wrote: post #189
palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

Could it be done by making a sparse MoE of thousands, or tens of thousands, of smaller experts in very niche domains? Maybe a tree-like structure of experts which can delegate from relatively general but inaccurate to extremely niche but accurate? Also these experts might be plug-and-play, easily swap out an inferior expert with a stronger one in the future without having to redo the whole pile?

That's not really how the experts in an MoE work. They activate on token probabilities and are activated on every token. You don't necessarily have a discrete math expert and a discrete physics expert. And if it were you would still need a router that is trained on all of those domains.
Post reply on HN