Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI must win

opensourceaimustwin.com

Post reply on HN
Print view View original

211–220 of 538 posts

Re: Open source AI must win

#211

Post by mewpmewp2 »

nstart wrote: post #198
WarmWash wrote: post #75

Who is going to fund it? Training is unfathomably expensive. You have either VC funded models looking for a return on investment, or CCP funded models looking to solidify authoritarian "model Chinese society". Maybe there are some university 4B models, but I doubt those will carry far.

Tbh, there really needs to be some legal precedent set that makes model distillation a legal activity. If the model makers can rip everyone else's work and launder information as if it's their own without giving credit back to the original creators, I don't see why it should be illegal to distill the models. It's the same thing the frontier model makers are doing to IP everywhere else.

And which leading country is going to go for allowing other countries to distill their models?

Re: Open source AI must win

#212

Post by rustcleaner »

thewebguyd wrote: post #34
em-bee wrote: post #20

Earlier quoted context omitted.

yes, but a model that runs on my own machine will never have the capacity of a model that runs in a datacenter. as i said, it can't compete with that.

If RAM prices ever come down, you can have a machine that can run a capable local model. Qwen 2.5 72B is surprisingly capable, almost on par with GPT-4o if not a little better. You can run it on a 128GB Mac Studio with 8-bit quantization. You need about 77GB for the weights and ~15GB for your context window & cache. Pricing remains to be seen, but there's also those new nvidia laptops coming out the surface laptop ul…

>If RAM prices ever come down

Maybe an unpopular opinion here (seening how Y-combinator is his baby), but I think OpenAI and Sam Altman should be financially decimated for cornering the DRAM market. What he's done is a step or two removed from what the Hunt brothers did. His buy-up of future DRAM silicon has measurably harmed personal computing, and he should not get to walk away with a 'win' from it.

Re: Open source AI must win

#213

Post by ux266478 »

sho wrote: post #197
palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…

AI hardware is for inference, not training. Training uses normal HPC crap. Superpods aren't really power efficient, it's kind of a meme, and it stems from limiting the power draw of other components by having less of them. It's more of a rounding error.

> you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.

Costs spread over a large population, it really doesn't matter. You're not getting hundreds of thousands of people to pitch half their monthly electric bill to pay for someone else's datacenter. They will pay the electricity themselves quite happily though, if all they need to do is give you compute. This isn't new.

Interconnect is the bottleneck for distributed training, nothing else really.

Re: Open source AI must win

#214

Post by labbett »

palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

Sounds like SETI@home but for AGI... SAGI@home?

Re: Open source AI must win

#215

Post by whiplash451 »

palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

This could be of interest to you: https://thealliance.ai/projects/tapestry

Re: Open source AI must win

#216

Post by rustcleaner »

google234123 wrote: post #173
bluejay2387 wrote: post #162

In the US -- once our nation finishes attacking our own education system -- this is definitely something a group of academic institutions could get together and accomplish. I assume the same is true in other countries. Companies like Nvidia and AMD might even support that effort, as they make money on the hardware and would probably be more than happy for there to be more reasons to use it. There may have not been a…

Wait, which side is blocking kids fork taking algebra or forcing universities to admit people that can't do math or read, or abandoning phonetics for unproven methods that don't work?

Both sides, since they are bought and paid for by the finance industrial complex.

Re: Open source AI must win

#217

Post by digitaltrees »

rustcleaner wrote: post #185
digitaltrees wrote: post #163

Earlier quoted context omitted.

1. Your assumption that a growing population is the metric of success is questionable. A population that grows but is subject to famine, epidemics, and natural disasters because they haven’t developed the scientific and technological capacity to escape the existential risks of the physical world is living on borrowed time. Not saying I agree with that, and I would actually agree that there is merit to the Amish hypot…

>Your assumption that a growing population is the metric of success is questionable. It's a better measure than GDP/S&P/401(k) line-go-up especially [re: America] when the native Euro-based population has been aging and dropping for decades, once you strip away all the post Hart-Cellar immigrant lineages.

What are hart-cellar immigrant lineages? And why is that in anyway relevant?

Let’s play a thought experiment.

Let’s say we have a million people that are so technically sophisticated that they are a space faring civilization capable of seeding the universe with living ecosystems capable of perpetuating life and evolutionary processes. But they are entirely infertile and will never give birth to another individual of their species.

And we have another population that doubles every single year but is incapable of leaving their home planet.

Which one is more valuable?

It depends on what your measure of value is, but if it is to maximize the amount of life in the universe, then population growth is not the right metric, expansion of life through technological means is the more appropriate metric.

Re: Open source AI must win

#218

Post by zozbot234 »

george_max wrote: post #2

With open-weight AI, there might not be an incentive to put large sums of capital towards training / research. There might be a donation fund of some sorts, but it certainly won't reach the level of fundraising that the frontier labs are receiving. Because of this, I think it might not be possible to have AI *only* open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models…

> Because of this, I think it might not be possible to have AI only open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models than open-source versions.

There's a more fundamental reason for this: some AI models are large enough that they can plausibly only be reasonably run in a state-of-the-art hyperscale datacenter. Open sourcing such models would be largely pointless. Note that this would be a significantly larger scale than even the largest open models available today, one that precludes even doing inference slowly on a small-scale, cheap makeshift cluster. But it's plausible that Fable is there already.

Re: Open source AI must win

#219

Post by cpdomina »

palisade wrote: post #44

I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…

there was a project trying to achieve some of those goals a few years ago using p2p: petals https://github.com/bigscience-workshop/petals

their bloom model was also a collaborative effort https://huggingface.co/docs/transformers/en/model_doc/bloom

Re: Open source AI must win

#220

Post by ux266478 »

abhinavsharma wrote: post #69

Open-source AI can, by definition, never "win". AI is just hillclimbing today, and closed labs can always absorb everything the open world does and build upon it. It doesn't really matter for most use cases, because the way AI is working is capability saturation. https://www.delanceyukschoolschesschallenge.com/the-rising-t... The only exception to this is fields that are inherently adversarial (to nature or others) a…

AllegroLisp is very far behind SBCL.
Post reply on HN