Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

671–680 of 936 posts

Re: Open source AI is the path forward

#671

Post by anon373839 »

tempfile wrote: post #224
enriquto wrote: post #176

Earlier quoted context omitted.

> The training data contains most likely insane amounts of copyrighted material. If that is the case then the weights must inherit all these copyrights. It has been shown (at least in image processing) that you can extract many training images from the weights, almost verbatim. Hiding the training data does not solve this issue. But regardless of copyright issues, people here are complaining about the malicious use o…

> If that is the case then the weights must inherit all these copyrights. Not if it's a fair use (which is obviously the defence they're hoping for)

Also, fair use is just one defense to a copyright infringement claim. The plaintiff first has to prove the elements of infringement; if they can't do this, no defense is needed.

Re: Open source AI is the path forward

#672

Post by frabcus »

suyash wrote: post #344

Open source is a welcome step but what we really need is complete decentralisation so people can run their own private AI Models that keep all the data private to them. We need this to happen locally on laptops, mobile phones, smart devices etc. Waiting for when that will become ubiquitous.

It is open weights not open source. If you can't train it and don't know the training data and can't use it to train your own models, it is a closed model aa a whole. Even if you have the binary weights.

Re: Open source AI is the path forward

#673

Post by frabcus »

zmmmmm wrote: post #559
mav3ri3k wrote: post #114

I am not deep into llms so I ask this. From my understanding, their last model was open source but it was in a way that you can use them but the inner working were "hidden"/not transparent. With the new model, I am seeing alot of how open source they are and can be build upon. Is it now completely open source or similar to their last models ?

they give you the code and they give you the model it runs, and you can customise and redistribute both. It's all open source in that respect. What people are complaining about (totally unreasonably in my view) is obviously Meta is not "open sourcing" all the training data, so nobody can retrain the model from scratch themselves. This argument to me is just silly. The whole point of these models is they distil pretra…

They could release the code that gathers and curates the data. Give a reproducible system for getting the pre training data. And presumably they own the post training RLHF stuff so could open that.

Without those you're locked in to them in terms of licensing of future versions.

Re: Open source AI is the path forward

#674

Post by jstummbillig »

anon373839 wrote: post #665
jstummbillig wrote: post #658

Earlier quoted context omitted.

What Meta is doing is borderline market distortion. It's not that they have figured out some magic sauce they are happy to share. They are just deciding to burn brute force money that they made elsewhere and give their stuff away below cost, first of all because they can.

I know, and it's beautiful to see. Bad actors like "Open"AI tried to get in first and monopolize this tech with lawfare. But that game plan has been mooted by Meta's scorched-earth generosity.

Meta has actually figured out where the moot is: Ecosystem, tooling. As soon as "we" build it, they an still do whatever they want with the core/llm, starting with Llama 4 or any other point in the future.

The best kind of open source: All the important ingredients to make it work (more and more data and money) are either not open source or in the hands of Meta. It's prohibitive by design.

People seem happy to help build Metas empire once again in return for scraps.

Re: Open source AI is the path forward

#675

Post by rizky05 »

nightski wrote: post #604
stavros wrote: post #573

Earlier quoted context omitted.

That's not the discussion. We're talking about what open source is, and it's having the weights and the method to recreate the model. If someone gives me an executable that I can run for free, and then says "eh why do you want the source, it would take you a long time to compile", that doesn't make it open source, it just makes it gratis.

Calling weights an executable is disingenuous and not a serious discussion. You can do a lot more with weights than you could with a binary executable.

This is debatable, even an executable is valuable artifact. You can also do a lot with executable in expert hand.

Re: Open source AI is the path forward

#676

Post by frabcus »

jdminhbg wrote: post #197
thenoblesunfish wrote: post #182

Earlier quoted context omitted.

You don't need to have the commit history to see "how it works". ML that works well does so in huge part due to the training data used. The leading models today aren't distinguished by the way they're trained, but what they're trained on.

I agree that you need training data to build AI from scratch, much like you need lots of really smart developers and a mailing list and servers and stuff to build the Linux kernel from scratch. But it's not like having the training data and training code will get you the same result, in the way something like open data in science is about replicating results.

Reproducible builds of software binaries are a thing, but they aren't routinely done. Likewise training an AI is deterministic if you do it the same each time. And slight variances lead to similar capability models.

Re: Open source AI is the path forward

#677

Post by frabcus »

lolinder wrote: post #94
mvkel wrote: post #46

It's a real shame that we're still calling Llama "open source" when at best it's "open weights." Not that anyone would go buy 100,000 H100s to train their own Llama, but words matter. Definitions matter.

Source versus weights seems like a really pedantic distinction to make. As you say, the training code and training data would be worthless to anyone who doesn't have compute on the level that Meta does. Arguably, the weights are source code interpreted by an inference engine, and realistically it's the weights that someone is going to want to modify through fine-tuning, not the original training code and data. The fa…

Meta could change the license of future releases of Llama and kill your business built on it.

If the training data was openly available, even if you can't afford to res train a new version, a competitor like Amazon could do it for you

Re: Open source AI is the path forward

#678

Post by frabcus »

sidcool wrote: post #626
paulhilbert wrote: post #108

Earlier quoted context omitted.

No, I would argue that from the three main ingredients - training data, model source code and weights - weights are the furthest away from something akin to source code. They're more like obfuscated binaries. When it comes to fine-tuning only however things shift a little bit, yes.

I don't expect them to release the data used to train the models. But I agree that the code is an important ingredient of 'open'.

Must include the code that curates the data

Re: Open source AI is the path forward

#679

Post by Havoc »

boringg wrote: post #592
Havoc wrote: post #542

Earlier quoted context omitted.

>locked into their system There are going to be loads of providers for these open models. Openrouter already has 3 providers for the new 405B model within hours.

Maybe for the time being. I don't see how else they monetize the incredible amount the spent on the models without forcing people to lock into models or benefits or something else. It's not going to stay like this I can assure you that :).

Not sure whether you mean by that post open router serving the 405b or meta producing more.

Open router is a paid api so that can absolutely be sustainable.

And meta has multiple reasons for going open route - some explained in their posts so less so (harms their competitors)

I reckon there will be a llama 4 and beyond

Re: Open source AI is the path forward

#680

Post by msnkarthik »

Interesting discussion! While I agree with Zuckerberg's vision, the comments raise valid concerns. The point about GPU accessibility and cost is crucial. Public clusters are great, but sustainable funding and equitable access are essential to avoid exacerbating existing inequalities. I also resonate with the call for CUDA alternatives. Breaking the dependence on proprietary technology is key for a truly open AI ecosystem. While existing research clusters offer some access, their scope and resources often pale in comparison to what companies like Meta are proposing. We need a multi-pronged approach: open-sourcing models AND investing in accessible infrastructure, diverse hardware options, and sustainable funding models for a truly democratic AI future.
Post reply on HN