Agreed.
If you can't reproduce the model then it's not open-source
51–60 of 117 posts
Re: If you can't reproduce the model then it's not open-source
#52Are there any true open-source LLM models, where all the training data is publicly-available (with a compatible license) and the training software can reproduce bit-identical models?
Is training nondeterministic? I know LLM outputs are purposely nondeterministic.
Re: If you can't reproduce the model then it's not open-source
#53I don't agree, and the analogy is poor. One can do the things he lists with a trained model. Having the data is basically a red herring. I wish this got more attention. Open/free software is about exercising freedoms, and they all can be exercised if you've got the model weights and code. https://www.marble.onl/posts/considerations_for_copyrighting...
But one of the four freedoms is being able to modify/tweek things, including the model. If all you have is the model weights, then you can't easily tweak the model. The model weights is hardly the preferred form for making changes to update the model. The equivalent would be someone which gives you only the binary to Libreoffice. That's perfectly fine for editing documents and spreadsheets, but suppose you want to fi…
You can fine tune a model, you ve got way more power to do so given the trained model than starting from scratch and the raw data.
Re: If you can't reproduce the model then it's not open-source
#54Earlier quoted context omitted.
Open Source and Free Software wasn't formulated to deal with the need for this level of gargantuan amounts of data and compute. Can the public compete? What percentage of the technical public could we expect to participate, and how much data, compute, and data quality improvement could they bring to the table? I suspect that large corporations are at least an order of magnitude advantaged economically.
The public doesn't have the resources to train the largest state-of-the-art LLMs, but training useful LLMs seems doable. Maybe not for most individuals but certainly for a range of nonprofits, research teams and companies.
Isn't is relatively easy for a smaller model to poke holes in the output of a larger model?
Re: If you can't reproduce the model then it's not open-source
#55I don't agree, and the analogy is poor. One can do the things he lists with a trained model. Having the data is basically a red herring. I wish this got more attention. Open/free software is about exercising freedoms, and they all can be exercised if you've got the model weights and code. https://www.marble.onl/posts/considerations_for_copyrighting...
But one of the four freedoms is being able to modify/tweek things, including the model. If all you have is the model weights, then you can't easily tweak the model. The model weights is hardly the preferred form for making changes to update the model. The equivalent would be someone which gives you only the binary to Libreoffice. That's perfectly fine for editing documents and spreadsheets, but suppose you want to fi…
If all you have are the model weights, you can very easily tweak the model. How else are all these "decensored" Llama2 showing up on Hugging Face? There's a lot of value in a trained LLM model itself and it's 100% a type of openness to release these trained models.
What you can't easily do is retrain from scratch using a heavily modified architecture or different training data preconditioning. So yes, it is valuable to have dataset access and compute to do this and this is the primary type of value for LLM providers. It would be great if this were more open — it would also be great if everybody had a million dollars.
I think it's pretty misguided to put down the first type of value and openness when honestly they're pretty independent, and the second type of value and openness is hard for anybody without millions of dollars to access.
Re: If you can't reproduce the model then it's not open-source
#56Earlier quoted context omitted.
> If a LLM knows a song as part of its training data, then it is copyright infringement. No it isn't. You can feed whatever you want into your LLM, including copyrighted data. The issues arise when you start reproducing or distributing copyrighted content.
>You can feed whatever you want into your LLM, including copyrighted data. That's currently the subject of considerable legal debate. https://edition.cnn.com/2023/07/10/tech/sarah-silverman-open...
That is mostly an issue of the latter, whether the service that Meta/OpenAI offers outputs content that is a violation of copyright. Technically, derivative works are a copyright violation, but if you're not distributing them, you normally have a good fair use argument, and/or nobody knows.
Re: If you can't reproduce the model then it's not open-source
#57> Imagine if Linux published only a binary without the codebase. Or published the codebase without the compiler used to make the binary. This is where we are today. This was such a helpful way to frame the problem! Something felt off about the "open source models" out there; this highlights the problem incredibly well.
The GPL describes the source as the "preferred form for modification".
And, that's obviously fun, because with LLMs, you have the LLM itself which cost hundreds of thousands in compute to train, but given you have the weights it's eminently fine-tunable. So it's actually not really like Linux - rather it's closer to something like a car, where you had no hope of making it in the first place but now you have it, maybe you can modify it.
Re: If you can't reproduce the model then it's not open-source
#58> Imagine if Linux published only a binary without the codebase. Or published the codebase without the compiler used to make the binary. This is where we are today. This was such a helpful way to frame the problem! Something felt off about the "open source models" out there; this highlights the problem incredibly well.
I think a better analogy is firmware binary blobs in the Linux kernel, or VM bytecodes. The LLM inference engine (architecture implementation) is like a kernel driver that loads a firmware binary blob, or a virtual machine that loads bytecode. The inference engine is open source. The problem is that the weights (firmware blobs, VM bytecodes) are opaque: you don't have the means to reproduce them. The Linux community…
If we're continuing the analogy, the compute required to turn the source into binaries costs millions of dollars. Not a license fee for the compiler, but the actual time on a computer.
Re: If you can't reproduce the model then it's not open-source
#59> Imagine if Linux published only a binary without the codebase. Or published the codebase without the compiler used to make the binary. This is where we are today. This was such a helpful way to frame the problem! Something felt off about the "open source models" out there; this highlights the problem incredibly well.
Open Source and Free Software wasn't formulated to deal with the need for this level of gargantuan amounts of data and compute. Can the public compete? What percentage of the technical public could we expect to participate, and how much data, compute, and data quality improvement could they bring to the table? I suspect that large corporations are at least an order of magnitude advantaged economically.
Open Source and Free Software are not about the amount of data.
Re: If you can't reproduce the model then it's not open-source
#60I think the process of data acquisition isn't so clear-cut. Take CERN as an example: they release loads of data from various experiments under the CC0 license [1]. This isn't just a few small datasets for classroom use; we're talking big-league data, like the entire first run data from LHCb [2]. On their portal, they don't just dump the data and leave you to it. They've got guides on analysis and the necessary tools…
You're right that most people have neither the need nor the ability to recreate the data themselves. But the same applies to using open-source software in the first place: most people who use OSS have neither the need nor the ability to compile the software from source themselves. But the whole point of OSS is that that source is available for those who want to use it, whether to study it, to diagnose a bug, or something else. I think the same is true for the LHC's technical details or a model's training data: most people won't recreate it at home, but it's important to make it available, and even someone who can't rebuild the whole thing themselves might spot an important bug or omission by going through the data collection details.