41–50 of 117 posts
https://www.marble.onl/posts/considerations_for_copyrighting...
> Imagine if Linux published only a binary without the codebase. Or published the codebase without the compiler used to make the binary. This is where we are today. This was such a helpful way to frame the problem! Something felt off about the "open source models" out there; this highlights the problem incredibly well.
Open Source and Free Software wasn't formulated to deal with the need for this level of gargantuan amounts of data and compute. Can the public compete? What percentage of the technical public could we expect to participate, and how much data, compute, and data quality improvement could they bring to the table? I suspect that large corporations are at least an order of magnitude advantaged economically.
I don't agree, and the analogy is poor. One can do the things he lists with a trained model. Having the data is basically a red herring. I wish this got more attention. Open/free software is about exercising freedoms, and they all can be exercised if you've got the model weights and code. https://www.marble.onl/posts/considerations_for_copyrighting...
The equivalent would be someone which gives you only the binary to Libreoffice. That's perfectly fine for editing documents and spreadsheets, but suppose you want to fix a bug in Libreoffice? Just having the binary is going to make it quite difficult to fix things.
Simiarly, suppose you find that the model has a bias in terms of labeling African Americans as criminals; or women as lousy computer programmers. If all you have is the model weights of the trained model, how easily can you fix the model? And how does that compare with running emacs on the Libreoffice binary?
Earlier quoted context omitted.
The standards applied are exactly the same regardless of what tools are used. It doesn't matter if you're talking about a dumb AI, a general purpose AI, or a Xerox machine. If you want an exception to copyright, you're going to want to start looking at a section 107 (of the copyright act) exception: https://www.copyright.gov/title17/92chap1.html#107 The reason someone walking down the street and humming a song is not…
I think this conversion has corrected some misgivings I had about the AI copyright argument. My takeaway is; Possession copyright material is not inherently infringing on a copyright. Disseminating copyright material is unless you meet section 107. AI runs afoul of section 107 when it verbatim shares copyright material from its dataset without attribution.
Technically, the AI doesn't run afoul. The person disseminating the copyrighted material does.
Same with open-core - if you can't self-host the thing on your own infra then its not REALLY OSS
I don't agree, and the analogy is poor. One can do the things he lists with a trained model. Having the data is basically a red herring. I wish this got more attention. Open/free software is about exercising freedoms, and they all can be exercised if you've got the model weights and code. https://www.marble.onl/posts/considerations_for_copyrighting...
But one of the four freedoms is being able to modify/tweek things, including the model. If all you have is the model weights, then you can't easily tweak the model. The model weights is hardly the preferred form for making changes to update the model. The equivalent would be someone which gives you only the binary to Libreoffice. That's perfectly fine for editing documents and spreadsheets, but suppose you want to fi…
That's textbook fine-tuning and is basically trivial. Adding another layer and training that is many orders of magnitude more efficient than retraining the whole model and works ~exactly as well.
Models are data, not instructions. Analogies to software are actively harmful. We do not fix bugs in models any more than we fix bugs in a JPEG.
Earlier quoted context omitted.
Somewhat unrelated, but here is a thought experiment... If a human knows a song "by heart" (imperfectly), it is not considered copyright infringement. If a LLM knows a song as part of its training data, then it is copyright infringement. But what if you developed a model with no prepared training data and forced it to learn from it's own sensory inputs. Instead of shoveling it bits, you played it this particular song…
> If a LLM knows a song as part of its training data, then it is copyright infringement. No it isn't. You can feed whatever you want into your LLM, including copyrighted data. The issues arise when you start reproducing or distributing copyrighted content.
That's currently the subject of considerable legal debate.
https://edition.cnn.com/2023/07/10/tech/sarah-silverman-open...
Earlier quoted context omitted.
> If a LLM knows a song as part of its training data, then it is copyright infringement. No it isn't. You can feed whatever you want into your LLM, including copyrighted data. The issues arise when you start reproducing or distributing copyrighted content.
>You can feed whatever you want into your LLM, including copyrighted data. That's currently the subject of considerable legal debate. https://edition.cnn.com/2023/07/10/tech/sarah-silverman-open...
Same with open-core - if you can't self-host the thing on your own infra then its not REALLY OSS
Many companies are using "open source" as marketing rather then actually releasing open source software and models. No data? Not open source. Special license cutting out self-hosting or competitive use? Not open source.
I don't agree, and the analogy is poor. One can do the things he lists with a trained model. Having the data is basically a red herring. I wish this got more attention. Open/free software is about exercising freedoms, and they all can be exercised if you've got the model weights and code. https://www.marble.onl/posts/considerations_for_copyrighting...
But one of the four freedoms is being able to modify/tweek things, including the model. If all you have is the model weights, then you can't easily tweak the model. The model weights is hardly the preferred form for making changes to update the model. The equivalent would be someone which gives you only the binary to Libreoffice. That's perfectly fine for editing documents and spreadsheets, but suppose you want to fi…
In other words there should be a reasonable line when model is called open source. In extreme view it's when the model, the training framework, and the data are available for free. This would mean open source model can be trained only on public domain data. Which makes class of open source models very, very limited.
More realistic is to make the code and the weights available. So that with some common knowledge new model can be trained, or old fine tuned, on available data. Important note: weights cannot be reproduced even if original training data is available. It will be always a new model with (slightly) different responses.