> Does the attribution requirement mean that the author’s information may not be removed as a data element from the content, even if inclusion might frustrate the TDM exercise or introduce noise into the system?
> Does the attribution need to be included in the data set at every stage?
The above two questions seem identical. My gut feeling is no; you don't need to intentionally train on attributions nor do you need to ensure that the data set columns have attribution data in them when provided to your training code. The attribution requirement of CC-BY and CC-BY-SA triggers when you do any of the things copyright law says you have to get permission in order to do, and the license further restricts that requirement to public instances of such. So privately shoving Creative Commons data into a neural net trainer is probably fine. Not having attribution on the data the model sees does not foreclose the possibility of providing attribution alongside the model at the time of publication.
> Does the result of the mining need to include attribution, even if hundreds of thousands of CC BY works were mined and the output does not include content from individual works?
This is an active legal question.
My personal opinion is that if you can draw a line from a particular output to something in the training set, then you're either copying or creating a derivative work, and you need to follow any relevant licenses. Generative models are capable of outputting their training set data, especially if overfit; so using them exposes you to the licensing requirements of anything the model saw that matches its output domain.
I'm actually considering this as part of PD-Diffusion; which is my attempt at building an art generator trained on public-domain images. Since I'm scraping Wikimedia Commons to get both images and labels, the labels are CC-BY-SA[0]. This means that my trained models will also need to be CC-BY-SA and ship with a very, very long text file listing attributions for all the labels I used[1]. But, notably, because this is an art generator and not a label generator, I don't need to worry about attributing model outputs. None of the label data will make its way into the final image.
If I did have a reliable way to attribute model outputs, then I could make "CC-Diffusion", trained on all CC-BY and CC-BY-SA images on Wikimedia Commons. But even then there's noncopyright ethical concerns with doing that. I'm not even using the full public domain as-is, just the PD-Old category, because Wikimedia Commons has a lot of uncopyrighted Italian images of living people that I do not want in my model.
> Also sued were a confusing mishmash of for profit and non-profit related entities all using a variation of the name OpenAI (OpenAI, Inc., OpenAI, LLC, OpenAI Startup Fund GP I, L.L.C.; you get the picture). OpenAI received one billion dollars in funding from Microsoft although they seem “officially unrelated.”
OpenAI's ownership structure is hilariously convoluted, even by the standards of, say, Mozilla having separate 501(c)(3) and for-profit arms. The goal of the company is to launder noncommercial research into commercial products, and they even have a laughable "capped profit corporation" explanation for this.
[0] There are two exceptions to this:
- Structured data, i.e. the caption field and Wikidata, is considered to be copyright-free and explicitly has a CC0 license applied to it.
- Some public domain images have contradicting copyright terms applied to their wikitext; i.e. CC-BY-NC-SA. Those will need to be detected and filtered out of the label set, but I haven't written the code to do this yet. That's also why I haven't released any trained models.
[1] Currently this is going to be in the form of Wikimedia Commons usernames, specifically all the users that were in the revision history for the images. I believe there are also some wikitext attributions that I need to write code to find and reproduce.