That's a great point about what we naturally do.
> In the case where an AI is coming up with the work, perhaps the mistake is in relying on humans to try and apply their own trained judgement to figuring out if the result is unoriginal. We need an AI that scores work for how likely it is to be infringing on a prior copyright.
Isn't that latter AI going to be more likely to need to contain verbatim copies of original works? Or maybe not?
This in turn (and the SFC's and the law firm's concern about the GPL) makes me think that there are several different things that people may be concerned about machine learning systems doing:
* they could allow you to access verbatim copies for "consumptive" use (like if you asked an AI a question about what the text of a chapter of a Harry Potter novel was, and it answered you correctly)
* they could facilitate intentional or unintentional plagiarism, and, in the case of publicly-available works that are published under a license, intentional or unintentional reproduction or creation of derivative works contrary to that license
* they could contain something like a verbatim representation and allow you to use that in various ways that themselves are not extracting or literally copying that representation, but where the original copyright holder might complain that the existence of an unlicensed copy inside the model is already objectionable
* they could contain representations of uncopyrightable subject matter which was learned through training on copyrighted works, which can then be used to compete with the original creators for jobs, prestige, or attention, or can be used to produce works that the original creators would have found offensive or objectionable (this case isn't supposed to be restricted by copyright at all, but that doesn't necessarily stop people from caring!)
Not only will the same measures not prevent or avoid these cases, but if you wanted to prevent the first and second situations, one of the easiest ways to do it might be to literally include verbatim copies of lots of works inside a machine learning model! (along with software specifically trained or programmed to warn you against unintentionally making uses the user or copyright holder finds objectionable ... to facilitate the exercise of "essentially, their ethical judgement", as you put it)