Earlier quoted context omitted.
> "look I prompted CoPilot for this piece of code that I already knew about and it spit it right out" https://twitter.com/docsparse/status/1581461734665367554 An english description plus three characters of a function name is enough to coax CoPilot into distributing LGPL-licensed code out of context, without a proper license. That's neither "emotional" nor "cherry-picked", it's a clear-cut license violation.
I mean, he clearly knew about that code in advance and used his prior knowledge to coax Copilot into spitting it out, yeah? Three characters can get you pretty far, that's 1 combination out of 125,580 (considering all english letters, upper and lower, along with most of the numbers and symbols on my keyboard), plus the description of a fairly complex algorithm. Also, this code is really just executing a mathematical…
So let's say I obtain an illegal copy of microsoft windows' source code. Under this precedent, what stops me from just (overfitting) training a neural network to produce the source code verbatim, sans any license notice?
But it doesn't end there. What stops me from making a neural network that exactly reproduces the bytes of Illegally_Ripped_Disney_Movie.mp4 that I obtain from the pirate bay? Copyright need not apply.
At what point can the neural network I've described (which is intentionally designed to violate copyright) distinguishable from a neural network like Copilot and others which violate copyright extrinsically?