Earlier quoted context omitted.
We can speculate all we want. The fact is: its impossible to apply any real restrictionsn ML model. Just like any DRM this one gonna be easy to bypass and will only degrade experience of legit users and gonna slow down technology advancement. We dont require image editors to have CSAM filters and requiring filter for ML model is madness.
You're talking about filters inside the ML model, but I think the more interesting question is morality and the law.
There are two separate questions to be answered here:
1. Is it immoral?
2. Is it sufficiently immoral to warrant imposing my own moral standard upon others, by force if necessary? i.e. is responding to or preventing it with violence itself moral?
Morality is subjective and relative, which is why it's interesting to ask both questions: given the subjectivity and relativity of morality, at what point does your own moral code take precedence over those of others?