In some ways, I think privacy advocates could be shooting themselves in the foot here. I think that mainstream storage encryption (mainstream as in WhatsApp and Signal, as opposed to encrypting your NAS using linux commands) is very important for privacy, and that deploying it without running into CSAM objections and legislation is a Sisyphean task for for-profit businesses. Messaging seems to be more legally defensible than storage from a business perspective.
If you take as an axiom that mainstream businesses will be forced to protect themselves against contributing to CSAM distribution, then the choice to offer encrypted cloud storage to non-technical end users REQUIRES doing the scan on a trusted computer (Apple has chosen the phone itself).
I think the arguments that this can be abused are very real, but it's worth talking about how to fix that, because I think the alternative might be sacrificing E2EE cloud storage in the mainstream (as has happened with every other mainstream company). Perhaps more thought should be put into making this process auditable by the device owner (or by a trusted 3rd party -- say the EFF).
Or perhaps the scanning could be federated -- say I don't want Apple doing that, but I might trust a privacy oriented non-profit to "certify" to Apple that my personal photo album is CSAM-free. Can that 3rd party scan be blinded, such that I send data that is representative of my images, but I've already anonymized my photos using a transformation?
Could we audit (similar to certificate transparency):
1) What data from the device is being scanned? What data is being uploaded?
2) What "hashes" are being matched against, and how are those changing over time? Can the data lineage of the NCMEC database be audited? Would that pick up malicious hashes injected into the database?
Generally, I think our privacy paradigm needs to be built in such a way that it can actually be deployed in our policy environment. More realpolitik, less ethical grandstanding.