In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :)
Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents using internet-enabled smartphones. The more we learn about the way Apple actually implemented this technology, the less likely it seems that it would make it radically easier for those bad actors to do so. For instance, the "it only scans photos uploaded to iCloud" element isn't just an arbitrary limitation that can be flipped with one line of code, as some folks seem to think; as Erik Neuenschwander, head of Privacy Engineering at Apple, explained in an interview on TechCrunch[1]:
> Our system involves both an on-device component where the voucher is created, but nothing is learned, and a server-side component, which is where that voucher is sent along with data coming to Apple service and processed across the account to learn if there are collections of illegal CSAM. That means that it is a service feature.
Will this stop those bad actors if they're determined? No, of course not, but there are so many ways they can do it already. They'll get cloud storage providers to give them access. If they can't, they'll get network providers to give them access. And if those bad actors are, as many people fear, government actors, then they have tools far more potent than code: they have laws. It was illegal to export "strong encryption" for many years, remember? I've seen multiple reports that European lawmakers are planning to require some kind of scanning for CSAM. If this goes into effect, technology isn't going to block those laws for you. Your Purism phone will either be forced to comply or be illegal.
I wrote in a previous comment on this that one of Silicon Valley's original sins is that we tend to treat all problems as if they're engineering problems. Apple is treating CSAM as an engineering problem. Most of the discussion on HN about how horrible and wrong Apple is here still treats it as an engineering problem, though: well, you can get around this by just turning off iCloud Photos or never using Apple software or throwing your iPhone in the nearest lake and switching to Android, but only the right kind of Android, or maybe just never doing anything with computers again, which admittedly will probably be effective.
Yet at the end of the day, this isn't an engineering problem. It's a policy problem. It's a governance problem. In the long run, we solve this, at least in liberal democracies, by voting people into office who understand technology, understand the value of personal encryption, and last but certainly not least, understand the value of, well, liberal democracy. I know that's easy to dismiss as Pollyannaism, but "we need to protect ourselves from our own government" has a pretty dismal track record historically. The entire point of having a liberal democracy is that we are the government, and we can pull it back from authoritarianism.
The one thing that Apple is absolutely right about is that expanding what those hashes check for is a policy decision. Maybe where those hashes get checked isn't really what we need to be arguing about.
[1]: https://techcrunch.com/2021/08/10/interview-apples-head-of-p...