One Bad Apple

I'm honestly shocked that Apple is buying into this because it's one of those well-intentioned ideas that is just incredibly bad. It also goes to show you can justify pretty much anything by saying it fights terrorism or child exploitation.

We went through this 20+ years ago when US companies then couldn't export "strong" encryption (being stronger than 40 bits if you can believe that). Even at the time that was ridiculously low.

We then moved onto cryptographic back doors, which seem like a good idea but aren't for the obvious reason that if a backdoor exists, it will be exploited by someone you didn't intend or used by an authorized party in an unintended way (parallel construction anyone?).

So these photos exist on Apple servers but what they're proposing, if I understand it correctly, is that that data will no longer be protected on their servers. That is, human review will be required in some cases. By definition that means the data can be decrypted. Of course it'll be by (or intended to be by) authorized individuals using a secured, audited system.

But a backdoor now exists.

Also, what controls exist on those who have to review the material? What if it's a nude photo of an adult celebrity? How confident are we that someone can't take a snap of that on their own phone and sell it or distribute it online? It doesn't have to be a celebrity either of course.

Here's another issue: in some jurisdictions it's technically a case of distributing CSAM to have a naked photo of yourself (if you're underage) on your own phone. It's just another overly broad, badly written statute thrown together in the hysteria of "won't anybody think of the children?" but it's still a problem.

Will Apple's system identify such photos and lead to people getting prosecuted for their own photos?

What's next after this? Uploading your browsing history to see if you visit any known CSAM trafficking sites or view any such material?

This needs to be killed.

> I equate this to Ring cameras.

But it's just a totally different situation isn't it? I'm not personally opposed to the general concept of security cameras in public spaces. If you choose to install one on your property that doesn't seem unreasonable. If you choose to share that footage with the police that doesn't seem unreasonable.

The problem with Apple's system isn't even the current implementation (which is on device, but iCloud only). It's that the system can be trivially expanded to all on device content, and report the user of the phone for any unacceptable behaviour.

It's more like a camera in your home that reports you to the police if you do anything unacceptable. Would anyone choose to have that?

But the bigger problem is that Apple have been selling products based on "privacy first" marketing. By reporting on their users off-line content they break that trust. And there are really few viable options in smartphones so you can't really just move to another provider.

That was never mentioned by Apple. If that was their intention then I suspect they would have mentioned it alongside this announcement to provide a justification and quell the (justified) outrage.

I also question the value of e2e there’s an arbitrary scanner that can send back the unencrypted files if it finds a match. If apple’s servers controls the db with “hashes” to match then is it all that different from apple’s servers holding the decryption keys?

Sure e2e still prevents routine large scale surveillance but at the end of the day if apple (or someone that forced apple) wants your data, they’ll get it.

There's a weird meme I the hacker community that law enforcement has no right to use any means to enforce the law, and can only bust people who turn themselves in.

Wow, 20% of images being misclassified as CP would be terrifying. I never thought I'd say it but my next phone will definitely not be an iPhone or any device that is uploading my information to another server without my permission for any reason.

Is there even any evidence that arresting people with the wrong bit pattern on the computer helps stop child rape/trafficking? If so, why aren't we also going after people who go to gore websites? There's tons of awful material out there easily accessible of people getting stabbed, shot, murdered, butchered, etc. Do we not want to find people who are keeping collections of this material on their computers? And if so, what about people who really like graphic horror movies like Saw or Hostel? Obviously it's not real violence, but it's definitely close enough, and if you like watching that stuff, maybe you should be on a list? If your neighbor to the left of you has videos of naked children, and your neighbor to the right has videos of people getting stabbed and tortured to death, only one should be arrested and put on a list?

This is all not even taking into account that someone might not even realize they are in possession of CP because someone else put it on their device. I've heard there's tons of services marketing on the dark net where you pay someone $X00 in bitcoin and they remotely upload CP to any target's computer.

It seems like we are going down a very scary and dangerous path.