One Bad Apple

so maybe i'm confused, but i thought it worked like this:

pre this thing:

* before syncing photos to icloud, the device encrypts them with a device local key, so they sit on apple's servers encrypted at rest and apple cannot look at them unless they push an update to your device that sends them your key or uploads your photos unencrypted somewhere else

after this thing:

* before syncing photos to icloud, the device encrypts them, but there are essentially two keys. one on your device, and one that can be derived on their servers under special circumstances. the device also hashes the image, but using one of these fancy hashes that are invariant to crops, rotations, translations and noise (like shazam, but for pictures)

* the encrypted photo is uploaded along with the hash (in a special crypto container)

* their service scans all the hashes, but uses crypto magic that does the following:

1) it does some homeomorphic encryption thing where they don't actually see the hash, but they get something like a zero knowledge proof if the image's hash (uploaded along with the image in the "special crypto container") is in their list of bad stuff

2) if enough of these hit, then there's a key that pops out of this process that lets them decrypt the images that were hits

3) the images get added to a list where a room full of unfortunate human beings look at them and confirm that there's nothing good going on in those photos

4) they alert law enforcement

a couple points of confusion to me:

1) i'm assuming they get their "one in a trillion" thing based on two factors. one being the known false positive rate of their perceptual hashing method and the other being their tunable number of hits necessary to trigger decryption. so they regularly benchmark their perceptual hash thing and compute a false positive rate, and then adjust the threshold to keep their overall system false positive probability where they want it?

2) all the user's photos are stored encrypted at rest, it seems that this thing isn't client side scanning, but rather assistance for server side scanning of end to end encrypted content put on their servers.

first off i think it's actually pretty cool that a consumer products company offers end to end encrypted cloud backup of your photos. i don't think google does this, or anyone else. they can just scan images on the server. second off, this is some pretty cool engineering (if i understand it correctly). they're providing more privacy than their competition aaand they've given themselves a way to ensure that they're not in violation of the law by hosting CSAM for their customers.

but i guess the big question is, if people don't like this, can't they just disable icloud?

>Demand drives creation. Getting rid of it on one of the largest potential viewing and sharing platforms is a move in the right direction in addressing the problem.

Yeah, that focus has worked really well in the "war on some drugs," hasn't it?

I don't pretend to have all (or any good ones for that matter) the answers, but we know interdiction doesn't work.

Those who are going to engage in non-consensual behavior (with anyone, not just children) are going to do so whether or not they can view and share records of their abuse.

The current legal regime (in the US at least) creates a gaping hole where even if you don't know what you have (e.g., if someone sends you a child abuse photo without your knowledge or consent) you are guilty, as possession of child abuse images is a felony.

That's wrong. I don't know what the right way is, but adding software to millions of devices searching locally for such stuff creates an environment where literally anyone can be thrown in jail for receiving an unsolicited email or text message. That's not the kind of world in which I want to live.

Many years ago, I was visiting my brother and was taking photos of his two sons, at that time aged ~4.5 and ~2.

I took an entire roll of my brother, his wife and their kids. In one photo, the two boys are sitting on a staircase, and the younger one (none of us noticed, as he wasn't potty trained and hated pants) wasn't wearing any pants.

I took the film to a processor and got my prints in a couple of days. We all had a good laugh looking at the photos and realizing that my nephew wasn't wearing any pants.

There wasn't, when the photos were taken, nor when they were viewed, any abuse or sexual motives involved.

Were that to happen today, I would be sitting in a jail cell, looking at a lengthy prison sentence. And when done "repaying my debt to society" I'd be forced to register as a sex offender for the rest of my life.

Which is ridiculous on its face.

Unless and until we reform these insane and inane laws, I can't support such programs.

N.B.: I strongly believe that consent is never optional and those under the age of consent cannot do so. As such, there should absolutely be accountability and consequences for those who abuse others, including children.

You're assuming that perceptual hashes are uniformly distributed, but that's not the case. If I post a picture of my kid at the beach I'm far, far more likely to generate perceptual hashes closer to the threshold. Not to mention intimate photos of/with my partner.

Why? It would be interesting to compare the two systems, but there is no reason to assume the trade-offs are the same.

If the visual proxy is enough to determine CSAM from non-CSAM, it's a significant invasion of privacy. Sure a thumbnail is less information than full-res but not that much less.