> If hashes are uploaded to devices, they can be extracted and images that clash against it can be created.
Many organizations have the hashes, so they could leak nonetheless. Either way, I don't think that's a major problem. If the system interprets a picture of a pineapple as CSAM, you only need to produce the picture of a pineapple to defend yourself against any accusations. If clashes are too commonplace, the entire system would become unreliable and would have to be scrapped.
In any case, I have looked it up. The database is indeed on the device, but it's encrypted:
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
> Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.
Overall, after reading the PDF, here is my understanding of the process:
1. Apple gathers a set of "bad hashes"
2. They upload to each device a map from a hashed bad hash to an encrypted bad hash
3. The device runs an algorithm that determines whether there are matches with hashed bad hashes
4. For each match, the device uploads a payload encrypted using a secret on-device key, and a second payload that contains a "share" of the secret key, encrypted using the neural hash and encrypted bad hash.
5. The device also periodically uploads fake shares with dummy data to obfuscate the number of matches that actually occurred. Apple can't tell fake shares from real ones unless they have enough real shares.
6. Once Apple has enough real shares, they can figure out the secret key and know which hashes caused a match.
The main concern I have, and as a non-expert, is step 2: it requires Apple to provide their key to an auditor who can cross-check with child protection agencies that everything checks out and no suspect hashes are included in the payload. In theory, that needs to be done every time a new on-device database is uploaded, but if it is done, or if child protection agencies are given the secret so that they can check it themselves, I think this is a fairly solid system (notwithstanding the specifics of the encryption scheme which I don't have the competence to evaluate).
The thresholding is also a reassuring aspect of the system, because (if it works as stated) the device can guarantee that Apple can't see anything at all until a certain number of images match, not even the count of matching images. The threshold could only be changed with an OS update.
There's certainly a lot of things to discuss and criticize about their system, but it's going to be difficult to do so if nearly no one even bothers reading about how it works. It's frustrating.