351–360 of 363 posts
Earlier quoted context omitted.
I think you're underestimating the severity of child abuse by orders of magnitude. CSAM is a database of child rape, not child nudity.
For now. You don't know what will be added next. China will demand it to include pictures of the Tiananmen massacre.
Earlier quoted context omitted.
honestly asking — why is it illegal?
It may not be, so honestly I think my objection is best dismissed. Once I ran down the actual chain I mostly sorted things out with a cooler head. However, the line of thinking was if Apple has a secondary classifier to run against visual derivatives, the intent is it can say "CSAM/Not CSAM". Since the NeuralHash can collide, that means they'd need something to take in the visual derivatives, and match it vs an NN tr…
Earlier quoted context omitted.
> E.g. just knowing some inputs and outputs of the secret model allows you to train your own classifier which behaves similarly enough to let you find perceptual hash collisions. This seems highly unlikely. You could train a model to find those exact known hashes, but I highly doubt you could get it to accurately find any other unknown hash. > You'd have to keep all of the following secret: the model, all its inputs,…
> This seems highly unlikely. You could train a model to find those exact known hashes, but I highly doubt you could get it to accurately find any other unknown hash. Your "highly doubt" is baseless. Black box attacks (where you create adversarial examples only using some inputs and outputs, but not the model) on machine learning models are not new. They have been demonstrated countless times [1]. You don't need to k…
This is not a machine learning model as such, though, and is used differently than they are.
> This is not the case, since regular, unprivileged Apple employees can and will look at the inputs and outputs of the model
Can they?
To me the most interesting findings from this fiasco were: 1. People actually do use generally publicly available services to store and distribute CP (as suggested by the amount of reports done by Facebook) 2. A lot of people evidently use iCloud Photo Library to store images of things other than pictures they took themselves. This is not really surprising, I've learned that the answer of "does anybody ever?" questio…
The number of reports is not the number of actual incidents. It could be Facebook's algorithms are really shitty and has millions of false positives. NCMEC and similar organizations like to brag about numbers of reports because Big Number Good.
Earlier quoted context omitted.
It may not be, so honestly I think my objection is best dismissed. Once I ran down the actual chain I mostly sorted things out with a cooler head. However, the line of thinking was if Apple has a secondary classifier to run against visual derivatives, the intent is it can say "CSAM/Not CSAM". Since the NeuralHash can collide, that means they'd need something to take in the visual derivatives, and match it vs an NN tr…
i did a bit of reading as well and came across this. you might find it useful or interesting: https://www.law.cornell.edu/uscode/text/18/2258A at the end (h1-4), it details that providers must preserve the information they submit and also take steps to limit access to only people who need it. in this sense then, it’s not illegal for companies to possess csam. it’s not a big leap to then assume that storing csam for t…
Thank you, that was what I was looking for that closes the gap somewhat.
I've seen a lot of comments "muddying the waters" (intentionally or not) about whether hash colliders like the one demonstrated above can be used to carry out an attack. So I wrote up a quick FAQ addressing the most common points. Part 1/2 Q: I heard that Apple employees inspect a "visual derivative" of your photos before reporting you to the authorities. Doesn't this mean that, even if you modify images so their has…
FAQ Part 2/2 Q: If the second, secret hash algorithm is based on a neural network, can we think of its weights (coefficients) as some kind of secret key in the cryptographical sense? A: Absolutely not. If (as many suspect) the second hash algorithm is also based on some feature-identifying neural network, then we can't think of the weights as a key that (when kept secret) protects the confidentiality and integrity of…
a) hashes to the same value as known csam image A with the public NeuralHash algorithm
b) has a derivative (e.g. lower res thumbnail) that when processed with a _private_ perceptual hash algorithm also matches known csam image A.
A lot has been said about using this as an attack vector by possibly poisoning a victims iPhone with an image that matches a CSAM hash. But could this not also be used to circumvent the CSAM scanning by converting images that are in the CSAM database to visually similar images that won't match the hash anymore? That would effectively defeat the CSAM scanning Apple and others are trying to put into place completely an…
Earlier quoted context omitted.
> A hash collision allows you to create material that matches CSAM signatures, without being CSAM. This is not correct. Hash collisions won’t match the visual derivative.
Sorry, this is not even wrong. The visual derivative is just a resized, very-low-resolution version of the uploaded image. "Matching the visual derivative" is completely meaningless. The visual derivative is not matched against anything, and there is no "original" visual derivative to match against. If enough signatures match, Apple employees can decrypt the visual derivatives, and see if these extremely low resoluti…
Probably a mistake to say things like this, when the public documentation contradicts you.
> The visual derivative is not matched against anything, and there is no "original" visual derivative to match against.
Bullshit.
Here is the relevant paragraph from Apple’s documentation:
“as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possi- bility that the match threshold was exceeded due to non-CSAM images that were ad- versarially perturbed to cause false NeuralHash matches against the on-device en- crypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.”
https://www.apple.com/child-safety/pdf/Security_Threat_Model...
Earlier quoted context omitted.
I just want to be clear if I understand this... many images can result in the same hash, but the hash can and will be reversible into one image? And that image is a low res porn photo derived from the algorithm's guesswork? So once a hash matches they don't check if there was a collision and the photo is completely unrelated, they just see the CG porn? If that's the case then why even look at the derived image?
No, this is not what's going on at all. The employees never see the original photos in the government CSAM hash database. Apple doesn't even have these photos: it's precisely the kind of content that they don't want to store on their servers. If some conditions are satisfied, the employees gain access to the visual derivatives (low-resolution copies) of your photos, and they judge whether these look like they could p…
The relevant parts are.
> but based on the technical summary that Apple provided, it almost certainly goes something like this.
It doesn’t go like that. You are simply wrong.
Earlier quoted context omitted.
I didn’t claim it was impossible, just that hash collisions won’t match both. Also, the information needed to create a full match simply is not available.
Are those not the same statement? Unless you're relying on it being computationally infeasible, but I'm not sure we know enough to consider that true at this point. Usually when we make statements on those grounds we do so with substantial proof. I don't think we know enough to do so here. I'm not even sure how feasible it is when you throw DL into the mix.