161–170 of 363 posts
I am strongly against Apple’s decision to do on-device CSAM detection, but: wasn’t there a secondary hash whose database is not shared? In theory you need to collide with both to truly defeat the design, right?
Earlier quoted context omitted.
Before they make it to human review, photos in decrypted vouchers have to pass the CSAM match against a second classifier that Apple keeps to itself. Presumably, if it doesn’t match the same asset, it won’t be passed along. This is explained towards the end of the threat model document that Apple posted to its website. https://www.apple.com/child-safety/pdf/Security_Threat_Model...
What happens if someone leaks or guesses the weights on that "secret" classifier? The whole system is so ridiculous even before considering the amount of shenanigans the FBI could pull by putting in non-CSAM hashes.
That was scary fast. Is there a point in using this algorithm for its intended purpose now?
Earlier quoted context omitted.
> But OK, say someone sends you a sunset that fools the hasher. Then what? Of course one match won’t do anything, so you’d need to download however many matching sunsets. Then what? A real attack would be to take legal porn images and make them collide with illegal images, so when a human goes to review the scaled down derivative images, those images very well look like they could be CSAM. Since there are many of the…
This where the "fog of war" kicks in. What with doors being busted down, police departments making press releases, etc. I can easily imagine that the victim could be prosecuted, convicted and sent away because no-one understood the subtlety that their legal porn was not in fact CSAM.
And that's assuming someone develops a hash collision which doesn't substantially mangle the photograph like the example offered on Github.
Specifically, only images categorised as "A1" are being included in the hash set on iOS. The category definitions are:
A = prepubescent minor
B = pubescent minor
1 = sex act
2 = "lascivious exhibition"
Earlier quoted context omitted.
LEO is not alerted automatically, where’d you get that idea?
They will be if you collide a low-res image that resembles CSAM. Why would person doing manual review risk his job in case if he’s unsure? Naturally he will just play it safe and report images.
Some people seem to be confused why a hash collision of a cat and a dog matters. Here's a potential attack: share (legal) NSFW pictures that are engineered to have a hash collision with CSAM to get someone else in trouble. The pictures are flagged as CSAM, and they also look suspicious to a human reviewer (maybe not enough context in the image to identify the subject's age). To show that this can be done with real NS…
You seem to be assuming a human cannot tell the difference from some random NSFW content, and some legit known CSAM, 30 times. Try again.
It seems unlikely in the event that there was anything that they verified as CSAM they wouldn't pass it on just because they found a false positive in those thumbnails.
Earlier quoted context omitted.
A hash collision allows you to create material that matches CSAM signatures, without being CSAM. This opens up a new class of attacks. Specifically, many criminal actors don't touch CSAM because it's wrong. But some of these criminal actors will happily abuse legal systems, e.g. SWATTing.
> A hash collision allows you to create material that matches CSAM signatures, without being CSAM. This is not correct. Hash collisions won’t match the visual derivative.
That said I think your statement is a bit too strong, but generally true. A hash collision is not going to inherently be visually confusing. However you claim that it is impossible for an image to be both visually confusing and a hash collision, which seems unlikely. The real question is going to be how much more effort it takes to do both.
Earlier quoted context omitted.
Those services will take your account through the same, if not more invasive, process if you are found with a hash match like the ones being proposed in these comments. Unlike Apple, they’ve built interfaces that surface all your account activity to reviewers.
> Unlike Apple, they’ve built interfaces that surface all your account activity to reviewers. You can't know this without independent audits.
Earlier quoted context omitted.
Why would anyone save CSAM to their photo library?
I posted another comment that was misunderstood as well. Folks, no one is proposing to download actual CSAM images to your photo lib. You could be duped thinking you downloaded an image of a beautiful sunset which was carefully manipulated to match the hash of an actual CSAM image.
If it was anything like the image used to demonstrate this technique on Github, it's unlikely that anyone would describe that sunset as "beautiful". They'd be more likely to describe it as "bugger, this JPEG file is corrupted."
Some people seem to be confused why a hash collision of a cat and a dog matters. Here's a potential attack: share (legal) NSFW pictures that are engineered to have a hash collision with CSAM to get someone else in trouble. The pictures are flagged as CSAM, and they also look suspicious to a human reviewer (maybe not enough context in the image to identify the subject's age). To show that this can be done with real NS…
You seem to be assuming a human cannot tell the difference from some random NSFW content, and some legit known CSAM, 30 times. Try again.