21–30 of 363 posts
I am strongly against Apple’s decision to do on-device CSAM detection, but: wasn’t there a secondary hash whose database is not shared? In theory you need to collide with both to truly defeat the design, right?
Earlier quoted context omitted.
Good that there’s some human supervision. But, I know I have more than 30 photos of my dog. Also don’t like the idea of false positives auto-sharing some of my camera roll.
It's only if you back it up to iCloud, the signatures of the CP used as references are rotated, and they're also not public. The chances of you randomly triggering the system is effectively 0 unless you're uploading CP to your iCloud.
They're scanning anything you upload to iCloud (and have been for some time) but now also scan everything on your device too.
The README ( https://github.com/anishathalye/neural-hash-collider#how-it-... ) explains in a bit more detail how these adversarial attacks work. This code pretty much implements a standard adversarial attack against NeuralHash. One slightly interesting part was replacing the thresholding with a differentiable approximation. I figured I'd share this here in case anyone is interested in seeing what the code to generate…
I'd like to share the following paper for anyone else who may be interested. It is about watermarking rather than a preimage attack.
"Adversarial Embedding: A robust and elusive Steganography and Watermarking technique" https://arxiv.org/abs/1912.01487
Unfortunately, the existence of invisible watermarking demonstrates a separate attack on the hash. Instead of a preimage attack, this might be able to change the hash of an image that is suspected of already being a match. A true-positive would be changed into a false-negative.
Apple has said this is not the final version of the hashing algorithm they will be using: https://www.vice.com/en/article/wx5yzq/apple-defends-its-ant...
If Apple wants to defend this they should try to explain how the system will work even if generating adversarial images is trivial.
Until that day, just send known CSAM to any person you'd like to get in trouble (make sure they have icloud sync enabled), be it your neighbour or a political figure, and start a PR campaign accusing the person of being investigated for it. The whole concept is so inherently flawed it's crazy they haven't been sued yet.
I am strongly against Apple’s decision to do on-device CSAM detection, but: wasn’t there a secondary hash whose database is not shared? In theory you need to collide with both to truly defeat the design, right?
They're hashing on feature space (so trivial cropping and such doesn't defeat this) but they have two totally separate methods of matching those hashes? Doesn't sound right to me...
Earlier quoted context omitted.
It's only if you back it up to iCloud, the signatures of the CP used as references are rotated, and they're also not public. The chances of you randomly triggering the system is effectively 0 unless you're uploading CP to your iCloud.
Wait, wasn't all the hullabaloo over this scanning not requiring an upload to iCloud anymore? They're scanning anything you upload to iCloud (and have been for some time) but now also scan everything on your device too.
I am strongly against Apple’s decision to do on-device CSAM detection, but: wasn’t there a secondary hash whose database is not shared? In theory you need to collide with both to truly defeat the design, right?
You just need a sample of something that is evidently so pervasive we're building a nation wide dragnet to stop.
This is just getting wilder and wilder by the day, how spectacularly this move has backfired. As others have commented, at this point all you need is someone willing to sell you the CSAM hashes on the darknet, and this system is transparently broken. Until that day, just send known CSAM to any person you'd like to get in trouble (make sure they have icloud sync enabled), be it your neighbour or a political figure, an…
This is just getting wilder and wilder by the day, how spectacularly this move has backfired. As others have commented, at this point all you need is someone willing to sell you the CSAM hashes on the darknet, and this system is transparently broken. Until that day, just send known CSAM to any person you'd like to get in trouble (make sure they have icloud sync enabled), be it your neighbour or a political figure, an…
With the previous status quo:
1. The attacker faces charges of possessing and distributing child pornography
2. The victim may be investigated and charged with child pornography if LEO is somehow alerted (which requires work, and can be traced to the attacker).
Poor risk/reward payoff, specifically the risk outweighs the reward. So it doesn't happen (often).
---
With the new status quo of lossy, on-device CSAM scanning and automated LEO alerting:
1. The attacker never sends CSAM, only material that collides with CSAM hashes. They will be looking at charges of CFAA, extortion, and blackmail.
2. The victim will be automatically investigated by law enforcement, due to Apple's "Safety Voucher" system. The victim will be investigated for possessing child pornography, particularly if the attacker collides legal pornography that may fool a reviewer inspecting a 'visual derivative'.
Great risk/reward payoff. The reward dramatically outweighs the risk, as you can get someone in trouble for CSAM without ever touching CSAM yourself.
If you think ransomware is bad, just imagine CSAM-collision ransomware. Your files will be replaced* with legal pornography that is designed specifically to collide with CSAM hashes and result in automated alerting to law enforcement. Pay X monero within the next 30 minutes, or quite literally, you may go to jail, and be charged with possessing child pornography, until you spend $XXX,XXX on lawyers and expert testimony that demonstrates your innocence.
* Another delivery mechanism for this is simply sending collided photos over WhatsApp, as WhatsApp allows for up to 30 media images in one message, and has settings that will automatically add these images to your iCloud photo library.