Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

71–80 of 725 posts

Re: Hash collision in Apple NeuralHash model

#71

Post by [deleted] »

y7 wrote: post #65
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…

[deleted]

Re: Hash collision in Apple NeuralHash model

#73

Post by OJFord »

ptidhomme wrote: post #5

Yes, just like rape accusations. It doesn't matter that you prove it was false afterwards. Edit : well that was a hint to Assange of course. Probably not true in general. So yes, I mean false accusations.

Vouched, because as I understand the comment, people must not be reading past 'rape' and just gut-flagging with completely the wrong impression.

Re: Hash collision in Apple NeuralHash model

#74

Post by hdjjhhvvhga »

sitharus wrote: post #56
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

My WhatsApp automatically saves all images to my photo roll. It has to be explicitly turned off. When the default is on, it's enough that the image is received and the victim has CP on their phone. After the initial shock they delete it, but the image has already been sent to Apple, where a reviewer marked it as CP. Since the user already gave them their full address data in order to be able to use the app store, Appla can automatically send a report to the police.

Re: Hash collision in Apple NeuralHash model

#75

Post by varajelle »

vermilingua wrote: post #54
cm2187 wrote: post #42

Earlier quoted context omitted.

Don’t imessage and whatsapp automatically store all images received in the iphone’s photo library?

iMessage no, WA by default yes, but can be disabled.

So no need of hash collisions then. One can simply directly send the child porn images to that person via WhatsApp and send her to jail.

Re: Hash collision in Apple NeuralHash model

#76

Post by kmbfjr »

zaptheimpaler wrote: post #32

This is so overblown. Scanning images for CSAM seems to be a requirement followed by Facebook, Google, Insta and Snap already [1]: > To put this in perspective, in 2019 Facebook reported 65 million instances of CSAM on its platform, according to The New York Times. Google reported 3.5 million photos and videos, while Twitter and Snap reported “more than 100,000,” Apple, on the other hand, reported 3,000 photos. ALL o…

It is not overblown. I refuse to be a perpetual suspect in the possession and transfer of child pornography. Having these checks on my phone is just a short hop to checking everything the camera sees.

I worked with a guy who was convicted of this very crime. Do you know what the FBI installs on his computer and phone? This kind of monitoring.

I am not going to be treated like I am on parole.

Re: Hash collision in Apple NeuralHash model

#77

Post by [deleted] »

y7 wrote: post #65
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…

[deleted]

Re: Hash collision in Apple NeuralHash model

#78

Post by FabHK »

y7 wrote: post #65
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…

Right. So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images).

But a conceivable novel avenue of attack would be to find an image that:

1. Does not look like CSAM to the innocent victim in the original

2. Does match known CSAM by NeuralHash

3. Does look like CSAM in the "visual derivative" reviewed by Apple, as you highlight.

Re: Hash collision in Apple NeuralHash model

#79

Post by gambiting »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

I don't know why you'd even go through this trouble. At least few years ago finding actual CP on TOR was trivial, not sure if the situation has changed or not. If you're going to blackmail someone, just send actual illegal data, not something that might trigger detection scanners.

>> What has changed wrt targeted attacks against innocent people?

Anecdote: every single iphone user I know has iCloud sync enabled by default. Every single Android user I know doesn't have google photos sync enabled by default.

Re: Hash collision in Apple NeuralHash model

#80

Post by WA »

nyuszika7h wrote: post #66
WA wrote: post #53

Earlier quoted context omitted.

Is it so hard to understand? Some people don’t use cloud storage for precisely the reason that the photos are not encrypted. Now they can’t even use their phone for storing photos. The thing with "only when iCloud is enabled" is only for now. It’s trivial to make Scanning all photos default in a future version.

That would require a software update and would definitely not go unnoticed. Would you rather they implement scanning on server side and never be able to enable end-to-end encryption for iCloud Photos? I imagine that might be the end goal, otherwise I don't see why they wouldn't have just done it on server side. Sure, this system still has the potential to be abused, but if I had to choose between "end-to-end encrypte…

Strawman.

I choose no cloud storage plus device that doesn’t scan my data.

You argue for the former, which isn’t implemented, vs the latter, which I assume is reality anyways for all cloud storage providers.

We can have this discussion if Apple implements the former. If this was the goal, they would’ve announced it like you suggested.

Post reply on HN