Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

51–60 of 725 posts

Re: Hash collision in Apple NeuralHash model

#51

Post by hughrr »

henrikeh wrote: post #37
hughrr wrote: post #26

Earlier quoted context omitted.

So there’s an office somewhere with computers full of illegal child porn that people are staring at and comparing your photos to? There’s some irony in that.

Yes. That is called NCMEC in the US and it is a core aspect of how this whole process works. If you don’t understand the details of this, I’ll recommend this podcast episode which sums it up and discusses the implications https://atp.fm/443

I am aware of the details. The episode of Brass Eye comes into context here, the relevant clip being as follows, showing exactly the issues of competence.

https://youtu.be/_U-7L1tmBAo

Re: Hash collision in Apple NeuralHash model

#52

Post by sneak »

brigandish wrote: post #49
ptidhomme wrote: post #5

Yes, just like rape accusations. It doesn't matter that you prove it was false afterwards. Edit : well that was a hint to Assange of course. Probably not true in general. So yes, I mean false accusations.

I think you may have attracted less downvotes if the phrasing was changed to "Yes, just like false accusations of rape , it doesn't matter that you prove it was false afterwards." I also think that those downvoting you might've applied the principle of charity and taken the best interpretation of what you've written or at least ask .

All criminal accusations, including true ones, should be treated as false until the accused is proven guilty.

This is a fundamental tenet of human rights in western, small-l liberal free societies.

The fact that this is controversial these days is literally insane to me.

The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was literally never even charged yet completely and thoroughly discredited due to headlines containing the word "rape" when no such thing ever happened.

(If you have been misled to believe otherwise, I encourage you to read the direct statements of the women involved.)

Re: Hash collision in Apple NeuralHash model

#53

Post by WA »

zaptheimpaler wrote: post #32

This is so overblown. Scanning images for CSAM seems to be a requirement followed by Facebook, Google, Insta and Snap already [1]: > To put this in perspective, in 2019 Facebook reported 65 million instances of CSAM on its platform, according to The New York Times. Google reported 3.5 million photos and videos, while Twitter and Snap reported “more than 100,000,” Apple, on the other hand, reported 3,000 photos. ALL o…

Is it so hard to understand? Some people don’t use cloud storage for precisely the reason that the photos are not encrypted.

Now they can’t even use their phone for storing photos.

The thing with "only when iCloud is enabled" is only for now. It’s trivial to make Scanning all photos default in a future version.

Re: Hash collision in Apple NeuralHash model

#54

Post by vermilingua »

cm2187 wrote: post #42
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Don’t imessage and whatsapp automatically store all images received in the iphone’s photo library?

iMessage no, WA by default yes, but can be disabled.

Re: Hash collision in Apple NeuralHash model

#55

Post by hughrr »

FabHK wrote: post #30
hughrr wrote: post #16

Earlier quoted context omitted.

That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract t…

You: > the hashes are on the device I understand and it’s not going to be that difficult to extract them. ---- A Review of the Cryptography Behind the Apple PSI System , Benny Pinkas, Dept. of Computer Science, Bar-Ilan University: > Do users learn the CSAM database? No user receives any CSAM photo, not even in encrypted form. Users receive a data structure of blinded fingerprints of photos in the CSAM database. User…

So fundamentally there’s a lot of mechanism around keeping secrets but the source material is available. Hmm.

https://youtu.be/eU2Or5rCN_Y

Re: Hash collision in Apple NeuralHash model

#56

Post by sitharus »

the8472 wrote: post #27
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

Re: Hash collision in Apple NeuralHash model

#57

Post by hughrr »

FabHK wrote: post #46
soziawa wrote: post #36

Earlier quoted context omitted.

> 6. Apple's CSAM detection then flags these, and they're manually reviewed Is the process actually documented anywhere? Afaik they are just saying that they are verifying a match. This could of course just be a person looking at the hash itself.

They look at the contents of the "safety voucher", which contains the neural hash and a "visual derivative" of the original image (but not the original image itself). https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

If it’s a visual derivative, whatever that means, then how does the reviewer know it matches the source image? Sounds like there’s a lot of non determinism in there.

Re: Hash collision in Apple NeuralHash model

#58

Post by nyuszika7h »

mannerheim wrote: post #38
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times Just because the PoC used a meaningless blob doesn't mean that collisions have to be those. Plenty of examples of adversarial attacks on image recognition perturb real images to get the network to misidentify them, but to a human eye the image is unchanged.

The whole point flew over your head. If it's unchanged to the human eye then surely the human reviewer will see that it's a false positive?

Re: Hash collision in Apple NeuralHash model

#59

Post by icelancer »

halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

> Of course, grey noise will never pass for CSAM and will fail that step.

Never? You sure that one or more human operators will never make this mistake, dooming someone's life / causing them immense pain?

Re: Hash collision in Apple NeuralHash model

#60

Post by nyuszika7h »

hughrr wrote: post #16
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract t…

> In the mean time they lock your account which means your entire digital life stops dead until their review process is done.

Do you have a source for this, or are you just making things up?

Post reply on HN