Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A catalog of naturally occurring images whose Apple NeuralHash is identical

github.com

Post reply on HN
Print view View original

111–120 of 304 posts

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#111

Post by FabHK »

dTal wrote: post #44
simondotau wrote: post #36

Earlier quoted context omitted.

That's a lot of words to say that you could sufficiently mangle an image that it could pass through all of Apple's algorithmic hurdles while not actually being CSAM. Of that I have no doubt. You could definitely generate a mangled image that fools multiple perceptual hash algorithms. Let's set aside the questions of where you got all these hashes to generate collisions with, how you got 30 of these mangled images int…

I think you missed the point of the first paragraph. The point is that you can now hide child porn by making its hash collide with innocent images. They won't ever make it to manual review. Ergo, NeuralHash is now useless.

So, the presumed attack (not against individuals, but to defeat the system) is

1. Identify some innocuous pictures that many many people have (memes, Beyoncé, whatever).

2. Produce CSAM.

3. Mangle it such that it is still CSAM visually, but NeuralHash-collides with the innocuous pictures from step 1.

4. Distribute.

5. Wait until they are (via some other mechanism) a) identified as CSAM, b) added to the NCMEC database, c) added to the Apple on-device database of blinded hashes in some iOS update.

6. Millions of people are suddenly incorrectly flagged for exceeding the threshold by NeuralHash (since they have the innocuous pictures in their library), and the review teams are flooded and can't pick out the small number of actual CSAM holders.

That is not without a certain elegance. However, it seems to me that

A) it is predicated on the assumption that you can easily mangle pictures to NeuralHash-collide with a desired target picture (out of a set of widely circulating innocuous pictures) without deteriorating the visual content too much.

B) it would be quickly defeated by amending the 2nd tier algorithm (between NeuralHash and human review), though, as you highlight, that might be tricky given that the team working on this presumably only has access to the innocuous false positive collision image, not the (purposefully mangled) CSAM.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#112

Post by halflings »

tgv wrote: post #82
scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

The hash is 96 bits long. When hashing 1 billion pictures, that gives a collision probability of 6e-12. If it were uniformly distributed. There's no way people have hashed billions of images already. It just shows that it's pretty probably there will be collisions, and on visual inspection, it looks as if the collisions will happen on visually similar images. So if there's a naked baby pic in the CSAM database, quite…

Clearly this is not a cryptographic hash, and hence it's known hashes are not uniformly distributed.

Apple explained in their technical summary [0] that they'll only consider this an offence if a certain number of hashes match. They estimated the likelihood of false positives there (they don't explain which dataset was used, but it was non-CSAM naturally) is 1 out of a trillion [1]

In the very unlikely event where that 1 in a trillion occurrence happens, they have manual operators to check each of these photos. They also have a private model (unavailable to the public) to double-check these perceptual hashes which also used before alerting authorities.

[0] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... [1] https://www.zdnet.com/article/apple-to-tune-csam-system-to-k...

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#113

Post by saithound »

mlajtos wrote: post #102
woofie11 wrote: post #80

Earlier quoted context omitted.

No. Most proper cryptographic hash systems (e.g. used for verifying files, rather than data structures) never have collisions. Try to find a SHA256 collision. Anywhere, ever, in the history of mankind. This isn't for lack of looking. A lot of very smart people have looked for them. If you find one, I bet you'll be eligible for a tenured faculty slot at a good university, if not more. A whole world of secure systems w…

If I found one collision by accident, would that be any significant?

Yes, absolutely. It's technically possible to find SHA256 collisions accidentally, but it's so unlikely, that if you found one, it would merit serious investigation. People would not believe your statement that you found them accidentally, and "oh, I guess mlajtos really just found the colliding pair by chance" wouldn't be declared until after a very thorough investigation. In the meantime, major stakeholders (e.g. Bitcoin) would probably move away to another hash function, just in case.

Dwyer calculated 1431168 NeuralHashes and found two collisions. Humanity collectively calculates over 120000000000000000000 SHA-256 hashes every second. Still, we're reasonably sure that this immense brute-force search will not lead to any collisions in any reasonable amount of time.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#114

Post by theshrike79 »

yeldarb wrote: post #57
saithound wrote: post #25

Earlier quoted context omitted.

Discussing the preimage attack on NeuralHash is not technical ignorance. Dismissing the preimage attack as irrelevant is. 0. Most importantly: the existence of a preimage attack makes Apple's system completely useless for its original purpose. The NeuralHash collider allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with pe…

> allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with perfectly innocent images That’s a really interesting attack vector I hadn’t seen mentioned previously. Most people are talking about the potential for adversarial images to be sent to users. If they were instead injected into the database itself (either by poisoning…

Because they're not CSAM?

People don't seem to grasp what kind of images end up in the CSAM databases. They are most definitely not "leaked celebrity nude selfie" level stuff.

Think of the most vile sexual thing you could do to a child and then times that by two and halve the child's age in your mind. That's the shit that gets in there.

It's not something even 4chan weebaboos share. It's stuff that makes Liveleak regulars go "ewwwww, gross".

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#115

Post by theshrike79 »

floor_ wrote: post #76

Why not combine this with a second different hash? E: Better yet, only run the second hash if you have a collision, which should be very rare.

How do you know they aren't doing this on the backend after the initial on-device match?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#117

Post by collaborative »

roody15 wrote: post #54
collaborative wrote: post #15

Earlier quoted context omitted.

Tell this to the victims of Pegasus. If anyone were able to get their hands on the "secret backend system" we wouldn't be talking about spy games, we'd be talking about people's lives being ruined

Apple still has not patched the security exploit in iMessage used by Pegasus. Apple has released two ios security updates since the Pegasus revelations but still has not patched it most widely used exploit… hmmmm. Now apple is getting a local client side scanning tool ready. Interesting timing.

I don't think what you are saying is far-fetched

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#118

Post by chefandy »

scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

... does it have to prove Apple wrong about something to be interesting?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#119

Post by pbronez »

specialist wrote: post #99
scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

Per ATP: Apple will compare hashes of local photos with a national registry of child pornography photos. Once a certain (unknown) threshold is reached, let's say 20 hits, some kind of escalation occurs, with some kind of manual (human) review steps. Accidental Tech Podcast - A Storm of Asterisks https://atp.fm/443 I haven't listened to the follow up episode yet. I still have zero opinion on this photo scanning kerfuf…

Thanks for the link. Daring Fireball’s analysis is the most helpful take I’ve seen on the topic so far:

https://daringfireball.net/2021/08/apple_child_safety_initia...

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#120

Post by saithound »

FabHK wrote: post #111
dTal wrote: post #44

Earlier quoted context omitted.

I think you missed the point of the first paragraph. The point is that you can now hide child porn by making its hash collide with innocent images. They won't ever make it to manual review. Ergo, NeuralHash is now useless.

So, the presumed attack (not against individuals, but to defeat the system) is 1. Identify some innocuous pictures that many many people have (memes, Beyoncé, whatever). 2. Produce CSAM. 3. Mangle it such that it is still CSAM visually, but NeuralHash-collides with the innocuous pictures from step 1. 4. Distribute. 5. Wait until they are (via some other mechanism) a) identified as CSAM, b) added to the NCMEC database…

> A) it is predicated on the assumption that you can easily mangle pictures to NeuralHash-collide with a desired target picture (out of a set of widely circulating innocuous pictures) without deteriorating the visual content too much.

Note that this requires no single "desired" target picture. There are millions of popular, innocuous pictures. As long as you can make your CSAM match any one of them without significant mangling, you're good to go. Not having to choose one specific target makes this much easier to accomplish.

Post reply on HN