Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

111–120 of 725 posts

Re: Hash collision in Apple NeuralHash model

#111

Post by FabHK »

avianlyric wrote: post #104
FabHK wrote: post #78

Earlier quoted context omitted.

Right. So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images). But a conceivable novel avenue of attack would be to find an image that: 1. Does not look like CSAM to the innocent victim in the original 2. Does match known CSAM by NeuralHash 3. Does look like CSAM in the "visual derivative" reviewed by Apple, as you highlight.

Reading the imagine scaling attack article, it’s looks like it’s pretty easy to manufacture an image that: 1. Looks like an innocuous image, indeed even an image the victim is expecting to receive. 2. Downscales in such a way to produce a CSAM match. 3. Downscales for the derivative image to create actual CSAM for the review process. Which is a pretty scary attack vector.

Depends very much on the process Apple uses to make the "visual derivative", though. Also, defence by producing the original innocuous image (and showing that it triggers both parts of Apple's process, NeuralHash and human review of the visual derivative) should be possible, though a lot of damage might've been done by then.

Re: Hash collision in Apple NeuralHash model

#112

Post by yayr »

How long did it take now to make the Apple algorithm ultimately useless or even harmful?

Apple announcement of neural hashing: 5.8.2021.

Generic algorithm to generate a different matching image: 8.8.2021.

one script was already released 10 days ago here https://gist.github.com/unrealwill/c480371c3a4bf3abb29856c29...

Re: Hash collision in Apple NeuralHash model

#113

Post by avianlyric »

sitharus wrote: post #56
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

The reviewer may not be looking at the original image. But rather the visual derivative created during the hashing process and sent as part of the safety voucher.

In this scenario you could create an image that looks like anything, but where it’s visual derivative is CSAM material.

Currently iCloud isn’t encrypted, so Apple could just look at the original image. But in future is iCloud becomes encrypted, then the reporting will be don’t entirely based on the visual derivative.

Although Apple could change this by include a unique crypto key for each uploaded images within their inner safety voucher, allowing them to decrypt images that match for the review process.

Re: Hash collision in Apple NeuralHash model

#114

Post by avianlyric »

zimpenfish wrote: post #88
hdjjhhvvhga wrote: post #74

Earlier quoted context omitted.

My WhatsApp automatically saves all images to my photo roll. It has to be explicitly turned off. When the default is on, it's enough that the image is received and the victim has CP on their phone. After the initial shock they delete it, but the image has already been sent to Apple, where a reviewer marked it as CP. Since the user already gave them their full address data in order to be able to use the app store, App…

> but the image has already been sent to Apple, where a reviewer marked it as CP No, the images are only decryptable after a threshold (which appears to be about 30) is breached. If you've received 30 pieces of CSAM from WhatsApp contacts without blocking them and/or stopping WhatsApp from automatically saving to iCloud, I gotta say, it's on you at that point.

You’re aware that people sleep at night, and phones for the most part don’t, right?

Re: Hash collision in Apple NeuralHash model

#115

Post by UncleMeat »

michaelt wrote: post #50
FabHK wrote: post #35

Earlier quoted context omitted.

Remains to be shown whether that is possible, though.

Just yesterday, here on HN there was an article [1] about adversarial attacks that could make road signs get misread by ML recognition systems I'd be astonished if it wasn't possible to do the same thing here. [1] https://news.ycombinator.com/item?id=28204077

NN classifiers work differently than perceptual hashes and the mechanism to do this sort of attack is entirely different, though they seem superficially similar.

Re: Hash collision in Apple NeuralHash model

#116

Post by basilgohar »

zimpenfish wrote: post #88
hdjjhhvvhga wrote: post #74

Earlier quoted context omitted.

My WhatsApp automatically saves all images to my photo roll. It has to be explicitly turned off. When the default is on, it's enough that the image is received and the victim has CP on their phone. After the initial shock they delete it, but the image has already been sent to Apple, where a reviewer marked it as CP. Since the user already gave them their full address data in order to be able to use the app store, App…

> but the image has already been sent to Apple, where a reviewer marked it as CP No, the images are only decryptable after a threshold (which appears to be about 30) is breached. If you've received 30 pieces of CSAM from WhatsApp contacts without blocking them and/or stopping WhatsApp from automatically saving to iCloud, I gotta say, it's on you at that point.

Just a side point, a single WhatsApp message can contain up to 30 images. 30 is the literal max of a single message. So ONE MESSAGE could theoretically contain enough images to trip this threshold.

Re: Hash collision in Apple NeuralHash model

#117

Post by Crontab »

deadalus wrote: post #3

Expectation : Political rivals and enemies of powerful people will be taken out because c-ild pornography will be found in their phone. Pegasus can already monitor and exfiltrate every ounce of data right now, it won't be that hard to insert compromising images on the infected device. Any news about "c-ild porn" being found on someone's phone is suspect now. This has been done before : 1) https://www.deccanchronicle.…

I remember reading the evil handyman story (link 2) a while ago and it scared me into using FDE. (Of course, FDE doesn't solve malware and things like that.)

Re: Hash collision in Apple NeuralHash model

#118

Post by Grollicus »

enedil wrote: post #102
wisienkas wrote: post #90

Not knowing too much of the NeuralHash model, but why are they using MD5 hash, they are known to have many collisions. We don't use MD5 for private/public keys for the same reason

We don't use md5 for private/public keys because md5 is a hashing algorithm, unrelated completely to encryption. Also, what are your reasons to believe that md5 has been used there?

Hashes are generally a part of the signature generation used with certificates. See for example "What role do hashes play in TLS/SSL certificate validation?" -> https://security.stackexchange.com/questions/67512/what-role...

In certificates, md5 - and sha1 - was used quite some time after it was known to be weak, I suspect OP was thinking of that.

This article seems to give a good summary what happened with sha1, mentions md5 in passing and links the related chromium issue: https://konklone.com/post/why-google-is-hurrying-the-web-to-...

Re: Hash collision in Apple NeuralHash model

#119

Post by avianlyric »

FabHK wrote: post #111
avianlyric wrote: post #104

Earlier quoted context omitted.

Reading the imagine scaling attack article, it’s looks like it’s pretty easy to manufacture an image that: 1. Looks like an innocuous image, indeed even an image the victim is expecting to receive. 2. Downscales in such a way to produce a CSAM match. 3. Downscales for the derivative image to create actual CSAM for the review process. Which is a pretty scary attack vector.

Depends very much on the process Apple uses to make the "visual derivative", though. Also, defence by producing the original innocuous image (and showing that it triggers both parts of Apple's process, NeuralHash and human review of the visual derivative) should be possible, though a lot of damage might've been done by then.

> Also, defence by producing the original innocuous image

At this point you’re already inside the guts of the justice system, and have been accused of distributing CSAM. Indeed depending on how diligent the prosecutor is, you might need to wait till trial before you can defend yourself.

At that point you’re life as you know is already fucked. The only thing proving your innocence (and the need to do so is itself a complete miscarriage of justice) will save you from is a prison sentence.

Re: Hash collision in Apple NeuralHash model

#120

Post by shapefrog »

lifthrasiir wrote: post #10
halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

But you can essentially perform DoS attack to human checkers, effectively rendering the entire system grind to a halt. The entire system is too reliant on the performance of NeuralHash which can be defaced in many ways. [1] (Added later:) I should note that the DoS attack is only possible with the preimage attack and not the second preimage attack as the issue seemingly suggests, because you need the original CSAM to…

I have seen it suggested that everyone should flood the system with flagged images to overwhelm it in protest to this move by apple.

Sounds pretty stupid to me to fill your phone with kiddie porn in protest, but you do you internet people.

Post reply on HN