Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

81–90 of 725 posts

Re: Hash collision in Apple NeuralHash model

#81

Post by incrudible »

FabHK wrote: post #41
spicybright wrote: post #31

Earlier quoted context omitted.

If you're in close physical contact with a person (like at a job) you just wait for them to put their phone down while unlocked, and do all this.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

Also, this XKCD:

https://xkcd.com/538/

People are getting nerd-sniped about hash collisions. It's completely irrelevant.

The real-world vector is that an attacker sends CSAM through one of the channels that will trigger a scan. Through iMessage, this should be possible in an unsolicited fashion (correct me if I'm wrong). Otherwise, it's possible through a hacked device. Of course there's plausible deniability here, but like with swatting, it's not a situation you want to be in.

Re: Hash collision in Apple NeuralHash model

#82

Post by et2o »

hughrr wrote: post #51
henrikeh wrote: post #37

Earlier quoted context omitted.

Yes. That is called NCMEC in the US and it is a core aspect of how this whole process works. If you don’t understand the details of this, I’ll recommend this podcast episode which sums it up and discusses the implications https://atp.fm/443

I am aware of the details. The episode of Brass Eye comes into context here, the relevant clip being as follows, showing exactly the issues of competence. https://youtu.be/_U-7L1tmBAo

I don’t have much of an opinion here except that it is silly to write “I am aware of the details” when someone gives you a helpful explanation and you are obviously not aware of the details, as you had just asked a question betraying.

Re: Hash collision in Apple NeuralHash model

#83

Post by spicybright »

FabHK wrote: post #41
spicybright wrote: post #31

Earlier quoted context omitted.

If you're in close physical contact with a person (like at a job) you just wait for them to put their phone down while unlocked, and do all this.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

Have you not worked a minimum wage job in the US? It's incredibly easy to gain phone access to semi-trusting people.

If you don't like someone (which happens very often in this line of work) you could potentially screw someone over with this.

Re: Hash collision in Apple NeuralHash model

#84

Post by superjan »

FabHK wrote: post #41
spicybright wrote: post #31

Earlier quoted context omitted.

If you're in close physical contact with a person (like at a job) you just wait for them to put their phone down while unlocked, and do all this.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

Great article!

Re: Hash collision in Apple NeuralHash model

#85

Post by saynay »

formerly_proven wrote: post #68
sitharus wrote: post #56

Earlier quoted context omitted.

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

Yes, the diligent review performed by the lowest-bidding subcontractor is an excellent defense against career-ending criminal accusations. Nothing can go wrong, this is fine.

I would think there is way easier ways to frame someone with CSAM then this. Like dump a thumbdrive of the stuff on them and report them to the police.

Re: Hash collision in Apple NeuralHash model

#86

Post by Hackbraten »

nyuszika7h wrote: post #66
WA wrote: post #53

Earlier quoted context omitted.

Is it so hard to understand? Some people don’t use cloud storage for precisely the reason that the photos are not encrypted. Now they can’t even use their phone for storing photos. The thing with "only when iCloud is enabled" is only for now. It’s trivial to make Scanning all photos default in a future version.

That would require a software update and would definitely not go unnoticed. Would you rather they implement scanning on server side and never be able to enable end-to-end encryption for iCloud Photos? I imagine that might be the end goal, otherwise I don't see why they wouldn't have just done it on server side. Sure, this system still has the potential to be abused, but if I had to choose between "end-to-end encrypte…

> and can be disabled with a jailbreak if you're really paranoid

Except that if I’m really paranoid, I’m not going to skip security updates. And those updates will probably render known jailbreak exploits useless.

Re: Hash collision in Apple NeuralHash model

#87

Post by eptcyka »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

I'm sure the reviewers will definitely be able to give each reported image enough time and attention they need, much like the people youtube employs to review videos discussing and exposing animal abuse, holocaust denial and other controversial topics.

Re: Hash collision in Apple NeuralHash model

#88

Post by zimpenfish »

hdjjhhvvhga wrote: post #74
sitharus wrote: post #56

Earlier quoted context omitted.

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

My WhatsApp automatically saves all images to my photo roll. It has to be explicitly turned off. When the default is on, it's enough that the image is received and the victim has CP on their phone. After the initial shock they delete it, but the image has already been sent to Apple, where a reviewer marked it as CP. Since the user already gave them their full address data in order to be able to use the app store, App…

> but the image has already been sent to Apple, where a reviewer marked it as CP

No, the images are only decryptable after a threshold (which appears to be about 30) is breached. If you've received 30 pieces of CSAM from WhatsApp contacts without blocking them and/or stopping WhatsApp from automatically saving to iCloud, I gotta say, it's on you at that point.

Re: Hash collision in Apple NeuralHash model

#89

Post by mannerheim »

nyuszika7h wrote: post #58
mannerheim wrote: post #38

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times Just because the PoC used a meaningless blob doesn't mean that collisions have to be those. Plenty of examples of adversarial attacks on image recognition perturb real images to get the network to misidentify them, but to a human eye the image is unchanged.

The whole point flew over your head. If it's unchanged to the human eye then surely the human reviewer will see that it's a false positive?

No, it's important to point that out lest people think collisions can only be generated with contrived examples. I haven't studied neural hashes in particular, but for CNNs it's extremely trivial to come up with adversarial examples for arbitrary images.

Anyway, as for human reviewers, depends on what the image being perturbed is. Computer repair employees have called the police on people who've had pictures of their children in the bath. My understanding is that Apple does not have the source images, only NCMEC, so Apple's employees wouldn't necessarily see that such a case is a false positive. One would hope that when it gets sent to NCMEC, their employees would compare to the source image and see that is a false positive, though.

Post reply on HN