Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

251–260 of 725 posts

Re: Hash collision in Apple NeuralHash model

#251

Post by dhosek »

Grustaf wrote: post #200
nbzso wrote: post #106

Earlier quoted context omitted.

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

How likely is it that you will have enough colliding images in your photo library to even trigger a review? I'm guessing you need at least 5 images, perhaps much more, to trigger it. In any case, 1 image is definitely not enough.

Apple has publicly stated 30. Plus, there would be a manual review of the images before forwarding to law enforcement. I don't see this collision attack leading to innocent people being handed over to the police. The worst-case (best-case?) scenario is that Apple gives up on the whole hashing system and then there will be zero chance of there ever being end-to-end encryption of photos in iCloud.

Re: Hash collision in Apple NeuralHash model

#252

Post by [deleted] »

sandworm101 wrote: post #201
nbzso wrote: post #106

Earlier quoted context omitted.

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

>> there is no law who requires them to "scan" on device. There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a…

[deleted]

Re: Hash collision in Apple NeuralHash model

#253

Post by markus92 »

endisneigh wrote: post #218
Koffiepoeder wrote: post #163

Earlier quoted context omitted.

E.g. send them a whatsapp message that looks innocent

They can see the image - why would they import a random image into their library from someone they don’t know?

Auto import could be turned on?

Re: Hash collision in Apple NeuralHash model

#254

Post by raspasov »

I'm not in favor of assuming that everyone's guilty until proven innocent.

But, as a side note...

I get the feeling that a lot of people assume that the CSAM hashes are going to be stored directly on everyone's phone so it's easy to get a hold of them and create images that match those hashes.

That does not seem to be the case. The actual CSAM hashes go through a "blinding" server-side step.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Hash collision in Apple NeuralHash model

#255

Post by ethbr0 »

sandworm101 wrote: post #246
Grustaf wrote: post #223

Earlier quoted context omitted.

Why would they extend the CSAM scanner? It would be much simpler to just use all the OCR and image classification functions they have already deployed. CSAM scanning is only useful for areas where Apple really doesn't want to even look at the actual material until they are extremely certain that it's a match. If they want to detect anti-government propaganda or something, there would be no such concerns, they would j…

>> useful for areas where Apple really doesn't want to even look at the actual material Correct. It has plausible deniability built in. Apple is unable to verify that the images the government are looking for are actually CSAM. They could be political. They could be protest images. They could be Winnie the Pooh. Apple can plead ignorance as it blindly scans for whatever the requesting government asks it to scan for.…

Why even keep Apple in the loop? Why not just allow government to submit scanning models directly?

Which Apple will dutifully install and run, because they're required by local laws.

Re: Hash collision in Apple NeuralHash model

#256

Post by Ajedi32 »

Majromax wrote: post #164
halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

> The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. It is, actually. Remember that hashes are supposed to be many-bit digests of the original; it should take O(2^256) work to find a message with a chosen 256-bit hash and O(2^128) work to find a "birthday attack" collision. Finding any collision at all with NeuralHash so soon after its release is…

NeuralHash is a perceptual hash, not a cryptographically secure hash. Perceptual hashes have trivially findable second preimages by design, as the entire point is for two different images which appear visually similar to return the same result.

It's not particularly surprising to me that a perceptual hash might also have collisions that don't look similar to the human eye, though if Apple ever claimed otherwise this counterexample is solid proof that they're wrong.

Re: Hash collision in Apple NeuralHash model

#257

Post by UseStrict »

incrudible wrote: post #81
FabHK wrote: post #41

Earlier quoted context omitted.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

Also, this XKCD: https://xkcd.com/538/ People are getting nerd-sniped about hash collisions. It's completely irrelevant. The real-world vector is that an attacker sends CSAM through one of the channels that will trigger a scan. Through iMessage, this should be possible in an unsolicited fashion (correct me if I'm wrong). Otherwise, it's possible through a hacked device. Of course there's plausible deniability here, b…

Plausible deniability or not, it could have real impact if Apple decides to implement the policy of locking your account after tripping the threshold, which you then have to wait or fight to get unlocked. Or now you have police records against you for an investigation that lead nowhere. It's not a zero impact game if I can spam a bunch of grey blobs to people and potentially have a chain of human failures that leads police knock down your door.

Re: Hash collision in Apple NeuralHash model

#258

Post by stetrain »

UncleMeat wrote: post #157
short_sells_poo wrote: post #127

Earlier quoted context omitted.

Google was not standing on a pedestal preaching privacy. In contrast Apple was trying to appear as the privacy conscious hardware/software vendor. To now implement such a blatantly obvious stepping stone to dragnet surveillance of actual devices is such a hypocritical move that it beggars belief. Ignoring the whataboutism in your question, we know that privacy once lost is practically impossible to get back. Once the…

It just seems to me that there are two very different conversations happening at the same time, with people swapping back and forth between them 1. There can be false positives or other mechanisms for innocent people to get flagged. 2. It is bad to do this sort of check on the local disk. The discussion at hand started as entirely #1. But now you've swapped to #2, talking about government spying on local files. It ma…

This exactly.

I am mostly convinced based on the technical details that have (slowly) come out from Apple that they have made this system sufficiently inconvenient to use as a direct surveillance system by a malicious government.

Yes a government could secretly order Apple to make changes to the system, but they could also order them to just give them all of your iCloud photos and backups, or send data directly from the Camera or Messages app, or any number of things that would be easier and more useful for the government. If you don't trust your government don't use a mobile device on a public cell network or store your data on servers.

But all of that said there is a still a line being crossed on principle and precedent for scanning images locally on device and then reporting out when something "bad" is found.

Apple thinks this is more private than just directly rifling through your photos in iCloud, but I can draw a line between what is on my device and what I send to Apple's servers and be comfortable with that.

Re: Hash collision in Apple NeuralHash model

#259

Post by bostonsre »

y7 wrote: post #65
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…

Why would someone do that? Why not just send the original if both are flagged as the original?

Re: Hash collision in Apple NeuralHash model

#260

Post by bscphil »

notRobot wrote: post #162
shapefrog wrote: post #122

Earlier quoted context omitted.

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

Many people never see the inside of a courtroom when false or unproven rape accusations are made against them, but their lives still get ruined because of the negative publicity.

Those cases are not comparable, because the whole reason they have that impact is that the accusations are usually made publicly (because the whole point is to harm the reputation of one's rapist and warn others, should a conviction prove to be impossible), while CSAM review goes through a neural hash privately on your phone, then privately and anonymously through an Apple reviewer, then is privately reviewed at NCMEC (who - I think - have access to the full size image), and only then is turned over to law enforcement (which should also have access to the full image).

It only becomes public knowledge if law enforcement then chooses to charge you - and if all that happens on the basis of an obvious adversarial net image, the result is a publicity shitshow for Apple and you become a civil rights hero after your lawyer (even an underpaid overworked public defender should be able to handle this one) demonstrates this.

As others have stated in this thread, I think the real failure case is not someone's life getting ruined by claims of CSAM possession somehow resulting from a bad hash match, but the fact that planted material (or sent via message) can now easily ruin your life because it gets automatically reported; you can't simply delete it and move on any more.

Post reply on HN