Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

581–590 of 725 posts

Re: Hash collision in Apple NeuralHash model

#581

Post by cyanite »

f3d46600-b66e wrote: post #305
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

What makes CSAM database private? It's my understanding that many tech companies (Microsoft? Dropbox? Google? Apple? Other?) (and many people in those companies) have access to the CSAM database, which essentially makes it public.

Well, the actual hash table on the device is blinded so the device doesn’t know if an image is a match or not. The server doesn’t learn the actual hash either, unless the threshold of 30 images is reached.

Re: Hash collision in Apple NeuralHash model

#582

Post by cyanite »

robertoandred wrote: post #463
mannerheim wrote: post #459

Earlier quoted context omitted.

Because the hashes are stored on the user's device?

No, encrypted and blinded hashes are stored. You can’t extract them.

This also means that the device doesn’t know if a given image is a match.

Re: Hash collision in Apple NeuralHash model

#583

Post by rootusrootus »

dannyw wrote: post #480
rootusrootus wrote: post #333

Earlier quoted context omitted.

> State actors have the resources You can end the conversation right there. If you are up against a state actor, you have already lost.

Incorrect. A Chinese state actor can't just go around imprisoning journalists they don't like in America, but they can now do this through planting child porngoraphy via remote malware (Pegasus) and watch their enemies get arrested by the US Feds.

Disagree. It isn't just jurisdiction. It is resource access. If the Chinese gov't were coming after little old me right now, I'd be properly worried, even though I'm safely within the boundaries of the US.

Re: Hash collision in Apple NeuralHash model

#584

Post by cyanite »

falcolas wrote: post #380
mannerheim wrote: post #378

Earlier quoted context omitted.

I believe the hash comparisons are made on Apple's end. Then the only way to get hashes will be a data breach on Apple's end (unlikely but not impossible) or generating it from known CSAM material.

That's not what Apple's plans state. The comparisons are done on phone, and are only escalated to Apple if there are more than N hash matches, at which point they are supposedly reviewed by Apple employees/contractors. Otherwise, they'd just keep doing it on the material that's actually uploaded.

> That's not what Apple's plans state. The comparisons are done on phone

Yes but as stated in the technical description, this match is against a blinded table, so the device doesn’t learn if it’s a match or not.

Re: Hash collision in Apple NeuralHash model

#585

Post by cyanite »

mannerheim wrote: post #397
falcolas wrote: post #380

Earlier quoted context omitted.

That's not what Apple's plans state. The comparisons are done on phone, and are only escalated to Apple if there are more than N hash matches, at which point they are supposedly reviewed by Apple employees/contractors. Otherwise, they'd just keep doing it on the material that's actually uploaded.

Ah, never mind, you're right: > Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices. https://www.a…

He is not right, though. The system used will not reveal matches to the device, only to the server and only if the threshold is reached.

Re: Hash collision in Apple NeuralHash model

#586

Post by cyanite »

Ajedi32 wrote: post #256
Majromax wrote: post #164

Earlier quoted context omitted.

> The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. It is, actually. Remember that hashes are supposed to be many-bit digests of the original; it should take O(2^256) work to find a message with a chosen 256-bit hash and O(2^128) work to find a "birthday attack" collision. Finding any collision at all with NeuralHash so soon after its release is…

NeuralHash is a perceptual hash, not a cryptographically secure hash. Perceptual hashes have trivially findable second preimages by design , as the entire point is for two different images which appear visually similar to return the same result. It's not particularly surprising to me that a perceptual hash might also have collisions that don't look similar to the human eye, though if Apple ever claimed otherwise this…

The problem is that you’d need the original NeuralHash, which isn’t stored on the device. The device only has a blinded version.

Re: Hash collision in Apple NeuralHash model

#587

Post by manmal »

Invictus0 wrote: post #426
manmal wrote: post #203

Earlier quoted context omitted.

Currently, the image would have to be imported into the photos library, and iCloud upload must be enabled.

Whatsapp has a feature where all images are automatically saved to device as they are received. If automatic iCloud upload is on, then all the conditions are there for a person to innocently click on a spammy Whatsapp message, see a bunch of nonsense grayscale images, and continue on with their day--not realizing they are now being monitored for CSAM.

I agree, that will become a problem.

Re: Hash collision in Apple NeuralHash model

#588

Post by cyanite »

dannyw wrote: post #492
shapefrog wrote: post #468

Earlier quoted context omitted.

The process would not trigger any action. The NCMEC, who can look at the material, and are the people to whom the matter is reported, would compare the flagged image with the source material and reject it as not matching known CSAM. What if the legal porn of a 21 year old that triggered the collision match looked really really really close? So close that a human can not distinguish between the image of a 12 year old…

You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar? I'm not talking about images of rape here. I'm taking about images that you'd see on a regular porn site, of adults and their body parts. You are also aware that CSAM covers anywhere from 0 to 17.99 years of age, and the legal obligation to report exists equally for the whole spectrum? So let's say I do…

> You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar?

I doubt images that look quite generic will make it into those hash sets, though.

Re: Hash collision in Apple NeuralHash model

#589

Post by cyanite »

bo1024 wrote: post #152
Edd314159 wrote: post #105

I think I am in dire need of some education here and so I have questions: * Is this a problem with Apple's CSAM discriminator engine or with the fact that it's happening on-device? * Would this attack not be possible if scanning was instead happening in the cloud, using the same model? * Are other services (Google Photos, Facebook, etc.) that store photos in the cloud not doing something similar to uploaded photos, w…

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation. A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash,…

That’s an oversimplified and misleading description of how the system works, but ok. I recommend reading the technical description, or even the paper linked from that: https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Hash collision in Apple NeuralHash model

#590

Post by tick_tock_tick »

cyanite wrote: post #575
fsloth wrote: post #70

Earlier quoted context omitted.

"How can you use it for targeted attacks?" Just insert a known CSAM image on target's device. Done. I presume this could be used against a rival political party to ruin their reputation - insert bunch of CSAM images on their devices. "Party X is revealed as an abuse ring". This goes oh-so-very-nicely with Qanon conspiracy theories which even don't require any evidence to propagate widely. Wait for Apple to find the i…

> Just insert a known CSAM image on target's device. Done. What do you mean “just”? That’s not usually very simple. It needs to go into the actual photo library. Also, you need like 30 of them inserted. > I presume this could be used against a rival political party Yes, but it’s not much different from now, since most cloud photo providers scan for this cloud-side. So that’s more an argument against scanning all toge…

> It needs to go into the actual photo library.

iMessage photos received are automatically synced so no. Finding 30 photos take zero time at all on Tor. Hell finding a .onion site that doesn't have CP randomly spammed is harder.....

Post reply on HN