Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

21–30 of 725 posts

Re: Hash collision in Apple NeuralHash model

#21

Post by nabakin »

user-the-name wrote: post #9
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

Agreed. This attack still has consequences for people but they will not run into any legal trouble as far as I'm aware.

Re: Hash collision in Apple NeuralHash model

#22

Post by user-the-name »

hughrr wrote: post #16
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract t…

The human reviewer would be able to check against the exact image that generated the hash in the first place. Taking another completely unrelated image and perturbing it would be immediately obvious.

Re: Hash collision in Apple NeuralHash model

#23

Post by jhugo »

kuu wrote: post #2

I don't understand the comment in the issue by an iPhone user. Can you see the hashes that the mobile generates for each image?? Why that is not "obfuscated" / hidden from the user? I mean, I would expect something complicated to validate that you have a collision.

You're holding the iPhone in your hand. You can inspect everything it does, the only thing that varies is the level of difficulty of inspecting it. Obfuscation doesn't work.

Re: Hash collision in Apple NeuralHash model

#24

Post by elisbce »

hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

You are insane.

1. Dumping iOS ecosystem and pick what? You think Android is better and won't have this? iOS is the strongest mobile system in terms of privacy protection available to this date. Hell, the FBI doesn't even need to ask Google to decrypt an Android.

2. Theoretically you can target attacks against anyone. It is just a matter of efforts. If you are a political target, they can already implant spywares around you to track and monitor you. They don't even need to break your phone.

3. If you are not possessing CSAM materials and not one of those targets, then you are not worth the efforts to be attacked or monitored. They don't care. And to be honest, this is the best(might be the only real) way to stay private.

Re: Hash collision in Apple NeuralHash model

#25

Post by user-the-name »

csande17 wrote: post #17
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

> It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography. I've never personally seen or looked for any images like this, but if they weren't already proliferating online and widely available to criminals, why would we need to build an elaborate client-side scanning system to detect and report people who have copies of them?

Do you have access to darknet child pornography trading networks? I don't.

Re: Hash collision in Apple NeuralHash model

#26

Post by hughrr »

user-the-name wrote: post #22
hughrr wrote: post #16

Earlier quoted context omitted.

That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract t…

The human reviewer would be able to check against the exact image that generated the hash in the first place. Taking another completely unrelated image and perturbing it would be immediately obvious.

So there’s an office somewhere with computers full of illegal child porn that people are staring at and comparing your photos to?

There’s some irony in that.

Re: Hash collision in Apple NeuralHash model

#27

Post by the8472 »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times

A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision.

https://openai.com/blog/adversarial-example-research/

Re: Hash collision in Apple NeuralHash model

#28

Post by [deleted] »

user-the-name wrote: post #25
csande17 wrote: post #17

Earlier quoted context omitted.

> It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography. I've never personally seen or looked for any images like this, but if they weren't already proliferating online and widely available to criminals, why would we need to build an elaborate client-side scanning system to detect and report people who have copies of them?

Do you have access to darknet child pornography trading networks? I don't.

[deleted]

Re: Hash collision in Apple NeuralHash model

#29

Post by neximo64 »

Maybe the best idea is for a sufficient number of people to replicate the hashes with collisions from the CSAM database and make copies of photos with nothing in them like this one and just let Apple deal with it. Maybe it can have text too.

Re: Hash collision in Apple NeuralHash model

#30

Post by FabHK »

hughrr wrote: post #16
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract t…

You:

> the hashes are on the device I understand and it’s not going to be that difficult to extract them.

----

A Review of the Cryptography Behind the Apple PSI System, Benny Pinkas, Dept. of Computer Science, Bar-Ilan University:

> Do users learn the CSAM database? No user receives any CSAM photo, not even in encrypted form. Users receive a data structure of blinded fingerprints of photos in the CSAM database. Users cannot recover these fingerprints and therefore cannot use them to identify which photos are in the CSAM database.

https://www.apple.com/child-safety/pdf/Technical_Assessment_...

----

The Apple PSI Protocol, Mihir Bellare, Department of Computer Science and Engineering University of California, San Diego:

> users do not learn the contents of the CSAM database.

https://www.apple.com/child-safety/pdf/Technical_Assessment_...

----

A Concrete-Security Analysis of the Apple PSI Protocol, Mihir Bellare, Department of Computer Science and Engineering University of California, San Diego:

> the database of CSAM photos should not be made public or become known to the user. Apple has found a way to detect and report CSAM offenders while respecting these privacy constraints.

https://www.apple.com/child-safety/pdf/Alternative_Security_...

Post reply on HN