A catalog of naturally occurring images whose Apple NeuralHash is identical • Hacker News Board

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#151

Post by [deleted] » Wed, Aug 25, 2021, 2:07 PM UTC

ajb wrote: post #97

scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

If you can get exact collisions, this can be gamed. For example, suppose there are two rival gangsters. One wants to set the police on his rival. He knows that a certain (innocuous) image is on his rival's phone. So he pays someone to generate a fake child-porn image with the same neuralhash, and ensure that it gets into the child porn DB. Then, apple reports the rival to the police, and they come and investigate him…

[deleted]

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#152

Post by teekert » Wed, Aug 25, 2021, 2:07 PM UTC

theshrike79 wrote: post #105

teekert wrote: post #85

I still think the biggest problem is that at some point a human is going to look at a false positive, this may be picture of my naked children and this human may not have the best intentions with my picture. That said, Nextcloud is my backend and I do not upload anything to iCloud (except for MS authenticator 2fa backups), so I'm safe right?

So your threat model here is that the person at Apple tasked to check for Child Porn is an actual Paedophile and might accidentally see a false positive of your child's naked photo? You do know that they don't see the whole photo at full megapixel resolution? They're just given "a visual derivative" of the photo for checking. Also, you really think that the persons tasked with this process are just randos off the str…

Are you saying randos flock to this position?

And where do you get the "visual derivative" information? Apple sure didn't communicate that to me. All I know is some person may look at my pics at some point.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#153

Post by dathinab » Wed, Aug 25, 2021, 2:07 PM UTC

Grustaf wrote: post #98

teekert wrote: post #85

I still think the biggest problem is that at some point a human is going to look at a false positive, this may be picture of my naked children and this human may not have the best intentions with my picture. That said, Nextcloud is my backend and I do not upload anything to iCloud (except for MS authenticator 2fa backups), so I'm safe right?

Since the risk of that is 1 in a trillion, a lot of people are quite happy to take that risk.

It's not 1 in a trillion, it's MUCH higher.

We are not speaking about a situation where not a "arbitrary" picture is miss-classifieds.

We are speaking about a situation where a innocent picture involving a naked or not fully clothed child is deemed similar to a non innocent picture of a naked or not fully clothed child.

Now you might argue that there should not be a picture a a naked or not fully clothed child of any form ever on any phone, but IMHO that is short sighed, discriminating and at best shows you don't know to much about the world and other cultures.

Let's list some simple reasons such a think could happen first:

- Photos meant for a doctor, or living partner to ask if something is normal or a problem. In many different ways.

- Photos of little children bathing or similar a e.g. dad sends to their mom who is currently on a business trip.

- etc.

Reasons people are less aware of is that not all countries are as stuck up about nakedness especially in the family. So it's totally normal for families that e.g. before or after taking a shower family member independent of age and gender walk through the apartment naked. Similar if you didn't got any shame about the naked body indoctrinated you might totally do thinks like visiting a "naked-beach" with your family (meeting other families and taking advantage of it often being less crowded) and in turn normal innocent beach family pictures contain naked children. And on itself that's not a problem. But with Apples approach stuff like this is like to trigger both systems Apple announced and wrongly label your while family as pedophiles...

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#154

Post by teekert » Wed, Aug 25, 2021, 2:08 PM UTC

Grustaf wrote: post #98

teekert wrote: post #85

I still think the biggest problem is that at some point a human is going to look at a false positive, this may be picture of my naked children and this human may not have the best intentions with my picture. That said, Nextcloud is my backend and I do not upload anything to iCloud (except for MS authenticator 2fa backups), so I'm safe right?

Since the risk of that is 1 in a trillion, a lot of people are quite happy to take that risk.

1 in a trillion with a billion devices with 10k pics is not a small chance. But what do we know? It's not like Apple is communicating any numbers anywhere so we can make a reasonable guess as to the number of false positives, and as to whether we may think it is actually worth it.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#155

Post by teekert » Wed, Aug 25, 2021, 2:11 PM UTC

dathinab wrote: post #153

Grustaf wrote: post #98

Earlier quoted context omitted.

Since the risk of that is 1 in a trillion, a lot of people are quite happy to take that risk.

It's not 1 in a trillion, it's MUCH higher. We are not speaking about a situation where not a "arbitrary" picture is miss-classifieds. We are speaking about a situation where a innocent picture involving a naked or not fully clothed child is deemed similar to a non innocent picture of a naked or not fully clothed child. Now you might argue that there should not be a picture a a naked or not fully clothed child of any…

Where I live some kids even play naked on normal beaches or near lakes or in their backyard, whatever. Up to 2 years old it's very normal. Within families kids and parents see each other naked up until puberty of the kids and the kids don't feel comfortable anymore. If you go to a sauna, everybody is naked.

Or what if a kid gets her/his hands on a phone and takes some pics (doesn't even require unlocking the phone) by accident?

The US may be one of the most up-tight countries about nakedness in general if you ask me. At the same time it's hyper sexualized and produces people like Nicky Minaj, but I guess there is still some fabric over their most "special" parts so it's ok. But oh god, what if Justin Timberlake rips it off... Pandemonium.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#156

Post by GistNoesis » Wed, Aug 25, 2021, 2:16 PM UTC

scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

Afiu, Apple's NeuralHash uses exact collisions when they do their Private Set Intersection.

The main advantage of using exact collision is that you can then blind the perceptual hash with a cryptographic hash and avoid any leak of information. (Taking for example sha256 of this perceptual hash won't allow any attacker to get any information on the features from the hash, but if the perceptual hash are the same then the input of the sha256 is the same and therefore the output of the sha256 is the same).

This is important because it alleviates the risks of an eventual leaking the database as Apple never touched and compared sensitive content but only cryptographic hashes of the perceptual hashes.

Some other system like PhotoDNA, rely on a euclidean distance between features being less than a threshold to register a match, which allows to quantify how far the image is from CSAM, but mean that the hash leak some information about the original content.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#157

Post by tzs » Wed, Aug 25, 2021, 2:16 PM UTC

foxfluff wrote: post #123

My take on this is that the system is by and large useless. It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images, or find a way to obfuscate their images enough to bypass the system (the lower the false positive rate, the easier it must be to trick the system). So what's left when all the criminals this is supposed to catch hav…

> It won't catch anything but the dumbest of dumb criminals

Dumb is a pretty accurate description of a large fraction of criminals. For the most part you only get smart criminals when you are talking about crimes where you have to be smart to even plan and carry out the crime.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#158

Post by simondotau » Wed, Aug 25, 2021, 2:17 PM UTC

rz2k wrote: post #148

simondotau wrote: post #70

Earlier quoted context omitted.

There is an interesting constitutional quirk which arises from the scanning being done client side, specifically for US citizens. If the US Government forced Apple to add other entries to the hash table, this would constitute a warrantless Government search of the private physical property of US citizens. This is a clear-cut, unambiguous breach of the 4th Amendment. Whereas if the CSAM scanning was performed exclusiv…

I think you're implying that scanning of private personal property by a corporation without a warrant protects users from searches of their content in the cloud that is authorized by a warrant or national security letter. I don't understand the mechanism if there isn't end to end encryption, and I don't understand the mechanism if there is end to end encryption. Scanning makes phones a greater threat, and also erodes…

I did not intentionally imply anything here, so if anything I wrote appears to include an implied component, that was not intended and may not represent my opinion.

All I'm saying is that the implementation Apple has described would be constitutionally blocked from being co-opted by US law enforcement. Obviously if there's no end-to-end encryption, any cloud operator could still be coerced into searching for material server side, as that falls under the so-called third party doctrine.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#159

Post by woofie11 » Wed, Aug 25, 2021, 2:25 PM UTC

StrLght wrote: post #89

woofie11 wrote: post #80

Earlier quoted context omitted.

No. Most proper cryptographic hash systems (e.g. used for verifying files, rather than data structures) never have collisions. Try to find a SHA256 collision. Anywhere, ever, in the history of mankind. This isn't for lack of looking. A lot of very smart people have looked for them. If you find one, I bet you'll be eligible for a tenured faculty slot at a good university, if not more. A whole world of secure systems w…

Yes, but cryptographic hashes are irrelevant here because they'd allow to easily bypass CSAM by modifying/appending a single byte.

Apple is claiming to have a visual equivalent to a cryptographic hash -- one which won't change with a single byte, but only if the image is substantially different.

At least their security analysis relies on that.

From their whitepaper: "The threshold is selected to provide an extremely low (1 in 1 trillion) probability of incorrectly flagging a given account"

If your claim is that their hash algorithm isn't cryptographic, their security analysis is incorrect.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#160

Post by tgv » Wed, Aug 25, 2021, 2:28 PM UTC

halflings wrote: post #112

tgv wrote: post #82

Earlier quoted context omitted.

The hash is 96 bits long. When hashing 1 billion pictures, that gives a collision probability of 6e-12. If it were uniformly distributed. There's no way people have hashed billions of images already. It just shows that it's pretty probably there will be collisions, and on visual inspection, it looks as if the collisions will happen on visually similar images. So if there's a naked baby pic in the CSAM database, quite…

Clearly this is not a cryptographic hash, and hence it's known hashes are not uniformly distributed. Apple explained in their technical summary [0] that they'll only consider this an offence if a certain number of hashes match. They estimated the likelihood of false positives there (they don't explain which dataset was used, but it was non-CSAM naturally) is 1 out of a trillion [1] In the very unlikely event where th…

> they have manual operators to check each of these photos

For now. But what will happen when there are thousands of false positives per day? Will they increase the staff? Or will they add another algorithmic layer? Or just up the threshold a bit? There's no guarantee. The only thing that's certain is that the NeuralHash doesn't inspire confidence.