A catalog of naturally occurring images whose Apple NeuralHash is identical • Hacker News Board

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#121

Post by verygoodname » Wed, Aug 25, 2021, 12:49 PM UTC

nextlevelwizard wrote: post #38

How are any of these "naturally occurring" when all (4) examples are things cut out of context on a white background. Yeah two sticks (ski and nail) are visually similar on a white background. Why is this news to anyone? EDIT: if you are going to downvote please leave a comment unless you are just downvoting for wrong think.

As it is explained in the "readme" part, in this specific context, "naturally occurring" means that no one has purposefully manipulated any of the images to make them collide: that the images were already published and "out there" and happen to collide. In other words, it does not necessarily imply that the images correspond to natural photographic scenes (which seems to be your interpretation of it).

Besides, you could probably "naturally" obtain such type of colliding images by photographing similar-looking objects against a white (or generally featureless) background. Furthermore, it suggests/demonstrates that similar-looking images with similar backgrounds can lead to unexpected collisions in practice (i.e. "naturally"), even if you do not assume an adversarial scenario.

Are you sure that, if you take a picture of a naked body part, it won't collide with anything that looks similar in their database?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#122

Post by saithound » Wed, Aug 25, 2021, 12:49 PM UTC

theshrike79 wrote: post #114

yeldarb wrote: post #57

Earlier quoted context omitted.

> allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with perfectly innocent images That’s a really interesting attack vector I hadn’t seen mentioned previously. Most people are talking about the potential for adversarial images to be sent to users. If they were instead injected into the database itself (either by poisoning…

Because they're not CSAM? People don't seem to grasp what kind of images end up in the CSAM databases. They are most definitely not "leaked celebrity nude selfie" level stuff. Think of the most vile sexual thing you could do to a child and then times that by two and halve the child's age in your mind. That's the shit that gets in there. It's not something even 4chan weebaboos share. It's stuff that makes Liveleak reg…

The issue is not that a celebrity photo might get into a CSAM database. It's that a true CSAM photo, which has been modified to have the same hash as a "leaked celebrity nude selfie", will probably get into a CSAM database.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#123

Post by foxfluff » Wed, Aug 25, 2021, 12:52 PM UTC

My take on this is that the system is by and large useless.

It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images, or find a way to obfuscate their images enough to bypass the system (the lower the false positive rate, the easier it must be to trick the system).

So what's left when all the criminals this is supposed to catch have figured it out?

False positives. Only false positives.

Is it really worth turning personal devices into snitches that don't even do a good job of protecting children?

Also, numbers about false positives must be taken with a grain of salt because of the non-uniform distribution of perceptual hashes. It might be that your random vacation photos and kitty pics have a 1-in-a-million chance of a fapo, but someone who happens to (say) live in an apartment that has been laid out very similarly to a scene in pictures appearing in the CSAM database may have a massively higher chance of fapos for photos taken in their home.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#124

Post by Retric » Wed, Aug 25, 2021, 12:55 PM UTC

foxfluff wrote: post #123

My take on this is that the system is by and large useless. It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images, or find a way to obfuscate their images enough to bypass the system (the lower the false positive rate, the easier it must be to trick the system). So what's left when all the criminals this is supposed to catch hav…

Perceptual hashes are only used to reduce the search space for human review. Apple doesn’t have images in the CSAM database to do a comparison, but if it’s just a picture of a door their going to reject it. Also, because human review is an expense Apple’s incentives are to minimize the number of times it happens, thus the requirement for multiple collisions.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#125

Post by simion314 » Wed, Aug 25, 2021, 12:59 PM UTC

theshrike79 wrote: post #110

simion314 wrote: post #35

Earlier quoted context omitted.

But if Apple really cares about children why they did not done this scans in iCloud like all the others? Did not care as much as Google or Facebook? Seems to me like Apple does not care at all and seems more like a dev with big ego wanted to add neural hashes to his CV but if you can explain how Apple cared for children all this years but only now are doing something I really want to see the explanation

Because Apple is the one company that actively tries not to know anything about you. FB and Google will exhaustively analyse every single facet of your online presence and use your pictures to train their ML models for face detection and object detection. Apple, on the other hand, even explicitly splits Map directions to segments so that they can't know where you left from and where you are going to.

Apple was sending unencrypted on the network what application you started, if they would "actively tries not to know anything about you" they could have implemented this better.

Anyway how is your assumption make sense , Apple cares about children and about your privacy so scanning your images in iCloud was wrong until 2021 when something changed, what changed? does Apple cares more about children starting from now or they care less about privacy? or are they forced to do it?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#126

Post by iamtheworstdev » Wed, Aug 25, 2021, 1:01 PM UTC

monkeynotes wrote: post #109

ajb wrote: post #97

Earlier quoted context omitted.

If you can get exact collisions, this can be gamed. For example, suppose there are two rival gangsters. One wants to set the police on his rival. He knows that a certain (innocuous) image is on his rival's phone. So he pays someone to generate a fake child-porn image with the same neuralhash, and ensure that it gets into the child porn DB. Then, apple reports the rival to the police, and they come and investigate him…

I think before a criminal investigation, or any investigation at all is pursued, a human verifying the images would dismiss the false positive. I would think surreptitiously placing actual child porn on a rival's phone/computer would be much, much more effective. Cybercriminals could likely do all this remotely. Phish for apple account login, upload images. Done.

you think the US government would respect the law before initiation of actions to take down a criminal? they rarely do that when dealing with people that dont have criminal backgrounds

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#127

Post by oauea » Wed, Aug 25, 2021, 1:03 PM UTC

StrLght wrote: post #75

I don't really get what this repository is trying to achieve and what's the point of collecting collisions. Collisions will happen, that's just how it is with hashes. It's already a public knowledge that Apple has 2 more systems (some server-side verification and a manual check later) to prevent false-positives. So what's the point of researching collisions in NeuralHash?

Are you fine with an apple employee looking at all your private pictures just because some hashing algorithm decided you're a pedophile? Personally, I'm not.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#128

Post by SiempreViernes » Wed, Aug 25, 2021, 1:05 PM UTC

FabHK wrote: post #111

dTal wrote: post #44

Earlier quoted context omitted.

I think you missed the point of the first paragraph. The point is that you can now hide child porn by making its hash collide with innocent images. They won't ever make it to manual review. Ergo, NeuralHash is now useless.

So, the presumed attack (not against individuals, but to defeat the system) is 1. Identify some innocuous pictures that many many people have (memes, Beyoncé, whatever). 2. Produce CSAM. 3. Mangle it such that it is still CSAM visually, but NeuralHash-collides with the innocuous pictures from step 1. 4. Distribute. 5. Wait until they are (via some other mechanism) a) identified as CSAM, b) added to the NCMEC database…

There's also the _very_ important assumption that Apple doesn't check for collisions with popular images before adding new things to the database.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#129

Post by ithkuil » Wed, Aug 25, 2021, 1:07 PM UTC

PolCPP wrote: post #106

ithkuil wrote: post #24

Earlier quoted context omitted.

Let's not forget what the alternative is: this is about images that are uploaded on icloud anyway. The alternative is to upload the image in clear (or with ane encryption key that apple controls), and let apple run the CSAM filter on their servers. Apple now has the ability to encrypt the images before sending them to icloud, with a private key you own. Except that some percentage of images that match the CSAM finger…

Theres not even a need to do that, upload the encrypted image and its neuralhash.

What should they do if the neuralhash matches CSAM? Should they trust that the nerualhash actually matches real CSAM and you will should be reported to the police? That's clearly wrong, since there are going to be false positives, by design.

The whole point for this is to be a probabilistic filter, so that they need to run the real CSAM scanner on a subset of files.

You can fall into two camps:

a) apple should never ever scan my private images I upload on their cloud. b) apple can scan the images once they reach their servers.

If you pick (a), then clearly neuralhash shouldn't exist and you can argue against that on the ground that you want utter privacy. But you have to be consistent:every other cloud service that does scan the images server side should receive the same critique.

If you pick (b), then you must recognize that this additional machinery doesn't increase their reach to your private data, but quite the opposite, it allows them to implement e2e encryption for 99.9% of your content. You may argue that it's unnecessary and confusing and spooky and be afraid of the slippery slope precendent for other uses.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#130

Post by BeefWellington » Wed, Aug 25, 2021, 1:07 PM UTC

simondotau wrote: post #26

read_if_gay_ wrote: post #19

Earlier quoted context omitted.

The “actual argument” against the system that this provides is that Apple lied about the likelihood of hash collisions. Therefore, why trust any of their other claims?

Apple's claims are based on the statistical likelihood that there would be 30 collisions with CSAM hashes within one user account. Just because someone has found an image of a nearly featureless diagonal thing which collides with another image of a nearly featureless diagonal thing, that doesn't disprove Apple's claims.

> Apple's claims are based on the statistical likelihood that there would be 30 collisions with CSAM hashes within one user account.

Given people can now generate images that collide, it seems like the statistical likelihood has drastically changed since it was originally announced.