Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A catalog of naturally occurring images whose Apple NeuralHash is identical

github.com

Post reply on HN
Print view View original

131–140 of 304 posts

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#131

Post by jtsiskin »

saithound wrote: post #113
mlajtos wrote: post #102

Earlier quoted context omitted.

If I found one collision by accident, would that be any significant?

Yes, absolutely. It's technically possible to find SHA256 collisions accidentally, but it's so unlikely, that if you found one, it would merit serious investigation. People would not believe your statement that you found them accidentally, and "oh, I guess mlajtos really just found the colliding pair by chance" wouldn't be declared until after a very thorough investigation. In the meantime, major stakeholders (e.g. B…

A good perspective on how big the SHA-256 hash space is: https://youtu.be/S9JGmA5_unY

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#132

Post by laszlokorte »

simondotau wrote: post #47
dTal wrote: post #44

Earlier quoted context omitted.

I think you missed the point of the first paragraph. The point is that you can now hide child porn by making its hash collide with innocent images. They won't ever make it to manual review. Ergo, NeuralHash is now useless.

Why would anyone go to the effort of technical concealment[1] of CSAM when they could just resist the urge to import child porn in their phone's photo library in the first place? I've managed to resist the urge to import regular porn into my photo library, and being caught with regular porn is (at most) embarrassing. It's not potentially life-destroying. It's inconceivable that anyone could desire possession of NCMEC…

I agree with you in that I do not understand why anybody doing something illegal would upload related data to a cloud storage.

But if nobody would import CSAM into their icloud library why do all the pictures need to be scanned in the first place? I would imagine anybody doing major illegal stuff being informed about important measures in order to not be caught.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#133

Post by SiempreViernes »

saithound wrote: post #83
SiempreViernes wrote: post #77

Earlier quoted context omitted.

Yeah, of course the collision rate in an adversarial dataset is likely to be much higher. But I really wonder why you think this is an important objection, do you think a lot of people want to go to the "get flagged for child porn" casino?

The existence of a preimage attack makes Apple's system completely useless for its nominal purpose. The NeuralHash collider allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with perfectly innocent images. If these new images never make it to the NCMEC database, then new CSAM content will be completely NeuralHash-proof. How…

> If these new images never make it to the NCMEC database

The remark that an image might not be in the database applies equally to all CSAM pictures, and so is not actually dependent on any technical aspect of the hash at all.

The fact that you can modify an image to the point that it gets a different hash isn't important compared to the (separate) issue of how you keep your database up to date. And detecting old CSAM is in any case not as important as tracking and interdicting production of new images.

The second remark is good, but mainly because it reminds us to demand Apple ensures that additions are scanned for malicious images before being added to the central database. If the operator determines that a certain images collides with some known public image they can notify Apple. Apple in turn modifies the hasher to dissolve the collision, roll out a patch, and can then update the database.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#134

Post by rangerdan »

simondotau wrote: post #70
roody15 wrote: post #33

Apple has yet to make a valid reason for implementing client side CSAM scanning. According to Apple only images that will be uploaded to iCloud will be scanned. If this is the case there is zero reason to scan locally and you can just scan the uploaded image once it is on the server. Apple has not implemented E2E nor has it released a statement indicating this will be implemented in the future.

There is an interesting constitutional quirk which arises from the scanning being done client side, specifically for US citizens. If the US Government forced Apple to add other entries to the hash table, this would constitute a warrantless Government search of the private physical property of US citizens. This is a clear-cut, unambiguous breach of the 4th Amendment. Whereas if the CSAM scanning was performed exclusiv…

> If the US Government forced Apple to add other entries to the hash table, this would constitute a warrantless Government search of the private physical property of US citizens. This is a clear-cut, unambiguous breach of the 4th Amendment.

There's no reason not to assume this isn't already happening, being closed source and proprietary. The question to ask is, what are we going to do about it?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#135

Post by WindyLakeReturn »

SiempreViernes wrote: post #77
saithound wrote: post #59

Earlier quoted context omitted.

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the blog post linked from the repo [2], are both derived without making any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the practical false positive rate is now expected to be much much higher. Imagine that y…

Yeah, of course the collision rate in an adversarial dataset is likely to be much higher. But I really wonder why you think this is an important objection, do you think a lot of people want to go to the "get flagged for child porn" casino?

Themselves? No. Other people and communities that they dislike? You bet.

Just look at political memes. What is the chance that both sides will have a few people try to create memes for their opponents which are adversarial. They'll take care to make sure those images aren't on their own apple devices before spreading it to areas where the other side likes to share memes.

Another example are the very people trying to be caught. Someone who is against current laws about the subject might seek to create as many false positives as possible to overwhelm the system. They might even specifically target otherwise legal baby photos to manipulate in this way as it would make it more likely they get past any sort of second tier manual review and result in law enforcement wasting resources.

Lastly, this is nothing new. Planting such material and flooding websites with it has long been a tactic used by some. Up until now it has been limited because doing so requires violating the law yourself and few people hate others enough to go through that level risk to self. But this is mostly risk free because creating adversarial images like this isn't outright illegal and even in cases where there are laws against it violating those laws is extremely different than violating actual laws against CSAM in every way that factors into a person's willingness to break laws.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#137

Post by dathinab »

JoshTko wrote: post #136

The threshold of collisions Apple is using before review is 40

I think no one is anymore afraid of 40 accidental natural image collisions.

But un-natural image collisions or bad images in the database and similar are a different matter and had been the main critique point from the get to go as far as I can tell.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#138

Post by simondotau »

rangerdan wrote: post #134
simondotau wrote: post #70

Earlier quoted context omitted.

There is an interesting constitutional quirk which arises from the scanning being done client side, specifically for US citizens. If the US Government forced Apple to add other entries to the hash table, this would constitute a warrantless Government search of the private physical property of US citizens. This is a clear-cut, unambiguous breach of the 4th Amendment. Whereas if the CSAM scanning was performed exclusiv…

> If the US Government forced Apple to add other entries to the hash table, this would constitute a warrantless Government search of the private physical property of US citizens. This is a clear-cut, unambiguous breach of the 4th Amendment. There's no reason not to assume this isn't already happening, being closed source and proprietary. The question to ask is, what are we going to do about it?

If you take that line of argument, you must also accept that you have no reason not to assume that binary distributions of Android and Windows haven't been doing similar things for the past decade.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#139

Post by dathinab »

dathinab wrote: post #137
JoshTko wrote: post #136

The threshold of collisions Apple is using before review is 40

I think no one is anymore afraid of 40 accidental natural image collisions. But un-natural image collisions or bad images in the database and similar are a different matter and had been the main critique point from the get to go as far as I can tell.

Also given how many people use IPhones, how many pictures they have and how often they have many similar pictures, thinks are not necessary that simple.

I wouldn't be surprised if some flat, small height fully adult (e.g. 30) woman does some sexting and goes from 0 to >40 collisions in a month. Not because of arbitrary collisions but because the similarity some of here sexting pictures might have with the ones from a 14y old but older looking girl (which e.g. where forced and ended up in the database).

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#140

Post by dhosek »

StrLght wrote: post #75

I don't really get what this repository is trying to achieve and what's the point of collecting collisions. Collisions will happen, that's just how it is with hashes. It's already a public knowledge that Apple has 2 more systems (some server-side verification and a manual check later) to prevent false-positives. So what's the point of researching collisions in NeuralHash?

I'd argue that the hash collisions (both natural and synthetic) that I've seen give me more confidence in the system, not less.

On the natural hash collisions (of which there are two), we have objects of similar shape against a solid background. It seems that a natural hash collision of a CSAM image would be unlikely (or if it does occur, it would be something that perhaps is also an infringing image).

As for the synthetic hash collisions, there are visible artifacts in the picture that, if you compare with the original picture, make the overlay of the original picture on the synthetically generated hash collision obvious. Could people get tricked into downloading memes¹ with synthetically generated hash collisions? Sure, people are idiots. But I'm guessing the majority of folks will look at the picture and say, this is a sh*t picture in this meme and download something else.

1. And that, of course, assumes that meme hosters don't apply similar scanning techniques to what they serve up.

Post reply on HN