A catalog of naturally occurring images whose Apple NeuralHash is identical • Hacker News Board

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#241

Post by rgovostes » Wed, Aug 25, 2021, 5:47 PM UTC

tsimionescu wrote: post #5

toxik wrote: post #3

Sigh, for the last time, it doesn't actually matter if the NeuralHash is identical. You need multiple images matching, and then the images are compared by another system on Apple's end, which you don't know anything about. The system is specifically designed so that colliding images does not pose a threat to the user. NeuralHash and the CSAM scanning is grotesque, but please, criticize it for what it is, not some bul…

Then let's get rid of the NeuralHash entirely, if it doesn't matter, right? If it's a critical part of the system, then it should be inspected thoroughly. If Apple claims a minuscule chance of a hash collision, and the reality is that collisions are relatively common, that significantly changes the requirements for the backend system, which Apple keeps secret. We have every right to believe, bbased oon ppublic info,…

The conclusion section of the article associated with the GitHub repo linked here is that collisions are not common and Apple’s published collision probability matches their findings. Furthermore the thresholding scheme requires 30+ independent collisions which is astronomically improbable.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#242

Post by nullc » Wed, Aug 25, 2021, 5:49 PM UTC

tzs wrote: post #157

foxfluff wrote: post #123

My take on this is that the system is by and large useless. It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images, or find a way to obfuscate their images enough to bypass the system (the lower the false positive rate, the easier it must be to trick the system). So what's left when all the criminals this is supposed to catch hav…

> It won't catch anything but the dumbest of dumb criminals Dumb is a pretty accurate description of a large fraction of criminals. For the most part you only get smart criminals when you are talking about crimes where you have to be smart to even plan and carry out the crime.

Yes, but when you admit that the target is just the dumb criminals, then why adopt a scheme that has false positives?

Decompress and downsample. Drop the least significant bit or two, maybe do it in the dct domain instead. SHA256. It'll preserve matching for at least some cases of recompression and downsampling. But finding an unrelated image that matches is as hard as attacking SHA256, the only false positives that could be found would be from erroneous database entries.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#243

Post by burnished » Wed, Aug 25, 2021, 5:52 PM UTC

sroussey wrote: post #219

woofie11 wrote: post #216

Earlier quoted context omitted.

I don't really want my family photos reviewed by strangers. "Reducing the search space" of photos on my phone isn't an outcome I want to live with. At the time someone is looking at photos of my, my wife/husband/girlfriend/boyfriend, and my kids, they'd better have a darned good reason (e.g. a search warrant). I'd also appreciate if Apple let me know if my false positives were reviewed and found to not be CASM.

Don’t upload an image anywhere, else it can be reviewed.

I think the issue is that the content review is happening on phone, and would be a small change to go from scanning uploaded photos to all photos

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#244

Post by nullc » Wed, Aug 25, 2021, 5:52 PM UTC

Retric wrote: post #124

foxfluff wrote: post #123

My take on this is that the system is by and large useless. It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images, or find a way to obfuscate their images enough to bypass the system (the lower the false positive rate, the easier it must be to trick the system). So what's left when all the criminals this is supposed to catch hav…

Perceptual hashes are only used to reduce the search space for human review. Apple doesn’t have images in the CSAM database to do a comparison, but if it’s just a picture of a door their going to reject it. Also, because human review is an expense Apple’s incentives are to minimize the number of times it happens, thus the requirement for multiple collisions.

> Perceptual hashes are only used to reduce the search space for human review.

False. The Apple proposed system leaks the cryptographic keys needed to decode the images conditional on the match (threshold of matches) of the faulty neuralhash perceptual hash.

Matching these hashes results in otherwise encrypted highly confidential data being decodable by apple, accessable on their servers to the relevant staff along with anyone who compromises them or coerces them.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#245

Post by nullc » Wed, Aug 25, 2021, 5:55 PM UTC

mayoff wrote: post #200

foxfluff wrote: post #123

My take on this is that the system is by and large useless. It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images, or find a way to obfuscate their images enough to bypass the system (the lower the false positive rate, the easier it must be to trick the system). So what's left when all the criminals this is supposed to catch hav…

> It won't catch anything but the dumbest of dumb criminals, because those who care about CSAM can surely figure out a better way to share images Apparently that better way is by using Facebook. Facebook made 20.3 million reports to NCMEC in 2020. https://www.missingkids.org/content/dam/missingkids/gethelp/...

> Facebook made 20.3 million reports to NCMEC in 2020.

Which appears to have resulted in what... 5 prosecutions?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#246

Post by nullc » Wed, Aug 25, 2021, 5:56 PM UTC

colejohnson66 wrote: post #196

jrockway wrote: post #185

Earlier quoted context omitted.

Apple could also encrypt every upload to iCloud, and not have any scanning on the client, and still be able to say to the government "sure, you can have the files; we can't read them and neither can you". Apple wants to reduce your privacy from the government above and beyond what the law requires. The questions is: why?

The simple answer is: password resets. I’m sure majority of people would be very upset if they lost everything by forgetting a password.

Doesn't the apple backup stuff work that way? The data is gone forever if you lose your password?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#247

Post by nullc » Wed, Aug 25, 2021, 5:59 PM UTC

wyager wrote: post #165

simondotau wrote: post #70

Earlier quoted context omitted.

There is an interesting constitutional quirk which arises from the scanning being done client side, specifically for US citizens. If the US Government forced Apple to add other entries to the hash table, this would constitute a warrantless Government search of the private physical property of US citizens. This is a clear-cut, unambiguous breach of the 4th Amendment. Whereas if the CSAM scanning was performed exclusiv…

This is already a warrantless search that’s effectively controlled by the government. Obviously there’s enough chaff in the air to prevent that from being legally useful in any way.

So far the courts have determined that since providers invade their customer privacy of their own free will with no incentive or coersion by any government agency, that it is not a search by the goverment.

It's just your friendly trillion dollar tech company putting on a mask and cape and engaging in a bit of vigilante fun. You know? Like batman! ( https://www.youtube.com/watch?v=Kr7AONv3FSg )

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#248

Post by nullc » Wed, Aug 25, 2021, 6:04 PM UTC

zepto wrote: post #233

Enginerrrd wrote: post #201

Earlier quoted context omitted.

Since the govt. supplies the hashes in an unauditable way, it absolutely is controlled by the government. What's to stop them from using hashes of non CSAM material?

The government doesn’t supply the hashes in an unauditable way, that is a totally false statement. The hashes are supplied by NCMEC, a non-profit which is auditable, not a secret government agency. In any case, even if a non-CSAM hash were somehow in the database, Apple reviews the images before making reports, and those reports are used in normal criminal prosecutions.

Courts have determined that for this purpose the NCMEC is an agent of the government. NEMEC is 99% funded by the government and its ability to handle child porn is directly deprived from a explicit legislative carveout for them by name. What they do would be a felony for you or I to do. The fact that they are technically non-profit rather than an agency makes them significantly less accountable to the public. We cannot FOIA their communications, their composition isn't subject to public review, we cannot vote them out. And we have no way to tell what their database contains, nor is there any avenue for redress should we somehow learn of an inappropriate listing.

To the extent that you can say that they're not exactly a government agency, they absolutely have been deputized by the government.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#249

Post by bigwavedave » Wed, Aug 25, 2021, 6:06 PM UTC

snowwrestler wrote: post #227

tgv wrote: post #160

Earlier quoted context omitted.

> they have manual operators to check each of these photos For now. But what will happen when there are thousands of false positives per day? Will they increase the staff? Or will they add another algorithmic layer? Or just up the threshold a bit? There's no guarantee. The only thing that's certain is that the NeuralHash doesn't inspire confidence.

Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. At most you could maybe temporarily lock someone’s iCloud account. But again, the collisions would need to be multiple and all look like CSAM at reduced resolution. In general, it se…

> Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible.

A quick search found four clear cases where law enforcement has favored technological false positives over evidence:

Ousmane Bah: https://www.businessinsider.com/teen-sues-apple-1-billion-fa...

Robert Williams: https://www.cbsnews.com/news/facial-recognition-60-minutes-2...

Nijeer Parks: https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-reco...

Michael Oliver: https://www.dailydot.com/debug/detroit-facial-recognition-wr...

The one on Ousmane Bah really frustrates me- not only was he on a date at prom during the theft, he was in another state! "Nothing to hide" does not mean "nothing to fear" and allegations (even false ones) of possessing CSAM will ruin lives.

At the end of the day, what it really comes down to is trust; personally, I do not have enough faith in due process to not ruin innocent people. But I'm just some guy online, I will readily admit I don't know anything about anything.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#250

Post by nullc » Wed, Aug 25, 2021, 6:07 PM UTC

scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

> They are not intended to be compared exactly.

Apple's private set intersection which leaks the keys to decrypt the images coniditional a neuralhash match requires an exact match.

They probably didn't realize they got different results on different toolchains/devices, since they target a mono-culture and the whole subsystem shows fairly little careful thought went into it. They could easily make an exact integerized version which would be consistent.

It would still be broken. :)