The Problem with Perceptual Hashes • Hacker News Board

Re: The Problem with Perceptual Hashes

#381

Post by ATsch » Sat, Aug 07, 2021, 6:16 PM UTC

ezoe wrote: post #110

The problem of hash or NN based matching is, the authority can avoid explaining the mismatch. Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good. Then, Apple report you have a file th…

The way I see it, this is the only possible purpose this system could have. With the press after this announcement, almost every single person in posession of those materials knows it's not safe to store them on an iPhone. By it's construction, this system can only be effective against things that the owner is not aware their phones are being searched for.

Re: The Problem with Perceptual Hashes

#382

Post by 0x426577617265 » Sat, Aug 07, 2021, 6:55 PM UTC

barsonme wrote: post #212

0x426577617265 wrote: post #197

Earlier quoted context omitted.

Couldn't the hack just be as simple as sending someone an iMessage with the images attached? Or somehow identify/modify non-illegal images to match the perceptual hash -- since it's not a cryptographic hash.

Does iCloud automatically upload iMessage attachments?

I think so, since the iMessages are synced across devices.

Re: The Problem with Perceptual Hashes

#383

Post by nullc » Sat, Aug 07, 2021, 7:01 PM UTC

ezoe wrote: post #110

The problem of hash or NN based matching is, the authority can avoid explaining the mismatch. Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good. Then, Apple report you have a file th…

> Demand the original intended file for the hash?

Even if they'd provide it-- the attacker need only perturb an image from an existing child abuse image database until it matches the target images.

Step 1. Find images associated with the race or political ideology that you would like to genocide and compute their perceptual hashes.

Step 2. Obtain a database of old widely circulated child porn. (Easy if you're a state actor, you already have it, otherwise presumably it's obtainable since if it wasn't none of this scanning would be needed).

Step 3. Scan for the nearest perceptual matches for the target images in the CP database. Then perturb the child porn images until they match (e.g. using adversarial noise).

Step 4. Put the modified child porn images into circulation.

Step 5. When these in-circulation images are added to the database the addition is entirely plausibly denyable.

Step 6. After rounding up the targets, even if they're allowed any due process at all you disallow them access to the images. If that dis-allowance fails, you can still cover by the images existing and their addition having been performed by someone totally ignorant of the scheme.

Re: The Problem with Perceptual Hashes

#384

Post by acdha » Sat, Aug 07, 2021, 7:04 PM UTC

t0mas88 wrote: post #270

new_realist wrote: post #52

Earlier quoted context omitted.

This is already possible using other services (Google Drive, gmail, Instagram, etc.) that already scan for CP.

Does Google scan all files you upload to them with an algorithm like the one now proposed? Or do they have only a list of exact (not perceptual) SHA hashes of files to flag on? The latter I think is also used for pirated movies etc being removed under DMCA?

Yes: it’s called PhotoDNA and is used by many, many services. See https://en.wikipedia.org/wiki/PhotoDNA

SHA hashes aren’t suitable for this: you can change a single bit in the header to bypass a hash check. Perceptual hashes are designed to survive cropping, rotation, scaling, and embedding but all of those things mean that false-positives become a concern. The real risk would be if someone figured out how to many plausibly innocent collisions where you could send someone a picture which wasn’t obviously contraband or highly suspicious and attempt to convince them to save it.

Re: The Problem with Perceptual Hashes

#385

Post by nullc » Sat, Aug 07, 2021, 7:07 PM UTC

nicce wrote: post #247

fogof wrote: post #155

Earlier quoted context omitted.

Well, presumably at that point, someone in that position would just reveal their own files with the hash an prove to the public that they weren't illegal. Sure, it would be shitty to be forced to reveal your private information that way, but you would expose a government agency as fabricating evidence and lying about the contents of the picture in question to falsely accuse someone. It seems like that would be a scan…

”Sorry, but collisions happen with all hashing algorithms, and you can’t prove otherwise. It is just a matter of time. Nothing to see here.”

In the past the FBI used some cryptographic hash. Collisions with a secure cryptographic hash are functionally unobservant in practice (or else the hash is broken).

The use of the perceptual hash is because some people might evade the cryptographic hash by making small modifications to the image. The fact that they'd discarded the protection of cryptographic hashing just to accommodate these extra matches is unsurprising because their behavior is largely unconstrained and unbalanced by competing factors like the public's right to privacy or your security against being subject to a false accusation.

Re: The Problem with Perceptual Hashes

#386

Post by acdha » Sat, Aug 07, 2021, 7:10 PM UTC

wizzwizz4 wrote: post #380

wpietri wrote: post #366

Earlier quoted context omitted.

Then perhaps you could explain it? I also don't understand why server-side versus client-side CSAM inspection makes a big difference.

If I ask you to store my images, and you therefore have access to the images, you can scan them for stuff using your computers . The scope is limited to the images I ask you to store, and your computers are doing what you ask them to. If you reprogram my computer to scan my images stored on my computer … different thing entirely. I don't have a problem with checking them for child abuse (in fact, I'd give up quite a…

So far, this is only for iCloud photos so currently it seems highly similar to what we have now except that it’s on the device and could be done with end to end encryption, unlike the current approach.

For me, the big concern is how it could be expanded. This is a real and valid problem but it’s certainly not hard to imagine a government insisting it needs to be expanded to cover all photos, even for people not using iCloud, and we’d like you to add these signatures from some images we can’t show you. Once the infrastructure is there it’s a lot easier to do that.

Re: The Problem with Perceptual Hashes

#387

Post by nullc » Sat, Aug 07, 2021, 7:12 PM UTC

judge2020 wrote: post #274

avnigo wrote: post #271

> These cases will be manually reviewed. That is, according to Apple, an Apple employee will then look at your (flagged) pictures. I'm surprised this hasn't gotten enough traction outside of tech news media. Remember the mass celebrity "hacking" of iCloud accounts a few years ago? I wonder how those celebrities would feel knowing that some of their photos may be falsely flagged and shown to other people. And that we…

They wouldn't be falsely flagged. It doesn't detect naked photos, it detects photos matching real confirmed CSAM based on the NCMEC's database.

If there were no false positives there would be no legitimate reason for Apple to review-- they would just be needlessly exposing their employees to child abuse material.

But the fact that there is no legitimate reason according to the system's design doesn't prevent there from being an illegitimate reason: Apple's "review" undermines your legal due process protection against warrantless search.

See US v. Ackerman (2016): The appeals court ruled that when AOL forwarded an email with an attachment whos hash matched the NCMEC database to law enforcement without anyone looking at it, and law enforcement looked at the email without obtaining a warrant was an unlawful search and had AOL looked at it first (which they can do by virtue of your agreement with them) and gone "yep, thats child porn" and reported it, it wouldn't have been an unlawful search.

Re: The Problem with Perceptual Hashes

#388

Post by l33t2328 » Sat, Aug 07, 2021, 7:13 PM UTC

jeromegv wrote: post #331

l33t2328 wrote: post #297

Earlier quoted context omitted.

I don’t see how the US is becoming “less stable” in any meaningful sense. Can you elaborate?

An attack on the capitol on January 6. A former president that spent weeks trying to delegitimize the election, trying to get people fired when they were just following the process to ratify the election, etc.

I’m struggling to see how a few people shitting on Pelosi’s desk and stealing a podium really changed the nature of American stability.

Re: The Problem with Perceptual Hashes

#389

Post by mirkules » Sat, Aug 07, 2021, 7:14 PM UTC

delusional wrote: post #272

ezoe wrote: post #110

The problem of hash or NN based matching is, the authority can avoid explaining the mismatch. Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good. Then, Apple report you have a file th…

What about trolling. Assume 4chan figures out apples algorithm. What now happens when they start generating memes that happen to match known child pornography? Will anyone who saves those memes (or repost them to reddit/facebook) be flagged? What will apple do once flagged false positive photos go viral?

One way this hair-brained Apple program could end is to constantly generate an abundance of false positives, and try to render it useless.

For those old enough to remember “Jam Echelon Day”, maybe it won’t have any effect. But what other recourse do we have other than to maliciously and intentionally subvert and break it?

Re: The Problem with Perceptual Hashes

#390

Post by l33t2328 » Sat, Aug 07, 2021, 7:20 PM UTC

gpm wrote: post #323

l33t2328 wrote: post #297

Earlier quoted context omitted.

I don’t see how the US is becoming “less stable” in any meaningful sense. Can you elaborate?

Both sides of the political spectrum think the other side is stupid, and evil. The gap between the two sides is getting bigger. Politicians and people (especially on the right, but to some extent on the left) are increasingly willing to cheat to remain in power. If you want some concrete examples: - Trump's attempted coup, the range of support it received, the lack of condemnation it received. - Law's allowing things…

Calling what Trump did an attempted coup is hyperbole beyond belief.

The support for packing the supreme court is mostly at the fringes of the party, and there’s always been some support.

There are almost no laws with any kind of support that have transparent goals of suppressing voters. Election security laws are clearly necessary after the doubt the democrats had it was secure in 2016, and the doubts the republicans had in 2020.

Laws absolving drivers of hitting protesters don’t exist. Laws absolving drivers of driving through violent rioters do, and such laws are necessary. I saw a a riot with my own eyes where a half dozen cars were flipped and destroyed, and anyone trying to drive through the intersection had people jumping on their car and smashing the windows. These laws are good.