The Problem with Perceptual Hashes • Hacker News Board

Re: The Problem with Perceptual Hashes

#341

Post by spacedcowboy » Sat, Aug 07, 2021, 2:50 PM UTC

avnigo wrote: post #285

judge2020 wrote: post #274

Earlier quoted context omitted.

They wouldn't be falsely flagged. It doesn't detect naked photos, it detects photos matching real confirmed CSAM based on the NCMEC's database.

The article posted, as well as many others we've seen recently, demonstrate that collisions are possible, and most likely inevitable with the number of photos to be scanned for iCloud, and Apple recognizes this themselves. It doesn't necessarily mean that all flagged photos would be of explicit content, but even if it's not, is Apple telling us that we should have no expectation of privacy for any photos uploaded to…

Apple already use the same algorithm on photos in email, because email is unencrypted. Last year Apple reported 265 cases according to the NYT. Facebook reported 20.3 million.

Devolving the job to the phone is a step to making things more private, not less. Apple don’t need to look at the photos on the server (and all cloud companies in the US are required to inspect photos for CSAM) if it can be done on the phone, removing one more roadblock for why end-to-end encryption hasn’t happened yet.

Re: The Problem with Perceptual Hashes

#342

Post by sunshinerag » Sat, Aug 07, 2021, 2:52 PM UTC

delusional wrote: post #272

ezoe wrote: post #110

The problem of hash or NN based matching is, the authority can avoid explaining the mismatch. Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good. Then, Apple report you have a file th…

What about trolling. Assume 4chan figures out apples algorithm. What now happens when they start generating memes that happen to match known child pornography? Will anyone who saves those memes (or repost them to reddit/facebook) be flagged? What will apple do once flagged false positive photos go viral?

>> Will anyone who saves those memes (or repost them to reddit/facebook) be flagged?

Shouldn't they be?

Re: The Problem with Perceptual Hashes

#343

Post by zekrioca » Sat, Aug 07, 2021, 3:05 PM UTC

jeromegv wrote: post #330

asimpletune wrote: post #105

Earlier quoted context omitted.

I have been a big Apple fan ever since my first computer. This is the first time I legitimately thought I need to start thinking about something else. It’s kind of sad.

Genuinely curious, why? This scanning was already happening server-side in your iCloud photos, just like Google Photos, etc. Now they are removing it from server-side to client-side (which still require this photo to be hosted in iCloud) What changed, really?

You answered your own question and still don’t get it.

Re: The Problem with Perceptual Hashes

#344

Post by goldenkey » Sat, Aug 07, 2021, 3:08 PM UTC

sobriquet9 wrote: post #303

CuriousCosmic wrote: post #266

Earlier quoted context omitted.

FYI Illiberal values aka non-liberal values (clarifying because the I is hard to read) use the word liberal in the traditional sense. Liberal values are liberty/freedom, consent of the governed, and equality before the law. All other liberal values build off of these three as a base. This implies that Non-liberal (or illiberal) values are the opposition of liberal values through censorship, gun control, etc like you…

Karl Marx quote on the right to keep and bear arms only applies to the proletariat. If you are a programmer and own the means of production (your laptop), you are not proletariat. All socialist and communist countries have strict gun control.

If you own AWS or GCP or Azure is a better example of owning means of production. A laptop cannot make you enough money to live by means of renting it out.

Re: The Problem with Perceptual Hashes

#345

Post by spacedcowboy » Sat, Aug 07, 2021, 3:14 PM UTC

siscia wrote: post #277

What I am missing from all this story, is what triggered Apple to put in place, or even think about, this system. It is clearly a no-trivial project, no other company is doing it, and it will be one of the rare case of a company doing something not for shareholders value but for "goodwill". I am really not understanding the reasoning behind this choice.

Er, every US company that hosts images in the cloud scans them for CSAM if they have access to the photo, otherwise they’re opening themselves up to a lawsuit.

US law requires any ESP (electronic service provider) to alert NCMEC if they become aware of CSAM on their servers. Apple used to comply with this by scanning images on the server in iCloud photos, and now they’re moving that to the device if that image is about to be uploaded to iCloud photos.

FWIW, the NYT says Apple reported 265 cases last year to NCMEC, and say Facebook reported 20.3 million. Google [1] are on for 365,319 for July->Dec.

I’m still struggling to see what has changed here, apart from people realising what’s been happening..

- it’s the same algorithm that Apple has been using, comparing NCMEC-provided hashes against photos

- it’s still only being done on photos that are uploaded to iCloud photos

- it’s now done on-device rather than on-server, which removes a roadblock to future e2e encryption on the server.

Seems the only real difference is perception.

[1] https://transparencyreport.google.com/child-sexual-abuse-mat...

Re: The Problem with Perceptual Hashes

#346

Post by spacedcowboy » Sat, Aug 07, 2021, 3:18 PM UTC

mattnewton wrote: post #94

emodendroket wrote: post #34

Earlier quoted context omitted.

And what do you think the content moderation teams employed by Facebook, YouTube, et al. do all day?

There's a big difference in the expectation of privacy between what someone posts on "Facebook, Youtube, et al" and what someone takes a picture of but doesn't share.

Odd, then, that Facebook reported 20.3 million photos to NCMEC last year, and Apple 265, according to the NYT that is.

Re: The Problem with Perceptual Hashes

#347

Post by stetrain » Sat, Aug 07, 2021, 3:22 PM UTC

Klonoar wrote: post #220

stetrain wrote: post #193

Earlier quoted context omitted.

But the barrier between “only happens for iCloud” and “happens for all photos on device” has been reduced to a very small barrier. Before it was the photos actually being sent to a separate server by my choice, now it’s Apple saying their on-device tool only runs given criteria X. And on a second note I think people are allowed to be freshly concerned at the idea of Apple scanning photo libraries given a government-p…

To be clear, I have no qualms about people being concerned. You can find my comments elsewhere on this site that I think people should scrutinize this entire thing. I'm just very tired of people (not necessarily you) spouting off as if the functionality is new . It dilutes an otherwise important conversation. So many of the threads on this site are just people privacy LARPing.

Agreed. I still think there is a distinction, even if only in principle and mostly psychological, between what a company does with my files on their server, and what they do with my files on my device.

Even if the outcome is theoretically the same, the means are different and it feels different.

Re: The Problem with Perceptual Hashes

#348

Post by stetrain » Sat, Aug 07, 2021, 3:22 PM UTC

bigiain wrote: post #229

stetrain wrote: post #215

Earlier quoted context omitted.

I’m pretty sure lots of mail gets x-rayed, perhaps even more looking for malicious packages or substances. I agree that data content scanning is more invasive than physical scanning. It was an intentionally simplistic example not meant to defend Apple.

Parcels, maybe. I’d bet it’s a tiny percentage though. I doubt the entire world has enough X-ray machines to scan even a vanishingly small percentage of the envelopes the postal service delivers every day.

Sorry my metaphor wasn’t good enough.

Re: The Problem with Perceptual Hashes

#349

Post by karmakaze » Sat, Aug 07, 2021, 3:27 PM UTC

jdavis703 wrote: post #62

karmakaze wrote: post #17

It really all comes down to if Apple has and is willing to maintain the effort of human evaluations prior to taking action on the potentially false positives: > According to Apple, a low number of positives (false or not) will not trigger an account to be flagged. But again, at these numbers, I believe you will still get too many situations where an account has multiple photos triggered as a false positive. (Apple sa…

Then law enforcement, a prosecutor and a jury would get involved. Hopefully law enforcement would be the first and final stage if it was merely the case that a person pressed “ok” by accident.

This is exactly the kind of thing that is to be avoided: premature escalation, tying up resources, increasing costs, and raising the stakes and probability of bad outcomes.

Re: The Problem with Perceptual Hashes

#350

Post by paulryanrogers » Sat, Aug 07, 2021, 3:30 PM UTC

sunshinerag wrote: post #342

delusional wrote: post #272

Earlier quoted context omitted.

What about trolling. Assume 4chan figures out apples algorithm. What now happens when they start generating memes that happen to match known child pornography? Will anyone who saves those memes (or repost them to reddit/facebook) be flagged? What will apple do once flagged false positive photos go viral?

>> Will anyone who saves those memes (or repost them to reddit/facebook) be flagged? Shouldn't they be?

Umm, no? If someone happens upon some funny cat meme that 4chan users made with an intentional hash collision then they're not guilty of anything.

A poor analogy could be trolls convincing a flash mob to dress like a suspect's description which they overheard with a police scanner. No one in the mob is guilty of anything more than poor fashion choice.