The deceptive PR behind Apple’s “expanded protections for children” • Hacker News Board

Re: The deceptive PR behind Apple’s “expanded protections for children”

#321

Post by farmerstan » Thu, Aug 12, 2021, 11:53 PM UTC

cucumb3rrelish wrote: post #313

farmerstan wrote: post #241

Earlier quoted context omitted.

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there. How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it ge…

By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?

If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#322

Post by FabHK » Thu, Aug 12, 2021, 11:54 PM UTC

XorNot wrote: post #292

FabHK wrote: post #223

Earlier quoted context omitted.

> how could give a guarantee that a system that finds any disguised/distorted CSAM won't tag a random baby picture with a similar appearance. Cannot guarantee, but by choosing a sufficiently high threshold, you can make the probability of that happening arbitrarily small. And then you have human review. > And given such collision, the picture might be looked at by Apple and maybe law enforcement No, not "the picture"…

> "visual derivative". Do you have any idea that means? Because I certainly don't - how could you possibly identify whether an image is CSAM without looking at something which is reasonably the same image? What is a visual derivative? Take that algorithm and run it over some normal images and show me what they look like. All of this is being aggressively talked around because everyone knows it's not going to stand up…

> Do you have any idea that means?

No, I don't know what that means. Presumably it is some sort of thumbnail, maybe color inverted or something.

> does Apple's implementation flag on any of those? Who knows - they're not going to refer to anything specific about how they got "1 in a trillion"

I assume they've tested NeuralHash on big datasets of innocuous pictures, and gotten some sort of bound on the probability of false positives p, and then chosen N such that p^N << 10^-12, and furthermore imposed some condition on the "distance" between offending images (to ensure some semblance of independence). At least that's what I'd do after thinking about the problem for a minute.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#323

Post by nabakin » Thu, Aug 12, 2021, 11:56 PM UTC

FabHK wrote: post #314

lixtra wrote: post #231

Earlier quoted context omitted.

The article claims that photoDNA is reversible to 26x26 images and claims that the hashes are therefore CP.

Ah, right. But Apple is using NeuralHash, not photoDNA, right? Does that suffer the same problem?

We don't know but I think there's a good chance it does. It's important to determine before release. Maybe it's possible to encrypt them in such a way, no one can access them despite being on the device and still be able to use them for comparison

Re: The deceptive PR behind Apple’s “expanded protections for children”

#324

Post by _carbyau_ » Thu, Aug 12, 2021, 11:57 PM UTC

andrei_says_ wrote: post #216

fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

Who looks at the photos in that database? How do we know it is a trustworthy source? That it doesn’t contain photos of let’s say activists or other people of interest unrelated to its projected use?

Don't forget staff rotation. A database can live forever. You not only have to trust what is now, but what the future leaders of Apple may do, what all the future image viewers may do and who they are...

This is a pandaora's box of trust. Once you open it, you have to trust in perpetuity.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#325

Post by acdha » Thu, Aug 12, 2021, 11:58 PM UTC

shapefrog wrote: post #99

acdha wrote: post #33

Earlier quoted context omitted.

> Gruber practically (no, perhaps actually) worships Apple. He'd welcome Big Brother into his house if it came with an Apple logo, and he'd tell us how we were all wrong for distrusting it. You mean the same Gruber who described the situation as “justifiably, receiving intense scrutiny from privacy advocates.”? The one who said “this slippery-slope argument is a legitimate concern”? I'm having a hard time reconciling…

If icloud backup works as advertised - it backs up your device. However, if we consider the slipery slope, under pressure from a shaddow government, the contents of your phone could have been uploaded to the CIA every day, including live recordings 24 hours a day.

> the contents of your phone could have been uploaded to the CIA every day, including live recordings 24 hours a day.

You’re describing new functionality which would have to be added in many places: in addition to building that service they have to turn off the recording indicators and prompts, coexist with other apps recording, not have recording pause playback in other apps like normal, masking data usage on both the phone and your carrier’s reports, concealing the battery loss and putting in bigger hardware batteries to compensate, etc.

That’s technically possible but there’s no link to this feature - that hypothetical government would need to do the same things either way. It’s similarly not Apple-specific: with that level of control the same thing would happen Android and anything else.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#326

Post by farmerstan » Thu, Aug 12, 2021, 11:58 PM UTC

stetrain wrote: post #312

farmerstan wrote: post #241

Earlier quoted context omitted.

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there. How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it ge…

If you don’t trust what Apple says about this, why even argue? Apple could be doing all of this and more without telling. I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.

There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#327

Post by noobermin » Thu, Aug 12, 2021, 11:58 PM UTC

stickfigure wrote: post #213

fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

The false positive rate for any given image is not 1 in a trillion. Perceptual hashing just does not work like that. It also suffers from the birthday paradox problem - as the database expands, and the total number of pictures expands, collisions become more likely. The parent poster does make the mistake of assuming that other pictures of kids will likely cause false positives. Anything could trigger a false positiv…

People really don't know about the birthday paradox, don't they? Should be required reading any time someone claims a false positive rate is "1 in a trillion."

Re: The deceptive PR behind Apple’s “expanded protections for children”

#328

Post by slownews45 » Thu, Aug 12, 2021, 11:59 PM UTC

vineyardmike wrote: post #311

slownews45 wrote: post #258

Earlier quoted context omitted.

Almost every single one. Facebook checks for potential CSAM when you upload from your client device (sometimes an iphone) to their servers or after it's on their system and a user flags it. Instagram also check once you upload. These are all transmissions. Apple checks if you upload. If you don't upload or attempt to upload to their servers, no check. All these are to flag potential CSAM. Some do more - nudity in gen…

> Facebook checks for potential CSAM when you upload from your client device (sometimes an iphone) to their servers or after it's on their system and a user flags it. Thats not the issue according to the linked source. Instagram transmits all photos and assumes it's not CSAM until flagged - thats safe content. Apple transmits ONLY CSAM - thats a no-no content because they're assuming it is CSAM and you can't transmit…

Huh?

Apple uploads all photos to iCloud photos (just as google does). This includes CSAM and not CSAM.

It keeps a counter going of how much stuff is getting flagged as possible CSAM. They don't even get an alert about anything until you hit some thresholds etc. And no one has reviewed anything yet at all, the system is flagging things up as other systems do.

Are you sure (legally) that one can't review flags from a moderation system? That is routine currently. No one is knowingly doing anything. Their system discusses being alerted to possible counts of CSAM.

Is your goal that apple go straight to child porn reports automatically with no human involvement at all? At scale (billions of photos) that's going to be a fair number of false positives with potentially very serious consequences.

The current approach is that images are flagged in various ways, folks look at them (yes, a large number have problems), then next steps are taken. But the flags are treated as possible CSAM.

Please look into all the false positives in youtube screening before you jump from a flag => sure thing. These databases and systems are not perfect.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#329

Post by CommieBobDole » Fri, Aug 13, 2021, 12:00 AM UTC

vineyardmike wrote: post #293

fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

That 1/1t rate apple gave is post human review according to a recent interview

Do you have a link to that interview? That's a really big asterisk on the "one in a trillion" claim if true.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#330

Post by FabHK » Fri, Aug 13, 2021, 12:01 AM UTC

stickfigure wrote: post #213

fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

The false positive rate for any given image is not 1 in a trillion. Perceptual hashing just does not work like that. It also suffers from the birthday paradox problem - as the database expands, and the total number of pictures expands, collisions become more likely. The parent poster does make the mistake of assuming that other pictures of kids will likely cause false positives. Anything could trigger a false positiv…

> Anything could trigger a false positive

Don't you think Apple has independently arrived at that very same possibility? Maybe even thought about it and extensively tested it? Then put a probability on it? And then chosen the threshold such that the overall probability of falsely flagging an account is, indeed, minuscule?