Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The deceptive PR behind Apple’s “expanded protections for children”

piotr.is

Post reply on HN
Print view View original

331–340 of 595 posts

Re: The deceptive PR behind Apple’s “expanded protections for children”

#331

Post by simondotau »

samstave wrote: post #279
fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

Explain to me how photos get into the NCMEC database to begin with

I've absolutely no knowledge of how they operate, but it occurs to me that there would be at least two very obvious avenues:

1) During the course of investigation an officer infiltrates a CSAM sharing ring and/or poses as a customer for CSAM. Material is shared with the officer as it would be to an actual consumer of CSAM.

2) When someone is charged with child abuse, possession of child porn, etc, their physical and electronic lives will be methodically and forensically searched for CSAM material. They will likely find material they already know about, but potentially uncover new material and/or new social networks.

Any material acquired would need to be analysed and classified for the purpose of effective prosecution. My understanding is (from other comments made by people on other websites) that images in the NCMEC database are tagged based on the severity of their content and that Apple is only scanning for the most extreme "A1" material.

I wasn't sure what A1 meant so I googled it. According to this[0] PowerPoint presentation, page 22:

  A = prepubescent minor
  B = pubescent minor
  1 = sex act
  2 = "lascivious exhibition"
If you want to ruin your day, the PDF provides very specific—depressingly, grossly specific—definitions for the above.

[0] https://www.prosecutingattorneys.org/wp-content/uploads/Pres...

Re: The deceptive PR behind Apple’s “expanded protections for children”

#332

Post by nicce »

treebot wrote: post #319
nicce wrote: post #256

Earlier quoted context omitted.

I used to use Ubuntu for many years, but it became a such bloatware. So many things what you don’t really need. Packages were sometimes also different compared to vanilla Debian. This caused issues in stability (talking more about feature set). Some advanced software just did not work, which worked on equivalent vanilla Debian. I might recommend Ubuntu for very beginner developer, but not to stick with it longer time…

Are Mac and Windows not also full of an enormous amount of crap that we don't need?

That does sound quite bad, doesn’t it? That some Linux distributions are getting closer to them? We have a freedom, let’s use it. I’m using it for something minimal, like Arch Linux.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#333

Post by stetrain »

farmerstan wrote: post #326
stetrain wrote: post #312

Earlier quoted context omitted.

If you don’t trust what Apple says about this, why even argue? Apple could be doing all of this and more without telling. I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.

There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.

You don’t seem to have read what Apple has said on the issue so that feels a bit extreme.

And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option.

If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#334

Post by nonbirithm »

stickfigure wrote: post #213
fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

The false positive rate for any given image is not 1 in a trillion. Perceptual hashing just does not work like that. It also suffers from the birthday paradox problem - as the database expands, and the total number of pictures expands, collisions become more likely. The parent poster does make the mistake of assuming that other pictures of kids will likely cause false positives. Anything could trigger a false positiv…

Then where are the news reports or articles of these false positives that would have shown up within the past decade? That's how long these companies have been using PhotoDNA on the server side. And the version of PhotoDNA from ten years ago would probably have been inferior to the version in place now. Is there even a single verifiable report of such a false positive? I feel that with the amount of attention brought to this issue, if there was such a report then it probably would have been brought up by now.

Beyond that, assume that a false positive occurs and the innocent person is taken to court. Why would they have to fear being convicted if they don't actually hold any incriminating evidence? At most, that would become evidence against using perceptual hashing in future court cases.

The issue in that case is the violation of the innocent person's privacy, not that they have a risk of being falsely convicted. The courts would still need admissible evidence, and I don't believe that only having a perceptual hash and a set of legally photographed images clears that bar.

However, it becomes a completely separate issue if the false positives are "coincidentally" used to persecute marginalized groups in other countries where the same set of laws don't apply. But Apple has stated that they have no intention of expanding the system's scope to follow those laws. There isn't any evidence yet that Apple will do such a thing, or that they've already done it in the past. We are free to disbelieve them, but that's what they've stated. We can only hope that they won't change their minds.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#335

Post by squarefoot »

tehnub wrote: post #270
squarefoot wrote: post #259

This whole mess brought back a memory of when I was 4 to 5 years old (so probably 1971). During a summer vacation we were walking at a harbor in Tuscany with my parents and they told me suddenly I had to take a dump. Problem was that there was no bathroom nearby, well it probably was since the place was filled with restaurants, but we were like a hundred meters from the nearest one, which was incompatible with the su…

There will only be an alert if that photo is extremely similar to an image in the NCMEC database, AND there are numerous other such photos on the account that match. The threshold number of matches to trigger an alert is tuned for a 1/trillion chance of false positive. Furthermore, if you were using say Google Photos to store your images, then you were already subject to this vulnerability.

So what if a small circle of people produce their CSAM material by themselves and share only among themselves? None of the pictures is being uploaded to that database, so either the algorithms are really really good at recognizing them, or it will require human intervention, that is, scanning one by one all phones, then deciding which picture matches the criteria and write down the names of the people involved. I can't think of a similar scenario that doesn't imply the total loss of privacy by anyone even remotely linked to one of these people.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#336

Post by hnaccy »

fortenforge wrote: post #137
querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

Even if this is true (I don't trust apple's claim) I think this should still be used as a talking point to scare normal uninformed people about the surveillance tech.

The other side started it first. Look at the government's dubious claims about terrorism prevention. John Walsh a founder of NCMEC testified to congress that millions of children were abducted every year and that america was "littered with mutilated, decapitated, raped, strangled children," (this was and is not true).

If fear mongering about Big Brother throwing normal people in prison for pictures of their children is what it takes to blunt the expansion of the surveillance state I say fair play.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#337

Post by FabHK »

nabakin wrote: post #323
FabHK wrote: post #314

Earlier quoted context omitted.

Ah, right. But Apple is using NeuralHash, not photoDNA, right? Does that suffer the same problem?

We don't know but I think there's a good chance it does. It's important to determine before release. Maybe it's possible to encrypt them in such a way, no one can access them despite being on the device and still be able to use them for comparison

Yes, that's what the paper A Concrete-Security Analysis of the Apple PSI Protocol from UC San Diego claims:

> Reciprocally, the database of CSAM photos should not be made public or become known to the user. Apple has found a way to detect and report CSAM offenders while respecting these privacy constraints.

https://www.apple.com/child-safety/pdf/Alternative_Security_...

Re: The deceptive PR behind Apple’s “expanded protections for children”

#338

Post by kevinpet »

EugeneOZ wrote: post #141
kevinpet wrote: post #123

Earlier quoted context omitted.

What are those cases where they might be checked by humans? To determine whether it's an innocent baby bath? If you have naked photos of a partner which happen to hit a statistical match for certain patterns that are similar to CSAM? These aren't far fetched scenarios, these are exactly the most likely types of photos that would be likely flagged. Are you okay with those photos being passed around Apple's security re…

I’m not even sure if it's a joke or you are serious. It is a check against existing hashes in a big database of confirmed CSAM. What are the chances that photos of your partner are in that database? If your partner is older than 12 - it's 0%. Who is taking more risk to be sued for the leakage of the photos, you or Apple? The last part doesn't worth to be discussed because children in that DB are younger than 12.

I've now read up on NeuralHash a bit more, and while I think the idea that this is just a hash is slightly overstated, you're right and my above comment assumed this was a classifier rather than a perceptual hash.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#339

Post by grishka »

Barrin92 wrote: post #214

>The worst part is: how do I put my money where my mouth is? Am I going back to using Linux on the desktop (2022 will be the year of Linux on the desktop, remember) people really need to retire this meme. On the desktop in particular as a dev environment Linux is completely fine at this point. I can understand people not wanting to run a custom phone OS because that really is a ton of work but for working software de…

Good luck if you have two screens with different DPIs.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#340

Post by simondotau »

cdolan wrote: post #283
fortenforge wrote: post #246

Earlier quoted context omitted.

Apple has stated point-blank that the only source of CSAM content to generate the hash list will be NCMEC and other child safety organizations. While I fully admit that NCMEC could do a better job with transparency and auditing, they are currently being used by several other platforms right now (Facebook, Google, Microsoft) without issue. Could bad actors inject hashes of non-CSAM content into the database somehow? W…

Wait a second. You’re telling me that you believe a low-wage worker reviewing the worst of the worst human depravity, is going to stand up on a soap box and defend another nameless and faceless denizen of the Earth, when the crux of the argument is basically this: “Yeah I know the person tripped the safety threshold for CSAM, but these images aren’t that bad !” It’s not reasonable to trust the human reviewer. They pu…

This is a misunderstanding of the system. It's not a classifier, it's a hash. The only images that are going to be seen by this low-wage worker are going to be:

A) Images which are a correct hash match to an image already known to NCMEC or other agencies which have already been assigned CSAM category A1 (A=prepubescent, B=sex act);

B) Images which are a hash collision. According to Apple, the likelihood of a collision is 1 in 1 trillion per user account.

This system isn't a child detector strapped to a porn detector, being backed up by a low-wage worker making legal or editorial judgement calls. It's searching for images already known to child safety organisations—and even then only the most unambiguously horrific classification within the set of known images, far far far beyond the point where any ambiguity could possibly reside.

Post reply on HN