Live data from Hacker News

The deceptive PR behind Apple’s “expanded protections for children”

piotr.is

421–430 of 595 posts

Re: The deceptive PR behind Apple’s “expanded protections for children”

#422

Earlier quoted context omitted.

Then where are the news reports or articles of these false positives that would have shown up within the past decade? That's how long these companies have been using PhotoDNA on the server side. And the version of PhotoDNA from ten years ago would probably have been inferior to the version in place now. Is there even a single verifiable report of such a false positive? I feel that with the amount of attention brought…

> Then where are the news reports or articles of these false positives that would have shown up within the past decade?... Is there even a single verifiable report of such a false positive? I feel that with the amount of attention brought to this issue, if there was such a report then it probably would have been brought up by now. Positives get reviewed by humans, at which point false positives are identified and dis…

The privacy violation here is smaller than the one we all used to have to go through to get photos developed at all.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#423

Earlier quoted context omitted.

NCMEC is not exactly a normal NGO; it was opened with Reagan present and Congress frequently gives it funding. It also works with other government organizations on a frequent basis.

NCMEC isn't really an NGO; it is an agent of the US government, written in to US law.

Except it's not a government agency; it's a private non-profit with strong governmental ties.

https://www.missingkids.org/blog/2020/four-ncmec-myths

Re: The deceptive PR behind Apple’s “expanded protections for children”

#424
post #245

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

How do we know the database itself does not have any false positives?

Because it has been vetted by trustworthy FBI agents who don't make any mistakes.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#425
Apple implemented a backdoor that scans your photos on your device, then alerts Apple and the authorities if there is a match against an un-auditable list of reference photos.

Currently it's been activated for CSAM only and only scans photos backed up to iCloud.

That's the framing I prefer and which much better explains the issue with it.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#426

Whoever controls the hash list controls your phone from now on. Period. End of sentence. Apple has not disclosed who gets to add new hashes to the list of CSAM hashes or what the process is to add new hashes. Do different countries have different hash lists? Because if the FBI or CIA or CCCP or KSA wants to arrest you, all they need to do is inject the hash of one of your photos into the “list” and you will be flagge…

There are numerous incorrect statements in your comment. First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safety organizations. https://twitter.com/AlexMartin/status/1424703642913935374/ph... Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists. Second: Why can't the FBI / CCCP inject a hash into the list. Here's a tweet thread…

You've done nothing to address OP's concerns. The linked twitter thread assumes each actor (NCMEC, FBI, Apple) act in a certain way. There's no "provable" guarantee against an actor acting in bad faith or in a manner inconsistent with certain interpretations of the law (which we've seen routinely with the NSA).

The FBI/NSA can absolutely inject something into the hash list. You're assuming that NCMEC needs to be involved. Or that it would be broadly known to Apple. The reality is that the hash list needs to be updated on a different cadence than iOS itself. So it's likely downloaded rather than baked into the OS build permanently. That means that you can't necessarily rely on an iOS build being signed to know if you have a different hash list from everyone else. Ultimately, a small team at Apple cooperating with a secret court order could release a different hashlist to a select set of devices. There's nothing really stopping that.

Even if Apple didn't comply, we've seen recently how sophisticated cybersecurity companies armed with zero days can manipulate devices easily. If the mechanism for hash lists scanning the device is already built in all it takes is an exploit changing the hashlist and where it reports to which might be much simpler than gaining full access to the device.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#427
post #220

Earlier quoted context omitted.

I think most people don't upload to facebook pictures of their kids taking a bath? But they more than likely store such pictures on their phones/laptops.

Sure, but none of those images will be a hash match to any material in NCMEC databases.

What if they decide next year to use AI to automatically detect potential violating images?

I hope you won't throw the "slippery slope is a fallacy" fallacy at me.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#428

Earlier quoted context omitted.

I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Things may work right now in 2021. Things will always be changing. New hashes will be introduced. New code will be introduce. New laws in different countries will be introduced. Now that Apple has introduced this technology that no other phone manufacturer has, it can and will be changed to decrease p…

> I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Speak for yourself, because that didn't surprise me at all. When your corpus of "copyrighted" material is so utterly massive and almost entirely devoid of defined rules or boundaries, this kind of error is inevitable. If anything I'm more surprised that we haven't seen even more of these kinds of ma…

You completely bypassed the point of my post. It doesn't matter about what happens now. What matters is in the future. If China creates a law saying that not only should this system work for CSAM but also for objectionable material or anti-government material, is Apple really going to say no if it means billions of dollars in losses and Apple execs being targets by the CCP? Of course they won't.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#429

Earlier quoted context omitted.

Apple’s whole marketing scheme was that they protected privacy. That was their market differentiation. So no, it wasn’t like this before and this is why so many people like myself feel betrayed by Apple. This is what most of the outrage is about. The largest most profitable company in the world was selling their products saying they believed in privacy and now created a Trojan horse wrapped around CSAM that can be us…

If they were going to implement a government scanning tool it could be done much more simply. And they would if legally required, or turn down some of their largest markets. This is a step too far (on-device vs in cloud) but all of the “but what if the govt…” is ridiculous because that happens anyway if the govt wants it. Maybe we should change the government.

You point that it "could have" been done more simply is completely irrelevant.

This "much more simply" mechanism is exactly what was created by Apple. It's done right now, and there's no need to worry about how much more simply they COULD HAVE.

It's a fait-accompli. The government wanted it, and now they have it. Now they can scan individual phones for whatever they want.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#430

Earlier quoted context omitted.

There are numerous incorrect statements in your comment. First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safety organizations. https://twitter.com/AlexMartin/status/1424703642913935374/ph... Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists. Second: Why can't the FBI / CCCP inject a hash into the list. Here's a tweet thread…

You've done nothing to address OP's concerns. The linked twitter thread assumes each actor (NCMEC, FBI, Apple) act in a certain way. There's no "provable" guarantee against an actor acting in bad faith or in a manner inconsistent with certain interpretations of the law (which we've seen routinely with the NSA). The FBI/NSA can absolutely inject something into the hash list. You're assuming that NCMEC needs to be invo…

It's worse than that. They don't even need to release a different hash list for you, all they need to do is add a few images they know you'll probably have (say from your Facebook or Instagram posts) to the regular DB to meet the match threshold. The people running the database aren't going to be continually going back through old images to double-check they're actually CSAM/theoretically CSAM-related.
Post reply on HN