Live data from Hacker News

In internal memo, Apple addresses concerns around new Photo scanning features

9to5mac.com

371–380 of 430 posts

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#371

Earlier quoted context omitted.

Ambiguous retorts like this make you sound intelligent but offer little to the discussion. If the adversary is the USA or China, I have bad news for you: every major democracy has planned encryption regulation which is unimaginably worse than what was announced here.

Telling me what I already know is not really contributing much, either.

Actually, the discussion benefits from everyone stating their arguments clearly, even if you don’t benefit. Most of the discussion of this change has been FUD, making it difficult to tease apart actual privacy regression from imagined ones.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#372
post #92

Earlier quoted context omitted.

Just because the slippery slope argument is a logical fallacy doesn’t mean we can’t use our minds to foresee possible next steps.

The slippery slope argument isn't even really a logical fallacy. In fact, I can think of a number of circumstances where it turned out to be true. edit: here's an example of the slippery slope "fallacy" in action "First they came for the socialists, and I did not speak out— Because I was not a socialist."

>isn't even really a logical fallacy. In fact, I can think of a number of circumstances where it turned out to be true.

This represents a very common misunderstanding of fallacies, IIRC it's called the Fallacy Fallacy: fallacies don't give you any information about the truth of the conclusion, they only indicate that the logic that ties that conclusion to the premises is unsound. So "slippery slope fallacy" means "worse things don't logical follow in increments from better things" but that doesn't mean that in any real situation "worse things follow in increments from better things" is not true.

The most common example of this misunderstanding is probably Occam's Razor, it says nothing about whether things are or aren't more complex in any real situation, only that it's easy to reason about things if you don't add additional aspects that aren't needed.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#373

Earlier quoted context omitted.

If they can decrypt in a “specialized” situation, then they can decrypt in any situation. All that has to be done is to broaden the classifier step by step. Or someone else gets access to the back door. That’s why there can be zero allowed back doors.

They can decrypt iCloud content currently, so it wouldn't be a step back. On the topic of backdoors, automatic update systems could be used as backdoors.

Probably why every system update requires passcode input.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#374

Apple's mistake is that they seemingly believe there is pushback because people misunderstand how it works. The reality is more nuanced: People understand exactly how it works, and how it works is that it is turn-key onboard spyware, that Apple pinky-swears isn't being used wrong today . For example if the scope/mission expands (e.g. foreign governments), suddenly you've created a drag-net for whatever "badness" is o…

Yeah, this is my reading of the situation too. From a technical point of view, what Apple is doing seems like a reasonable approach to me; what I absolutely do not trust is any claim that it will never ever be expanded to include, say, governments requiring similar tech be applied to all messages before they are encrypted in order to look for dissidents… and then generally at opposition parties. (And of course there are enough competent technologists available for making custom encrypted chat apps that the hostile governments could only achieve their goals by putting the AI in the frame buffer, which opens the door to subliminal attacks against political enemies — targeting the AI with single frames such that the device user doesn’t even notice anything).

But I know I have paradoxical values here:

On the one hand, I regard abusive porn as evil. I want it stopped.

On the other, I have witnessed formerly-legal acts become illegal, and grew up in a place where there are sexual acts you can legally perform yet not legally possess images of. I think the governments (and people in general) follow a sense of disgust-minimisation, even when this is at the expense of harm-minimisation.

And any AI which can genuinely detect a category of image can also be used to generate novel examples that are also in that category without directly harming a real human.

I don’t have a strong grasp on what I expect the future to look like. My idea of what “the singularity” is, is that rather than a single point in the future where all tech gets invented at once, it’s an event horizon beyond which we can no longer make reasonably accurate predictions. I think this horizon is currently 2025-2035, and that tech is changing the landscape of what morality looks like so fast that I can’t see past that.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#375

The NCMEC director claimed "many thousands of sexually exploited victimized children will be rescued" because of this system. It can't detect new abuse images if I understand correctly. And don't most arrests happen after a victim talks to another adult? Or someone notices signs? Are many thousands of active child molesters discovered through scanning online photos?

Yes: https://law.justia.com/cases/california/court-of-appeal/2016... But also she’s a director at NCMEC. What else is she supposed to say? “Cool I guess, thanks Apple.”

The question was about many thousands. Not 1.

Is NCMEC's mission protecting children or self promotion? Facts are best for the 1st.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#376
post #369

It's odd that a "private" organization, the "National Center for Missing and Exploited Children" gets access to your photos first, before law enforcement. Why would we trust law enforcement to a posse of private citizens (albeit with a congressional charter)?

>Why would we trust law enforcement to a posse of private citizens (albeit with a congressional charter)? What they're doing isn't law enforcement. Charging people with crimes, arresting them and throwing them in jail is law enforcement.

How do they get an exemption from the "strict liability" of possession of illegal images?

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#377
post #22

Earlier quoted context omitted.

A lot of people not in government are also fiercely opposed to raping children and are willing to compromise, and understand that Apple can already do whatever they want to your phone, so this specific thing isn't the straw that breaks the camel's back. All the "what if Apple turns evil one day" applies equally well to iOS without this feature.

I believe a lot of the outrage brewing now against Apple’s CSAM scanning is misdirected, and might actually be hurting the larger cause. Most ordinary people, especially those who have kids, won’t have a problem with a well-implemented, E2E encryption compatible scheme that is legally limited to only apply to this type of material. If explained the hash collision issue, they’d reasonably point out that Apple does man…

> Most ordinary people, especially those who have kids

What are you basing this assumption on?

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#378

Earlier quoted context omitted.

Add hashes of police uniforms and now you cannot take pictures of law enforcement. Add hashes of politicians. Businessmen in suits. Army uniforms. At the end you have a weapon that can only shoot civilians.

Ok but PhotoDNA has been doing this scanning for a decade, I believe, and none of these things have happened despite how easy it would seem to be? Or have Microsoft, Google, Apple, Facebook, etc., been doing this with such stealth that no-one has actually noticed it happening yet?

Those aren't run on edge devices. Apple's model saves them money and creates a capability - local scanning - which didn't exist before.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#379

Apple's mistake is that they seemingly believe there is pushback because people misunderstand how it works. The reality is more nuanced: People understand exactly how it works, and how it works is that it is turn-key onboard spyware, that Apple pinky-swears isn't being used wrong today . For example if the scope/mission expands (e.g. foreign governments), suddenly you've created a drag-net for whatever "badness" is o…

It's just a matter of time until someone can create a "ThisChildPornDoesNotExist" site, then send links to targets to have their lives ruined. I am extremely worried by humans policing the Intertubes, and even more worried by algorithms doing so because they make the perfect excuse to do sloppy checking. See Youtube DMCA related takedowns for multiple examples.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#380
Let's propose Amazon announced all Alexa devices would start scanning everything it hears for illegal activity. This new feature has the same security semantics as Apple's CSAM detection: on device hashing using a government controlled audio database of known illegal activity, violations are only reported once a threshold is met, etc.

Would you be comfortable with this? Would you be comfortable with a company searching our physical lives in the way Apple intends to search our digital lives? How are these 2 things any different?

Post reply on HN