In internal memo, Apple addresses concerns around new Photo scanning features • Hacker News Board

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#61

Post by feross » Sat, Aug 07, 2021, 3:08 AM UTC

sumeno wrote: post #49

feross wrote: post #17

Earlier quoted context omitted.

Now that a fully built system for breaking end-to-end encryption is shipped directly in the OS, we're one configuration change away from massive scope creep. First terrorist content, then "misinformation", then political speech. Apple will be unable to resist government demands to use this preexisting backdoor with a different set of perceptual hashes.

It's not built to break end-to-end encryption because photos in iCloud aren't end-to-end encrypted https://support.apple.com/en-us/HT202303

Then ask yourself why they shipped this scanning on the client-side. This is the first step towards normalizing client-side scanning of encrypted content across the entire device.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#62

Post by strogonoff » Sat, Aug 07, 2021, 3:09 AM UTC

pseudalopex wrote: post #57

strogonoff wrote: post #56

Earlier quoted context omitted.

> And Apple's system doesn't use homomorphic encryption. Do you work for Apple?

I read the technical summary.[1] [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Did you learn the same thing—that E2E encryption doesn’t need to be broken for this to work?

The only event in which Apple can gain access to your content is if you happen to have multiple CSAM matches; then they can access only the matching content, and only then if it’s manually confirmed by a human to be CSAM an action is taken.

The issue is if this type of matching is done for other purposes than CSAM; and unfortunately they gave themselves legal permission to do it back in 2019. That’s what we should object to, not CSAM reporting.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#63

Post by sumeno » Sat, Aug 07, 2021, 3:11 AM UTC

modeless wrote: post #60

pseudalopex wrote: post #46

Earlier quoted context omitted.

The technical summary describes a new system where Apple just has keys for suspect images.

The technical summary describes the detection system, not the rest of the product. The rest of iCloud photos is not end-to-end encrypted. If Apple was planning to introduce end-to-end encryption for iCloud photos then they should have announced it at the same time.

If they are using this to introduce something closer to end-to-end encryption then it seems like a clear win for users.

Today photos are not end-to-end encrypted, there is nothing preventing Apple from decrypting your photos if they want (or if they are asked by law enforcement). If a part of this implementation is to make it so only the user keys OR the CSAM keys in the case of a match are able to decrypt the photos then that is a clear step in the right direction over the current system. It's not real end-to-end encryption, but it still prevents Apple from just decrypting your photos without probable cause of a very specific crime.

If that is the case they should have made it a lot clearer in the initial announcement though.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#64

Post by sumeno » Sat, Aug 07, 2021, 3:16 AM UTC

feross wrote: post #61

sumeno wrote: post #49

Earlier quoted context omitted.

It's not built to break end-to-end encryption because photos in iCloud aren't end-to-end encrypted https://support.apple.com/en-us/HT202303

Then ask yourself why they shipped this scanning on the client-side. This is the first step towards normalizing client-side scanning of encrypted content across the entire device .

Hopefully so they can remove their current ability to decrypt user photos for whatever reason they want. The current state is they can decrypt any user photos on iCloud. Doing client side scanning and this CSAM detection implementation could allow them to remove their ability to decrypt EXCEPT in very specific situations.

It's not true end-to-end encryption since in some cases the content can be decrypted without the user key but it's significantly closer than what they have today.

That being said I don't know if that is their plan or not, but it is a plausible reason to make this change.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#65

Post by modeless » Sat, Aug 07, 2021, 3:20 AM UTC

sumeno wrote: post #63

modeless wrote: post #60

Earlier quoted context omitted.

The technical summary describes the detection system, not the rest of the product. The rest of iCloud photos is not end-to-end encrypted. If Apple was planning to introduce end-to-end encryption for iCloud photos then they should have announced it at the same time.

If they are using this to introduce something closer to end-to-end encryption then it seems like a clear win for users. Today photos are not end-to-end encrypted, there is nothing preventing Apple from decrypting your photos if they want (or if they are asked by law enforcement). If a part of this implementation is to make it so only the user keys OR the CSAM keys in the case of a match are able to decrypt the photos…

Yeah. After thinking about it this is probably the missing motivation for building this feature client side. It's a huge blunder to announce/enable this before actually doing the end-to-end encryption for iCloud. It renders the whole thing pointless.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#66

Post by shuckles » Sat, Aug 07, 2021, 3:26 AM UTC

feross wrote: post #61

sumeno wrote: post #49

Earlier quoted context omitted.

It's not built to break end-to-end encryption because photos in iCloud aren't end-to-end encrypted https://support.apple.com/en-us/HT202303

Then ask yourself why they shipped this scanning on the client-side. This is the first step towards normalizing client-side scanning of encrypted content across the entire device .

Why would Apple want to do this? It doesn’t benefit them at all. Their competitive advantage is having people trust their devices, if not their values.

People are making an extreme claim that Apple went out of their way to implement a fancy system to ruin their own value proposition, and the evidence they have to offer is mere speculation.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#67

Post by jtsiskin » Sat, Aug 07, 2021, 3:35 AM UTC

How will this catch anyone? Won’t the people (or definitely the very worst of them) who this feature is intended to catch just stop using iPhone now? So in the end, is the effort really worth it?

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#68

Post by pseudalopex » Sat, Aug 07, 2021, 3:44 AM UTC

strogonoff wrote: post #62

pseudalopex wrote: post #57

Earlier quoted context omitted.

I read the technical summary.[1] [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Did you learn the same thing—that E2E encryption doesn’t need to be broken for this to work? The only event in which Apple can gain access to your content is if you happen to have multiple CSAM matches; then they can access only the matching content, and only then if it’s manually confirmed by a human to be CSAM an action is taken. The issue is if this type of matching is done for other purposes than CSAM; and unfort…

I didn't say anything about breaking E2E encryption. Anything a human in the middle can review in any event isn't E2E encrypted. Call it something else.

The issue is the hash algorithm is secret. The decryption threshold is secret. The database of forbidden content can't be audited. People claim it includes entirely legal images. And it's a small step from scanning local only files.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#69

Post by mickotron » Sat, Aug 07, 2021, 4:04 AM UTC

Someone1234 wrote: post #38

Apple's mistake is that they seemingly believe there is pushback because people misunderstand how it works. The reality is more nuanced: People understand exactly how it works, and how it works is that it is turn-key onboard spyware, that Apple pinky-swears isn't being used wrong today . For example if the scope/mission expands (e.g. foreign governments), suddenly you've created a drag-net for whatever "badness" is o…

They make the premise so innocuous that it would seem unreasonable to object, eg who would object to keeping kids safe?

However it's what comes after this, the slippery slope of what to enforce and not. The genie doesn't go back in the bottle.

Re: In internal memo, Apple addresses concerns around new Photo scanning features

#70

Post by tengbretson » Sat, Aug 07, 2021, 4:19 AM UTC

I honestly don't get the uproar here. This is opt-in, so how is it any more outrageous than existing MDM software that people opt-in to for work reasons which are way more invasive?