Security Threat Model Review of the Apple Child Safety Features [pdf] • Hacker News Board

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#271

Post by simondotau » Sat, Aug 14, 2021, 5:58 AM UTC

jdlshore wrote: post #254

simondotau wrote: post #189

Here's what I don't get: who is importing CSAM into their camera roll in the first place? I for one have never felt the urge to import regular, legal porn into my camera roll. Who the hell is going to be doing that with stuff they know will land them in prison? Who the hell co-mingles their deepest darkest dirtiest secret amongst pictures of their family and last night’s dinner? I can believe that some people might b…

Apparently 70 million images have been reported by other providers (if I'm reading [1] correctly, which I'm not sure I am, since the paywall hides it before I can read closely). Assuming that's correct, the answer is "a lot of people." I find that eminently plausible. [1] https://www.nytimes.com/2020/02/07/us/online-child-sexual-ab...

It looks like well over 90% of that is Facebook and the bulk of the remainder is Google and Yahoo who crawl the web and run large email services. None of those are analogous to importing CSAM into your smartphone's photo library.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#272

Post by dwaite » Sat, Aug 14, 2021, 5:58 AM UTC

new299 wrote: post #167

chipotle_coyote wrote: post #154

Earlier quoted context omitted.

> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. Absolutely. I don't know whether there's a reason for this timing (that is, if they are planning E2E encryption, why they announced this first), but this is probably the biggest PR bungle Apple has had since "you're holding it wrong," if not ever.…

Right, and in the interview linked [1] above they state: > The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. But they do appear to do "something" server-side. It's possible that all data in scanned as it is ingested for example. I dislike this statement, because it's probably technically correct but d…

> But they do appear to do "something" server-side. It's possible that all data in scanned as it is ingested for example. I dislike this statement, because it's probably technically correct but doesn't help clarify the situation in a helpful way. It makes me trust Apple less.

The qualifier is "Photos" - different services have different security properties.

Email transport is not E2E encrypted because there are no interoperable technologies for that.

Other systems are encrypted but apple has a separate key escrow system outside the cloud hosting for law enforcement requests and other court orders (such as a heir/estate wanting access).

Some like iCloud Keychain use more E2E approach where access can't be restored if you lose all your devices and paper recovery key.

iCloud Photo Sharing normally only works between AppleID accounts, with the album keys being encrypted to the account. However, you can choose to publicly share an album, at which point it becomes accessible via a browser on icloud.com. I have not heard Apple talking about whether they scan photos today once they are marked public (going forward, there would be no need).

FWIW this is all publicly documented, as well as what information Apple can and can't provide to law enforcement.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#273

Post by simondotau » Sat, Aug 14, 2021, 6:02 AM UTC

zepto wrote: post #266

simondotau wrote: post #202

Earlier quoted context omitted.

If Apple was performing scans on their cloud servers, you'd be absolutely right. But if the scanning is being done on the individual's device, I'm not sure it's that straightforward. The third party doctrine surely cannot apply if the scanning is performed prior to the material being in third party hands. Therefore if the Government forces Apple to change the search parameters contained within private devices, I cann…

This point was made in the economist today. https://www.economist.com/united-states/2021/08/12/a-38-year...

I read the article; I don't think they highlighted this specific point that on-device scanning has a potential, hypothetical constitutional advantage in comparison to Google, Microsoft and Facebook who scan exclusively in the cloud.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#274

Post by laserlight » Sat, Aug 14, 2021, 6:02 AM UTC

jolux wrote: post #84

LexGray wrote: post #80

Earlier quoted context omitted.

Do we have any idea how the NCMEC database is curated? Are there cartoons from Hustler depicting underage girls in distress? Green text stories stating they are true about illegal sexual acts? CGI images of pre-pubescent looking mythical creatures? Manga/Anime images which are sold on the Apple Store? Legitimate artistic images from books currently sold? Images of Winnie the Pooh the government has declared pornograp…

Apple is manually reviewing every case to ensure it’s CSAM. You do have to trust them on that. But if your problem is with NCMEC, you’ve got a problem with Facebook and Google who are already doing this too. And you can’t go to jail for possessing adult pornography. So even if you assume adult porn images are in the database, and Apple’s reviewers decide to forward them to NCMEC, you would still not be able to be pro…

> You do have to trust them on that.

If this system didn't exist, nobody would have to trust Apple.

> you would still not be able to be prosecuted

But I wouldn't want to deal with a frivolous lawsuit, or have a record in the social media of being brought CSA charges.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#275

Post by laserlight » Sat, Aug 14, 2021, 6:11 AM UTC

zepto wrote: post #201

unityByFreedom wrote: post #190

Earlier quoted context omitted.

The part Federighi's "interview" where he can't understand how people perceive this as a back door [1] seems incredibly out of touch. The back door is what everyone is talking about. Someone at Apple should at least be able to put themselves in their critics' shoes for a moment. I guess we need to wait to hear Tim Cook explain how this is not what he described 5 years ago [2]. [1] https://youtu.be/OQUO1DSwYN0?t=425 […

It’s not a back door in any sense of the word. That’s why he is surprised people see it as one. It really only does what they say it does, and it really is hard to abuse. But that doesn’t matter. The point is that even so, it makes everyone into a suspect, and that feels wrong .

I would say that it is a back door, because it would work even if iCloud Photos were E2E encrypted. It may not be a generic one, but one that is specific to a purpose Apple decided is rightful. And there is no guarantee that Apple (or authorities) won't decide that there are other rightful purposes.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#276

Post by dwaite » Sat, Aug 14, 2021, 6:11 AM UTC

new299 wrote: post #164

shuckles wrote: post #142

Earlier quoted context omitted.

> This is an area we’ve been looking at for some time, including current state of the art techniques which mostly involves scanning through entire contents of users’ libraries on cloud services that — as you point out — isn’t something that we’ve ever done; to look through users’ iCloud Photos. https://techcrunch.com/2021/08/10/interview-apples-head-of-p... > This moment calls for public discussion, and we want our c…

> The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. I really dislike this statement. It's likely designed to be "technically true". But it's been reported elsewhere that they do scan iCloud content: https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-... Perhaps they scan as the data is being…

https://www.forbes.com/sites/thomasbrewster/2020/02/11/how-a...

My interpretation is Sophos got it wrong (they don't give a quote from the Apple Officer involved and manage to have a typo in the headline).

Apple does scanning of data which is not encrypted, such as received and sent email over SMTP. They presumably at that time were using PhotoDNA to scan attachments by hash. This is likely what Apple was actually talking about back at CES 2020.

They may have been also scanning public iCloud photo albums, but I haven't seen anyone discuss that one way or another.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#277

Post by alwillis » Sat, Aug 14, 2021, 6:12 AM UTC

rustymonday wrote: post #238

alwillis wrote: post #230

Earlier quoted context omitted.

They should not work to potentially incriminate its owner. But that ship has long sailed, right? Every packet that leaves a device potentially incriminates its owner. Every access point and router is a potential capture point.

When I use a web service, I expect my data to be collected by the service, especially if it is free of charge. A device I own should not be allowed to collect and scan my data without my permission.

A device I own should not be allowed to collect and scan my data without my permission.

It's not scanning; it's creating a cryptographic safety voucher for each photo you upload to iCloud Photos. And unless you reach a threshold of 30 CSAM images, Apple knows nothing about any of your photos.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#278

Post by unityByFreedom » Sat, Aug 14, 2021, 6:21 AM UTC

zepto wrote: post #270

unityByFreedom wrote: post #256

Earlier quoted context omitted.

Trusting Apple to not scan my device in the past was easy because as an engineer I know I would speak up if I saw that kind of thing secretly happening, and I know security researchers would speak up if they detected it. Now Apple will scan the device and we must trust that 3rd parties will not abuse the technology by checking for other kinds of imagery such as memes critical of heads of state. The proposed change is…

> checking for other kinds of imagery such as memes critical of heads of state. Do you live in a country where the head of state wants to check for such memes?

Probably. You underestimate humans if you don't think any of us will try to squash things that make us look bad.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#279

Post by ec109685 » Sat, Aug 14, 2021, 6:31 AM UTC

dwaite wrote: post #272

new299 wrote: post #167

Earlier quoted context omitted.

Right, and in the interview linked [1] above they state: > The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. But they do appear to do "something" server-side. It's possible that all data in scanned as it is ingested for example. I dislike this statement, because it's probably technically correct but d…

> But they do appear to do "something" server-side. It's possible that all data in scanned as it is ingested for example. I dislike this statement, because it's probably technically correct but doesn't help clarify the situation in a helpful way. It makes me trust Apple less. The qualifier is "Photos" - different services have different security properties. Email transport is not E2E encrypted because there are no in…

Even without publicly sharing your iCloud photos, they are accessible on iCloud.com (e.g. you can see your camera role there).

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#280

Post by ec109685 » Sat, Aug 14, 2021, 6:36 AM UTC

notJim wrote: post #56

kemayo wrote: post #26

Earlier quoted context omitted.

Yeah, it's weird. Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. But this is clearly not a universal opinion. The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this prog…

> Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. What I find interesting is that so many people find it worse to do it on device, because of the risk that they do it to photos you don't intend to upload. This is clearly where Apple got caught off-guard, because to them, on-device = private.…

Them adding encrypted hashes to photos you don’t intend to upload is pointless and not much of a threat given the photo themselves are they. They don’t do it, but it doesn’t feel like a huge risk.