Security Threat Model Review of the Apple Child Safety Features [pdf] • Hacker News Board

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#251

Post by heavyset_go » Sat, Aug 14, 2021, 4:57 AM UTC

kevin_thibedeau wrote: post #194

simondotau wrote: post #178

Earlier quoted context omitted.

I'm not American, but my understanding is that as soon as Government is forcing Apple to search our devices for something, 4th Amendment protections apply. (Unless they hold a search warrant for that specific person, of course.) Is this not correct?

No. The 4A protections don't apply to third parties. This is part of why the US has nearly nonexistent data protection laws.

> The 4A protections don't apply to third parties.

The government can't pay someone to break into your house and steal evidence they want without a warrant. I mean, they can, but the evidence wouldn't be admissible in court.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#252

Post by Spivak » Sat, Aug 14, 2021, 4:59 AM UTC

unityByFreedom wrote: post #203

zepto wrote: post #201

Earlier quoted context omitted.

It’s not a back door in any sense of the word. That’s why he is surprised people see it as one. It really only does what they say it does, and it really is hard to abuse. But that doesn’t matter. The point is that even so, it makes everyone into a suspect, and that feels wrong .

> It’s not a back door in any sense of the word. It is. It's a simple matter for two foreign governments to decide they don't want their people to criticize the head of state with memes, and then insert such images into the database Apple uses for scanning. Apple's previous position on privacy was to make such snooping impossible because they don't have access to the data. Now they are handing over access. What I and…

How does this attack even work? So some government poisons the database with political dissident memes and suddenly Apple starts getting a bunch of new reports which when reviewed are obviously not CSAM.

If the government can force Apple to also turn over these reports then they could have just made Apple add their political meme database directly and it's already game over.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#253

Post by heavyset_go » Sat, Aug 14, 2021, 5:03 AM UTC

bredren wrote: post #181

jimworm wrote: post #64

Earlier quoted context omitted.

But in that case they would eventually be caught red-handed and won't get to do the "for the children" spiel and get it swept under the rug like it's about to be.

The goal is not for it to be swept under the rug. The goal is for it to deflect concerns over the coming Privacy Relay service.

The government cares far more about other things than CSAM, like terrorism, human and drug trafficking, organized crime, and fraud. Unless the CSAM detection system is going to start detecting those other things and report them to authorities, as well, it won't deflect any concerns over encryption or VPNs.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#254

Post by jdlshore » Sat, Aug 14, 2021, 5:08 AM UTC

simondotau wrote: post #189

Here's what I don't get: who is importing CSAM into their camera roll in the first place? I for one have never felt the urge to import regular, legal porn into my camera roll. Who the hell is going to be doing that with stuff they know will land them in prison? Who the hell co-mingles their deepest darkest dirtiest secret amongst pictures of their family and last night’s dinner? I can believe that some people might b…

Apparently 70 million images have been reported by other providers (if I'm reading [1] correctly, which I'm not sure I am, since the paywall hides it before I can read closely).

Assuming that's correct, the answer is "a lot of people." I find that eminently plausible.

[1] https://www.nytimes.com/2020/02/07/us/online-child-sexual-ab...

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#255

Post by shuckles » Sat, Aug 14, 2021, 5:09 AM UTC

mediumdeviation wrote: post #241

shuckles wrote: post #196

Earlier quoted context omitted.

Their “you can’t compel us to build something” argument was for building a change to the passcode retry logic, which is presumably as simple as a constant change. Certainly building a back door in this system is at least as difficult, so the argument still stands.

In that specific case at least the FBI is asking Apple to produce new firmware that will bypass existing protection on an existing device - basically asking Apple to root a locked down phone which would likely require them breaking their own encryption or finding vulnerabilities in their own firmware. This is not exactly technically trivial since anything of this sort that’s already known would be patched out. In thi…

No, they were asked to sign a software update which would bypass the application processor enforced passcode retry limit on an iPhone 5C which had no secure element.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#256

Post by unityByFreedom » Sat, Aug 14, 2021, 5:14 AM UTC

zepto wrote: post #245

unityByFreedom wrote: post #242

Earlier quoted context omitted.

You can't solve the human-bias problem with technology. That's the whole reason Apple didn't want to build in a back door in the first place.

You may not be able to solve the bias problem altogether, but you can definitely change the threat model and who you have to trust. Apple’s model has always involved trusting them . This model involves trusting other people in narrow ways. The architecture determines what those ways are.

Trusting Apple to not scan my device in the past was easy because as an engineer I know I would speak up if I saw that kind of thing secretly happening, and I know security researchers would speak up if they detected it.

Now Apple will scan the device and we must trust that 3rd parties will not abuse the technology by checking for other kinds of imagery such as memes critical of heads of state.

The proposed change is so much worse than the previous state of things.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#257

Post by Spivak » Sat, Aug 14, 2021, 5:17 AM UTC

naasking wrote: post #47

jorgesborges wrote: post #33

I keep changing my mind on this. On the one hand I already operate on the assumption that uploading data to a cloud service renders that data non-private, or at least in great risk of becoming non-private in the future. This simply makes my operating assumption explicit. Also this particular implementation and its stated goals aren’t egregious. But then there’s the slippery slope we’ve all been discussing — and the g…

> But I sympathize with Apple for making transparent what I assume happens behind closed doors anyway. Using your devices' CPU and battery seems more egregious than doing it on their servers. If they want to help law enforcement, then they should pay for it. Of course they want to help law enforcement by forcing other people to pay the costs. Imagine if Ford came out with a cannabis sensor in their cars that automati…

Let's do a more realistic example. Ford introduces a "driver safety" mechanism where the car requires a clean breathalyzer reading in order to start. If it fails it pops up a message that reminds you that drunk driving is illegal but doesn't actually stop you from starting the engine. It then sends Ford the results in an encrypted payload along with 1/30th of the decryption key.

After 30 reports someone at Ford opens the payloads, looks at the results and, decides whether to contact the police based on how damning the evidence is.

Because all the tests are performed locally you're never giving up any of your privacy unless you actually are driving drunk or are the 1/billion and get 30 false positives. I feel like this is a much stronger than the "if you have nothing to hide" meme because local testing like this lets you can have everything to hide while still revealing specific bad behavior.

Like if the TSA had scanners that somehow could only reveal banned items and nothing else I’m not sure I would even consider it a privacy violation.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#258

Post by jdlshore » Sat, Aug 14, 2021, 5:18 AM UTC

616c wrote: post #102

I have long been interested in what a professional-grade threat model from a large FAANG/SV organization is. Is this a representative model? Microsoft came up with DREAD and STRIDE and they suggest there threat models are more elaborate. Would love to see more representative examples!

This feels more like a marketing document to me than a workaday threat model. It's fairly handwavey, and the goal seems to be to convince rather than to do a hardnosed analysis.

Not that it's not useful—I found it convincing—but I doubt this is what a real threat model looks like. Not that I actually know. I'd be interested in seeing a real one too.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#259

Post by new299 » Sat, Aug 14, 2021, 5:18 AM UTC

selsta wrote: post #199

new299 wrote: post #169

Earlier quoted context omitted.

If you have references to also help me piece this together I'd find that really helpful.

> Last year, for instance, Apple reported 265 cases to the National Center for Missing & Exploited Children, while Facebook reported 20.3 million According to [1] it does seem like Apple didn't do any wide scale scanning of iCloud Data. [1] https://www.nytimes.com/2021/08/05/technology/apple-iphones-...

If they weren't doing any scanning why would they find any to report? The data is encrypted as rest so... why would they find any to report. This clearly doesn't include search requests [1].

iCloud has perhaps 25% of the users of Facebook. Of that 25% it's not clear how many actively use the platform of backups/photos. iCloud is not a platform for sharing content like Facebook. So how many reports should we expect to see from Apple? It's unclear to me.

So, I'm not saying the number isn't suspiciously low. But it doesn't really clarify what's going on to me...

[1] https://www.apple.com/legal/transparency/pdf/requests-2018-H...

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#260

Post by dwaite » Sat, Aug 14, 2021, 5:19 AM UTC

new299 wrote: post #134

chipotle_coyote wrote: post #92

In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :) Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents us…

> there's a legitimate "slippery slope" argument The slippery slope argument is the only useful argument here. The fundamental issue with their PSI/CSAM system is that they already were scanning iCloud content [1] and that they're seemingly not removing the ability to do that. If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in…

> The fundamental issue with their PSI/CSAM system is that they already were scanning iCloud content

This scanning was of email attachments being sent through an iCloud-hosted account, not of other iCloud hosted data (which is encrypted during operation.)