Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security Threat Model Review of the Apple Child Safety Features [pdf]

apple.com

Post reply on HN
Print view View original

211–220 of 393 posts

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#211

Post by arvinsim »

FabHK wrote: post #115
Syonyk wrote: post #78

Earlier quoted context omitted.

I understand the technologies they're proposing deploying at a decent level (I couldn't implement the crypto with my current skills, but what they're doing in the PSI paper makes a reasonable amount of sense). The problem is that this "hard" technological core (the crypto) is subject to an awful lot of "soft" policy issues around the edge - and there's nothing but "Well, we won't do that!" in there. Plus, the whole T…

> Even if this, alone isn't enough to convince you to move off Apple, are you comfortable with the trends now clearly visible? Still much better than all but the most esoteric inconvenient alternatives.

Then people need to start asking themselves if privacy is really a value they really hold or is just an empty, bandwagon idealism.

Because sacrificing privacy for convenience is why we got to this point.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#212

Post by unityByFreedom »

shuckles wrote: post #191
zepto wrote: post #186

A key point that needs to be mentioned: we strongly dislike being distrusted. It might well be a genetic heritage. Being trusted in a tribe is crucial to survival, and so is likely wired deep into our social psychology. Apple is making a mistake by ignoring that. This isn’t about people not trusting Apple. It’s about people not feeling trusted by Apple. Because of this, it doesn’t matter how trustworthy the system is…

But the catch is: all the incumbents already treated your data as if you were guilty until proven innocent. Apple’s transparency about that change may have led people to internalize that, but it’s been the de facto terms of most cloud relationships. What I personally don’t understand is why Apple didn’t come out with a different message: we’ve made your iPhone so secure that we’ll let it vouch for your behalf when it…

> What I personally don’t understand is why Apple didn’t come out with a different message: we’ve made your iPhone so secure that we’ll let it vouch for your behalf when it sends us data to store. We don’t want to see the data, and we won’t see any of it unless we find that lots of the photos you send us are fishy.

That is PR speak that would have landed worse in tech forums. I respect them more for not doing this.

The core issue is this performs scans, without your approval, on your local device. Viruses already do that and it's something techies have always feared governments might impose. That private companies are apparently being strong-armed into doing it is concerning because it means the government is trying to circumvent the process of public discussion that is typically facilitated by proposing legislation.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#213

Post by least »

kevin_thibedeau wrote: post #194
simondotau wrote: post #178

Earlier quoted context omitted.

I'm not American, but my understanding is that as soon as Government is forcing Apple to search our devices for something, 4th Amendment protections apply. (Unless they hold a search warrant for that specific person, of course.) Is this not correct?

No. The 4A protections don't apply to third parties. This is part of why the US has nearly nonexistent data protection laws.

The sort of questions about 4A protections here haven't really been tested. Third party doctrine might not apply in this circumstance and the court is slowly evolving with the times.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#214

Post by unityByFreedom »

zepto wrote: post #200
shuckles wrote: post #191

Earlier quoted context omitted.

But the catch is: all the incumbents already treated your data as if you were guilty until proven innocent. Apple’s transparency about that change may have led people to internalize that, but it’s been the de facto terms of most cloud relationships. What I personally don’t understand is why Apple didn’t come out with a different message: we’ve made your iPhone so secure that we’ll let it vouch for your behalf when it…

I agree this is vastly better than what anyone else is doing, and you know I understand the technology. However, I don’t think any framing would have improved things. I think it was always going to feel wrong. I would prefer they don’t do this because it feels bad to be a suspect even in this abstract and in-practice harmless way. Having said that, having heard from people who have investigated how bad pedophile acti…

> Having said that, having heard from people who have investigated how bad pedophile activity actually is, I can imagine being easily persuaded back in the other direction.

There are terrible things out there that we should seek to solve. They should not be solved by creating 1984 in the literal sense, and certainly not by the company that became famous for an advertisement based on that book [1].

Apple, take your own advice and Think Different [2].

[1] https://www.youtube.com/watch?v=VtvjbmoDx-I

[2] https://www.youtube.com/watch?v=5sMBhDv4sik

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#215

Post by zepto »

unityByFreedom wrote: post #203
zepto wrote: post #201

Earlier quoted context omitted.

It’s not a back door in any sense of the word. That’s why he is surprised people see it as one. It really only does what they say it does, and it really is hard to abuse. But that doesn’t matter. The point is that even so, it makes everyone into a suspect, and that feels wrong .

> It’s not a back door in any sense of the word. It is. It's a simple matter for two foreign governments to decide they don't want their people to criticize the head of state with memes, and then insert such images into the database Apple uses for scanning. Apple's previous position on privacy was to make such snooping impossible because they don't have access to the data. Now they are handing over access. What I and…

Edit: I reconsidered my previous reply.

That really doesn’t sound like anything I’d describe as a “back door”. A back door implies general purpose access. A system which required the collision of multiple governments and Apple and their child abuse agencies simply is not that.

One of the casualties of this debate is that people are using terms that make things sound worse than they are. If you can’t get at my filesystem, you don’t have a back door. I understand the motive for stating the case as harshly as possible, but I think it’s misguided.

Having said this, I would find it interesting to hear what Federighi would say about this potential abuse case.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#217

Post by shuckles »

zepto wrote: post #200
shuckles wrote: post #191

Earlier quoted context omitted.

But the catch is: all the incumbents already treated your data as if you were guilty until proven innocent. Apple’s transparency about that change may have led people to internalize that, but it’s been the de facto terms of most cloud relationships. What I personally don’t understand is why Apple didn’t come out with a different message: we’ve made your iPhone so secure that we’ll let it vouch for your behalf when it…

I agree this is vastly better than what anyone else is doing, and you know I understand the technology. However, I don’t think any framing would have improved things. I think it was always going to feel wrong. I would prefer they don’t do this because it feels bad to be a suspect even in this abstract and in-practice harmless way. Having said that, having heard from people who have investigated how bad pedophile acti…

I don’t think I agree. If you think this boils down to instinct, do you think a story about coming together to save the next generation will work well on people cynical enough to see TLAs around every corner? At the very least, I feel like Apple should probably make a concession to the conspiracy minded so that they can bleach any offensive bits from their device and use them as offline-first DEFCON map viewing devices, or some such.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#218

Post by unityByFreedom »

zepto wrote: post #215
unityByFreedom wrote: post #203

Earlier quoted context omitted.

> It’s not a back door in any sense of the word. It is. It's a simple matter for two foreign governments to decide they don't want their people to criticize the head of state with memes, and then insert such images into the database Apple uses for scanning. Apple's previous position on privacy was to make such snooping impossible because they don't have access to the data. Now they are handing over access. What I and…

Edit: I reconsidered my previous reply. That really doesn’t sound like anything I’d describe as a “back door”. A back door implies general purpose access. A system which required the collision of multiple governments and Apple and their child abuse agencies simply is not that. One of the casualties of this debate is that people are using terms that make things sound worse than they are. If you can’t get at my filesys…

> A system which required the collision of multiple governments and Apple and their child abuse agencies simply is not that.

Agree to disagree.

> I would find it interesting to hear what Federighi would say about this potential abuse case.

Personally I would not. That's a political consideration and not something I want to hear a technologist weigh in on while defending their technology. Apple's previous stance, with which I agree, was to not give humans any chance to abuse people's personal data,

https://youtu.be/rQebmygKq7A

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#219

Post by threatofrain »

bredren wrote: post #181
jimworm wrote: post #64

Earlier quoted context omitted.

But in that case they would eventually be caught red-handed and won't get to do the "for the children" spiel and get it swept under the rug like it's about to be.

The goal is not for it to be swept under the rug. The goal is for it to deflect concerns over the coming Privacy Relay service.

Their private relay service appears orthogonal to CSAM… it won’t make criminals and child abusers easier or harder to catch, and it doesn’t affect how people use their iCloud Photos storage.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#220

Post by kelnos »

chipotle_coyote wrote: post #92

In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :) Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents us…

> For instance, the "it only scans photos uploaded to iCloud" element isn't just an arbitrary limitation that can be flipped with one line of code, as some folks seem to think; as Erik Neuenschwander, head of Privacy Engineering at Apple, explained in an interview on TechCrunch[1]:

> > Our system involves both an on-device component where the voucher is created, but nothing is learned, and a server-side component, which is where that voucher is sent along with data coming to Apple service and processed across the account to learn if there are collections of illegal CSAM. That means that it is a service feature.

I don't see how that proves what Neuenschwander says it proves. Sending the voucher to the server is currently gated on whether or not the associated photo is set to be uploaded to iCloud. There's no reason why that restriction couldn't be removed.

The hard part about building this feature was doing the scanning and detection, and building out the mechanism by which Apple gets notified if anything falls afoul of the scanner. Changing what triggers (or does not trigger) the scanning to happen, or results being uploaded to Apple, is trivial.

The only thing that I think is meaningfully interesting here is that it's possible that getting hold of a phone and looking at the vouchers might not tell you anything; only after they vouchers are sent to Apple and algorithm magic happens will you learn anything. But I don't think that really matters in practice; if you can get the vouchers off a phone, you can almost certainly get the photos associated with them as well.

I read through the article you linked, and it feels pretty hand-wavy to me. Honestly I don't think I'd believe what Apple is saying here unless they release a detailed whitepaper on how the system works, one that can be vetted by experts in the field. (And then we still have to trust that what they've detailed in the paper is what they've actually implemented.)

The bottom line is that Apple has developed and deployed a method for locally scanning phones for contraband. Even if they've designed things so that the result is obfuscated unless there are many matches and/or photos are actually uploaded, that seems to be a self-imposed limitation that could be removed without too much trouble. The capability is there; not using it is merely a matter of policy.

> I know that's easy to dismiss as Pollyannaism, but "we need to protect ourselves from our own government" has a pretty dismal track record historically. The entire point of having a liberal democracy is that we are the government, and we can pull it back from authoritarianism.

Absolutely agree, but I fear we have been losing this battle for many decades now, and I'm a bit pessimistic for our future. It seems most people are very willing to let their leaders scare them into believing that they must give up liberty in exchange for safety and security.

Post reply on HN