Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security Threat Model Review of the Apple Child Safety Features [pdf]

apple.com

Post reply on HN
Print view View original

101–110 of 393 posts

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#101

Post by Veserv »

kemayo wrote: post #22
almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

Yes, they can technically already do so, but that is not the question. The question is what can they legally do and justify with high confidence in the event of a legal challenge.

Changes to binding contractual terms that allow broad readings and provide legal justification for future overreach are dangerous. If they really are serious that they are going to use these new features in a highly limited way then they can put their money where their mouth is and add legally binding contractual terms that limit what they can do with serious consequences if they are found to be in breach. Non-binding marketing PR assurances that they will not abuse their contractually justified powers are no substitute for the iron fist of legal penalty clause.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#102

Post by 616c »

I have long been interested in what a professional-grade threat model from a large FAANG/SV organization is. Is this a representative model?

Microsoft came up with DREAD and STRIDE and they suggest there threat models are more elaborate.

Would love to see more representative examples!

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#103

Post by shuckles »

drodgers wrote: post #97
shuckles wrote: post #87

Earlier quoted context omitted.

Your understanding is incorrect. Apple can, and is in fact required to, verify that they have actual CSAM before forwarding it to the Cyber Tip line. At that point, they must delete the information within 60 days.

Interesting. In that case, do you know why they talk about reviews only seeing "visual derivatives" (from the second perceptual hash)? Either these 'derivatives' basically contain the original image (so reviewers can verify that it's actual CSAM) and there's no point in using derivatives at all, or they're more abstract (eg. 8x8 pixellated images) in which case the reviewer can't see the actual content (but could con…

Apple claims that “data minimization” is one of their privacy pillars, and this was probably an attempt at that. You could imagine an inverted colors or lower res image gets the point across without subjecting you or Apple to all the high fidelity images you have in your library.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#104

Post by pseudalopex »

majormajor wrote: post #31
userbinator wrote: post #27

Earlier quoted context omitted.

Yes, you had to trust Apple, but the huge difference with this new thing is that hiding behind CSAM gives them far more (legally obligated, in fact --- because showing you the images those hashes came from would be illegal) plausible deniability and difficulty of verifying their claims. In other words, extracting the code and analysing it to determine that it does do what you expect is, although not easy , still lega…

Surely they could do their image matching against all photos in iCloud without telling you in advance, and then you'd be in exactly the same boat? Google was doing this for email as early as 2014, for instance, with the same concerns about its extensibility raised by the ACLU: https://www.theguardian.com/technology/2014/aug/04/google-ch... So in a world where Apple pushes you to set up icloud photos by default, and c…

It isn't startling people trust they can opt out of iCloud photos.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#105

Post by LexGray »

jolux wrote: post #84
LexGray wrote: post #80

Earlier quoted context omitted.

Do we have any idea how the NCMEC database is curated? Are there cartoons from Hustler depicting underage girls in distress? Green text stories stating they are true about illegal sexual acts? CGI images of pre-pubescent looking mythical creatures? Manga/Anime images which are sold on the Apple Store? Legitimate artistic images from books currently sold? Images of Winnie the Pooh the government has declared pornograp…

Apple is manually reviewing every case to ensure it’s CSAM. You do have to trust them on that. But if your problem is with NCMEC, you’ve got a problem with Facebook and Google who are already doing this too. And you can’t go to jail for possessing adult pornography. So even if you assume adult porn images are in the database, and Apple’s reviewers decide to forward them to NCMEC, you would still not be able to be pro…

Facebook I completely approve of. You are trafficking data at that point if you are posting it. I just recall the days of Usenet and Napster when I would just download at random and sometimes the evil would mislabel things to cause trauma. I do not download things at random any more but when I was that age it would have been far more appropriate to notify my parents then it would be to notify the government.

In any case it is likely the government would try to negotiate a plea to get you into some predator database to help fill the law enforcement coffers even if they have no lawful case to take it to court once they have your name in their hands.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#106

Post by dang »

zepto wrote: post #48
dang wrote: post #12

Earlier quoted context omitted.

Reports are not all we act on. We read the threads too. No one can read all of them though. Discussion on this topic has certainly been mixed, but "mostly hysterical" sounds like an exaggeration to me. People's reactions to these things are conditioned by the cognitive bias that causes us to weight the things we dislike much more strongly than the things we agree with ( https://hn.algolia.com/?dateRange=all&page=0&pr…

Mostly hysterical may technically be an exaggeration, but mostly misinformed, and often dis-informative is not. This isn’t just about people weighing things they dislike more strongly. It’s also about groupthink, confirmation bias, and a lack of curiosity. HN doesn’t have an immune system against straight up misinformation.

> HN doesn’t have an immune system against straight up misinformation.

It certainly doesn't! Misinformation and disinformation are terms du jour, but as far as I can tell they are indistinguishable from old-fashioned people-being-wrong-on-the-internet. If you expect an internet forum to be immune from that...well, that's too much to expect. As far as I can tell (and moderating HN for years has really hammered this home), nearly everyone is wrong about nearly everything.

Not only that, but no one (or perhaps we can say for decorum's sake, nearly no one) really cares about the truth. We care about what we like and we want it to win against what we dislike; all the rest is rationalization. Moderating HN has really hammered that one into me as well. Such is human nature, and trying to moderate against it would be futile, not to mention a fast track to burnout. I've tried, and have the scars.

And at the same time I completely sympathize with the frustration of watching discourse on a major topic being dominated by shallow, indignant repetition that isn't engaging with the specifics of a situation.

From a moderation point of view, when a tsunami of a story washes over HN for a week or more, there's not that much we can do about it. We can prevent it from completely dominating the site; we can ask a few people not to break the site guidelines when we see that happening; that's about it. Pretending we can do much more than that would be like Canute commanding the waves.

(Btw, maybe I'm mistaking you for someone else but I have the feeling that your comments have gone, over the years, from frequently breaking the site guidelines to doing a pretty good job of respecting them. If that's true, that's amazing and I appreciate it a ton.)

Edit: I could keep adding new paragraphs to this post for almost forever, but here's another factor that moderating HN has hammered into me. It's possible for people to be wrong on both the facts and the arguments and yet for there to be some deeper justice or truth in what they're saying. Oftentimes, conflicts play out like this: someone responds by correcting facts or pointing out flaws in arguments, but they only succeed in provoking an intensified fury and end up drawing it to themselves. That's because they're answering on the level of facts-and-arguments as a way of dismissing, rather than acknowledging, that deeper level of truth or justice that people have strong feelings about. Mostly all this does is increase the amplitude of the conflict. This is very much a co-creation between the conflicting parties—i.e. between the ones who are right (or feel they are) and the other ones who are right (or feel they are). This is the real answer to complaints about groupthink, in my opinion.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#107

Post by ipv6ipv4 »

notJim wrote: post #81
almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

> Until a 1-line code change happens that hooks it into UIImage. I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. If you only use open source software and advocate for others to do the same, I would understand it more.

Did you verify all the binaries that you run are from compiled source code that you audited? Your BIOS? What about your CPU and GPU firmware?

There is always a chain of trust that you end up depending on. OSS is not a panacea here.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#108

Post by dang »

shuckles wrote: post #18
dang wrote: post #12

Earlier quoted context omitted.

Reports are not all we act on. We read the threads too. No one can read all of them though. Discussion on this topic has certainly been mixed, but "mostly hysterical" sounds like an exaggeration to me. People's reactions to these things are conditioned by the cognitive bias that causes us to weight the things we dislike much more strongly than the things we agree with ( https://hn.algolia.com/?dateRange=all&page=0&pr…

I think the fact that no technical document has lasted on the front page but various rumors have is a strong point for my view. An article about internal dissent at Apple has hung on for most of the day, yet almost no comments on it engage with any of the key concepts in the article: what does it mean for an 800 post slack thread to exist? Why does it matter that the security employees don’t seem against the idea on…

https://news.ycombinator.com/item?id=28173134 is #1 on the front page right now.

Matthew Green and Alex Stamos both wrote things about this that were on HN's front page for a long time. I'm pretty sure there have been other technical threads as well.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#109

Post by notJim »

almostdigital wrote: post #96
notJim wrote: post #81

Earlier quoted context omitted.

> Until a 1-line code change happens that hooks it into UIImage. I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. If you only use open source software and advocate for others to do the same, I would understand it more.

> I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. And I don't understand why it has to be black and white, I think the N is very important in this formula and if it is low that is a cause for concern. Like an enemy building a missile silo on an island just off your coast but p…

The size of N doesn't really matter. I'm sure Apple ships large PRs in every release, as any software company does.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#110

Post by kemayo »

pseudalopex wrote: post #104
majormajor wrote: post #31

Earlier quoted context omitted.

Surely they could do their image matching against all photos in iCloud without telling you in advance, and then you'd be in exactly the same boat? Google was doing this for email as early as 2014, for instance, with the same concerns about its extensibility raised by the ACLU: https://www.theguardian.com/technology/2014/aug/04/google-ch... So in a world where Apple pushes you to set up icloud photos by default, and c…

It isn't startling people trust they can opt out of iCloud photos.

If you trust that you can opt out of iCloud Photos to avoid server-side scanning, trusting that this on-device scanning only happens as part of the iCloud Photos upload process (with the only way it submits the reports being as metadata attached to the photo-upload, as far as I can tell) seems equivalent.

There's certainly a slippery-slope argument, where some future update might change that scanning behavior. But the system-as-currently-presented seems similarly trustable.

Post reply on HN