Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security Threat Model Review of the Apple Child Safety Features [pdf]

apple.com

Post reply on HN
Print view View original

31–40 of 393 posts

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#31

Post by majormajor »

userbinator wrote: post #27
majormajor wrote: post #23

Earlier quoted context omitted.

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking poi…

Yes, you had to trust Apple, but the huge difference with this new thing is that hiding behind CSAM gives them far more (legally obligated, in fact --- because showing you the images those hashes came from would be illegal) plausible deniability and difficulty of verifying their claims. In other words, extracting the code and analysing it to determine that it does do what you expect is, although not easy , still lega…

Surely they could do their image matching against all photos in iCloud without telling you in advance, and then you'd be in exactly the same boat? Google was doing this for email as early as 2014, for instance, with the same concerns about its extensibility raised by the ACLU: https://www.theguardian.com/technology/2014/aug/04/google-ch...

So in a world where Apple pushes you to set up icloud photos by default, and can do whatever they want there, and other platforms have been doing this sort of of thing for years, it's a bit startling that "on device before you upload" vs "on uploaded content" triggers far more discontent?

Maybe it's that Apple announced it at all, vs doing it relatively silently like the others? Apple has always had access to every photo on your device, after all.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#33

Post by jorgesborges »

I keep changing my mind on this. On the one hand I already operate on the assumption that uploading data to a cloud service renders that data non-private, or at least in great risk of becoming non-private in the future. This simply makes my operating assumption explicit. Also this particular implementation and its stated goals aren’t egregious.

But then there’s the slippery slope we’ve all been discussing — and the gradual precedents set for it, this being a nail in the coffin. But I sympathize with Apple for making transparent what I assume happens behind closed doors anyway. My optimistic side thinks maybe this is a sign that extensions of this project will also be made explicit. Maybe this is the only way for them to take this fight if they’re being pressured. In any case now is the time to speak out. If they extend it to texts and other documents I’m out - even as it stands I’m thinking of alternatives. But the fact we’re all discussing it now is at least a point of hope.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#34

Post by majormajor »

himaraya wrote: post #30
majormajor wrote: post #23

Earlier quoted context omitted.

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking poi…

No, the threat model differs entirely. Local scanning introduces a whole host of single points of failure, including the 'independent auditor' & involuntary scans, that risk the privacy & security of all local files on a device. Cloud scanning largely precludes these potential vulnerabilities.

Your phone threat model should already include "the OS author has full access to do whatever they want to whatever data is on my phone, and can change what they do any time they push out an update."

I don't think anyone's necessarily being too upset or paranoid about THIS, but maybe everyone should also be a little less trusting of every closed OS - macOS, Windows, Android as provided by Google - that has root access too.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#35

Post by PaulDavisThe1st »

> Apple generates the on-device perceptual CSAM hash database through an intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions – that is, not under the control of the same government. Any perceptual hashes appearing in only one participating child safety organization’s database, or only in databases from multiple agencies in a single sovereign jurisdiction, are discarded by this process, and not included in the encrypted CSAM database that Apple includes in the operating system.

Well, that's quite clever, isn't it.

It's just the sort of thing that someone who had "never heard of" the Five Eyes, or extradition treaties or illegal rendition operations involving multiple nations, might come up with.

Genius!

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#36

Post by theonlybutlet »

kemayo wrote: post #29
almostdigital wrote: post #25

Earlier quoted context omitted.

Yeah that's true, although to do some sort of mass scanning stealthily they would need a system exactly like what they built with this, if they tried to upload everything for scanning the data use would be enormous and give it away. I guess it comes down to that I don't trust an OS vendor that ships an A.I. based snitch program that they promise will be dormant.

Speaking cynically, I think that them having announced this program like they did makes it less likely that they have any sort of nefarious plans for it. There's a lot of attention being paid to it now, and it's on everyone's radar going forwards. If they actually wanted to be sneaky, we wouldn't have known about this for ages.

They'd have to be transparent about it as someone would easily figure it out.You have no way of verifying the contents of that hash database. once the infrastructure is in place (i.e. on your phone) it's a lot easier to expand on it. People have short memories and are easily desensitized, after a year or two of this, everyone will forget and we'll be in uproar about it expanding to include this or that...

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#37

Post by himaraya »

majormajor wrote: post #34
himaraya wrote: post #30

Earlier quoted context omitted.

No, the threat model differs entirely. Local scanning introduces a whole host of single points of failure, including the 'independent auditor' & involuntary scans, that risk the privacy & security of all local files on a device. Cloud scanning largely precludes these potential vulnerabilities.

Your phone threat model should already include "the OS author has full access to do whatever they want to whatever data is on my phone, and can change what they do any time they push out an update." I don't think anyone's necessarily being too upset or paranoid about THIS, but maybe everyone should also be a little less trusting of every closed OS - macOS, Windows, Android as provided by Google - that has root access…

Sure, but that doesn't change the fact that the vulnerabilities with local scanning remain a significant superset of cloud scanning's.

Apple has built iOS off user trust & goodwill, unlike most other OSes.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#38

Post by jolux »

almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc.

They describe a process for third parties to audit that the database was produced correctly.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#39

Post by lttlrck »

himaraya wrote: post #30
majormajor wrote: post #23

Earlier quoted context omitted.

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking poi…

No, the threat model differs entirely. Local scanning introduces a whole host of single points of failure, including the 'independent auditor' & involuntary scans, that risk the privacy & security of all local files on a device. Cloud scanning largely precludes these potential vulnerabilities.

It differs, but iOS already scans images locally and we really don't know what they do with the meta data, and what "hidden" categories there are.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#40

Post by shuckles »

almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

Apple shipped iCloud Private Relay which is a “1-line code change that hooks into CFNetwork” away from MITMing all your network connections, by this standard.
Post reply on HN