Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security Threat Model Review of the Apple Child Safety Features [pdf]

apple.com

Post reply on HN
Print view View original

21–30 of 393 posts

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#21

Post by staticassertion »

dogma1138 wrote: post #16
underseacables wrote: post #8

What’s not discussed in this Paper is how it won’t be used by third-party actors to falsely accuse someone. Does nobody remember the iCloud hack? What about instead of downloading Jennifer Lawrence’s photos, a hacker uploaded children to get someone falsely accused?

Google, Microsoft and a bunch of other services already scan for CSAM materials, yet this hasn’t been an issue so far. In fact I can’t really find a single case when someone was framed like this, despite how easy it is to fill someone’s account with CSAM and call the cops/wait. I don’t like the privacy and property rights issues of this but the whole someone will use this to frame someone is quite BS.

As I recall someone's neighbor once tried to frame them by downloading CSAM on their network and then calling the police or something like that.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#22

Post by kemayo »

almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc.

Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that.

At some point you have to trust your OS vendor.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#23

Post by majormajor »

kemayo wrote: post #22
almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud).

And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc.

It's curious that this is the particular breaking point on the slope for people.

The "on device" aspect just makes it more immediate feeling, I guess?

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#24

Post by fuckcensorship »

dogma1138 wrote: post #16
underseacables wrote: post #8

What’s not discussed in this Paper is how it won’t be used by third-party actors to falsely accuse someone. Does nobody remember the iCloud hack? What about instead of downloading Jennifer Lawrence’s photos, a hacker uploaded children to get someone falsely accused?

Google, Microsoft and a bunch of other services already scan for CSAM materials, yet this hasn’t been an issue so far. In fact I can’t really find a single case when someone was framed like this, despite how easy it is to fill someone’s account with CSAM and call the cops/wait. I don’t like the privacy and property rights issues of this but the whole someone will use this to frame someone is quite BS.

How would someone prove that they were framed by someone planting CSAM? Without evidence, how likely is it that they would be believed?

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#25

Post by almostdigital »

kemayo wrote: post #22
almostdigital wrote: post #19

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already si…

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

Yeah that's true, although to do some sort of mass scanning stealthily they would need a system exactly like what they built with this, if they tried to upload everything for scanning the data use would be enormous and give it away.

I guess it comes down to that I don't trust an OS vendor that ships an A.I. based snitch program that they promise will be dormant.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#26

Post by kemayo »

majormajor wrote: post #23
kemayo wrote: post #22

Earlier quoted context omitted.

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking poi…

Yeah, it's weird. Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. But this is clearly not a universal opinion.

The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this program as the PR shield from law enforcement to be able to do it, that'd leave us having to only trust the OS vendor.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#27

Post by userbinator »

majormajor wrote: post #23
kemayo wrote: post #22

Earlier quoted context omitted.

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking poi…

Yes, you had to trust Apple, but the huge difference with this new thing is that hiding behind CSAM gives them far more (legally obligated, in fact --- because showing you the images those hashes came from would be illegal) plausible deniability and difficulty of verifying their claims.

In other words, extracting the code and analysing it to determine that it does do what you expect is, although not easy, still legal. But the source, the CSAM itself, is illegal to possess, so you can't do that verification much less publish the results. It is this effective legal moat around those questioning the ultimate targets of this system which people are worried about.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#28

Post by thephyber »

underseacables wrote: post #8

What’s not discussed in this Paper is how it won’t be used by third-party actors to falsely accuse someone. Does nobody remember the iCloud hack? What about instead of downloading Jennifer Lawrence’s photos, a hacker uploaded children to get someone falsely accused?

> Does nobody remember the iCloud hack?

The iCloud hack was a few hundred cases of spear phishing.

How does your concern differ from physical access to the phone by an unauthorized user?

I think the law is the problem. Simple possession of that type of content is criminalized, as opposed to requiring the traditional elements of a crime to prosecute it.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#29

Post by kemayo »

almostdigital wrote: post #25
kemayo wrote: post #22

Earlier quoted context omitted.

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

Yeah that's true, although to do some sort of mass scanning stealthily they would need a system exactly like what they built with this, if they tried to upload everything for scanning the data use would be enormous and give it away. I guess it comes down to that I don't trust an OS vendor that ships an A.I. based snitch program that they promise will be dormant.

Speaking cynically, I think that them having announced this program like they did makes it less likely that they have any sort of nefarious plans for it. There's a lot of attention being paid to it now, and it's on everyone's radar going forwards. If they actually wanted to be sneaky, we wouldn't have known about this for ages.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#30

Post by himaraya »

majormajor wrote: post #23
kemayo wrote: post #22

Earlier quoted context omitted.

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust yo…

If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking poi…

No, the threat model differs entirely. Local scanning introduces a whole host of single points of failure, including the 'independent auditor' & involuntary scans, that risk the privacy & security of all local files on a device. Cloud scanning largely precludes these potential vulnerabilities.
Post reply on HN