Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security Threat Model Review of the Apple Child Safety Features [pdf]

apple.com

Post reply on HN
Print view View original

311–320 of 393 posts

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#311

Post by simondotau »

0xy wrote: post #301
simondotau wrote: post #296

Earlier quoted context omitted.

You don't know what's in it, but you do know it's full of false positives? I wonder, do you know how many of those false positives are flagged as A1?

I know for a fact that it is full of false positives, there's also public sources making the same claim. [1] [1] https://www.hackerfactor.com/blog/index.php?/archives/929-On...

I clicked on your link hoping for serious analysis. I would have settled for interesting analysis. I was disappointed. It's just a guy who found one MD5 hash collision. The paragraph was written in a way that makes it unclear whether source of this specific hash was in fact NCMEC or if it was "other law enforcement sources". So which was it? Did this person follow up with the source to confirm whether his hash match was a false positive or a hash collision?

So in short, the source for your claims has evidence of between zero and one false positives.

Unimpressive would be an understatement.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#312

Post by laserlight »

dwaite wrote: post #297
laserlight wrote: post #275

Earlier quoted context omitted.

I would say that it is a back door, because it would work even if iCloud Photos were E2E encrypted. It may not be a generic one, but one that is specific to a purpose Apple decided is rightful. And there is no guarantee that Apple (or authorities) won't decide that there are other rightful purposes.

> I would say that it is a back door, because it would work even if iCloud Photos were E2E encrypted. Backdoor is defined by the Oxford Dictionary as "a feature or defect of a computer system that allows surreptitious unauthorized access to data." The system in question requires you to upload the data to iCloud Photos for the tickets to be meaningful and actionable. Both your phone and iCloud services have EULA which…

I see. My interpretation doesn't hold up given your definitions of back door.

I bet the authorities would be happy with a surveillance mechanism disclosed in the EULA, though. Even if such a system is not technically a back door, I am opposed to it and would prefer Apple to oppose it.

Edit: I just noticed that you had already clarified your argument in other replies. I am sorry to make you repeat it.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#313

Post by dwaite »

LexGray wrote: post #124
kemayo wrote: post #99

Earlier quoted context omitted.

You've misread that page -- the table just refers to content being stored in an encrypted form, even if Apple still possesses a key. There's a list below it of fully end-to-end encrypted content, which doesn't include Photos.

True enough. Apple does have the key. Well then instead the answer is that they are having you pay the CPU and energy cost instead of wasting twice the energy decrypting and hashing your photos server side.

You were sort of both right.

Apple has the keys in escrow across several HSMs separate from the cloud hosting environment. Nothing within iCloud can actually see the content, but there are processes to get access to a particular account for various reasons (such as government order). There is a separation of permissions as well as hardware-generated audit logs. Reportedly the HSM systems were specifically manipulated to not be extensible via software updates.

So it is E2E with a separate key escrow system, which Apple (wisely) does not attempt to call E2E.

Apple couldn't implement the PhotoDNA-based scanning other providers have done because Apple would need access to the escrowed keys for each user photo uploaded.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#314

Post by strogonoff »

chipotle_coyote wrote: post #92

In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :) Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents us…

> For instance, the "it only scans photos uploaded to iCloud" element isn't just an arbitrary limitation that can be flipped with one line of code, as some folks seem to think

It might be a happy incident if their architecture limits this feature to CSAM now, but their ToS are clearly much more general-purpose than that, strategically allowing Apple to pre-screen for any potentially illegal content. If ToS remain phrased this way, surely the implementation will catch up.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#315

Post by simondotau »

0xy wrote: post #291
gandalfgeek wrote: post #138

Sounds like it still comes down to trust. Quoting from the paper: "Apple will refuse all requests to add non-CSAM images to the perceptual CSAM hash database; third party auditors can confirm this through the process outlined before. Apple will also refuse all requests to instruct human reviewers to file reports for anything other than CSAM materials for accounts that exceed the match threshold."

It's already a lie, considering NCMEC's database already has non-CSAM images. It won't ever be true, since the hundreds and possibly thousands of entities with direct and indirect access to the database can upload SunnyMeadow.jpg labeled as CSAM content, and it's blindly accepted. If Apple is receiving a hash labeled as "CSAM", how can they possibly make this guarantee? They do not know it's CSAM. It's unverifiable a…

> "hundreds and possibly thousands of entities with direct and indirect access to the database can upload SunnyMeadow.jpg labeled as CSAM content, and it's blindly accepted."

According to whom?

> "it is known to be a mess already."

According to whom? You've posted a number of claims about the CSAM database administration process on HN and the one instance where you've cited a source, it's been a junk source.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#316

Post by stjohnswarts »

celeritascelery wrote: post #210
Jerry2 wrote: post #126

What strikes me about this paper is that there are no names of the people who wrote it on the title page. Why were they afraid to put their name(s) on this?

Seeing HN over the last week, I can think of a few reasons…

Yeah seeing as its just a white paper there really isn't much need, and the possibility of retribution is immense on this one. I know a lot of pissed off people, let alone some of the nuts who this is angering and who could dox or do worse to the authors.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#317

Post by stjohnswarts »

cmsj wrote: post #57

This seems like a pretty helpful document, and really shows how carefully the CSAM system has been designed. Two things that stuck out to me: 1) The CSAM hash database is encrypted on-device 2) The device doesn't know the results of hash comparisons These suggest to me that the hash comparison is going to happen inside the Secure Enclave - give it the encrypted database and the NeuralHash of an image, it gives you ba…

That's fine, but keep it on the server I don't want this crap on my phone. I don't want to be guilty until proven innocent.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#318

Post by loceng »

chipotle_coyote wrote: post #92

In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :) Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents us…

Could you not potentially infer a lot about a person if they habe X photo content saved - via gathering that data elsewhere?

E.g. a simple one: anyone that may have saved a "winnie the pooh" meme image?

And it's not like Apple's going to publish a list of images that the system searched; though it'd be easy to differentiate between child abuse content vs. other types of images - but would it ever get out of Apple if someone went rogue or was "experimenting" to say "gather stats" of how many people have X image saved?

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#319

Post by sphinxcdi »

gary17the wrote: post #307
ODILON_SATER wrote: post #195

Earlier quoted context omitted.

Speaking of mobile OS. I am a bit of a newbie myself in this area. I am an Android user but I want to decouple from Google as much as possible. Is there an mobile OS out there that offers a similar experience to, say, Android in terms of functionalities, apps, etc without the drawback of privacy concerns?

The problem is that many Android apps require Google services to function properly. You can try two Android derivatives: CalyxOS[1] that implements a privacy-conscious subset of Google services allowing many Android apps to work properly, and GrapheneOS[2] that excludes Google services altogether at the cost of lower app compatibility[3]. Both require using Google Pixel hardware. [1] https://calyxos.org/ [2] https://…

> GrapheneOS[2] that excludes Google services altogether at the cost of lower app compatibility[3].

It now has https://grapheneos.org/usage#sandboxed-play-services providing broader app compatibility.

That video is quite misleading and it's not the best source for accurate information about GrapheneOS.

Re: Security Threat Model Review of the Apple Child Safety Features [pdf]

#320

Post by 0xy »

simondotau wrote: post #311
0xy wrote: post #301

Earlier quoted context omitted.

I know for a fact that it is full of false positives, there's also public sources making the same claim. [1] [1] https://www.hackerfactor.com/blog/index.php?/archives/929-On...

I clicked on your link hoping for serious analysis. I would have settled for interesting analysis. I was disappointed. It's just a guy who found one MD5 hash collision. The paragraph was written in a way that makes it unclear whether source of this specific hash was in fact NCMEC or if it was "other law enforcement sources". So which was it? Did this person follow up with the source to confirm whether his hash match…

It was not a hash collision. It was a false positive. The way the hashing solution works doesn't really allow for a false positive except in extraordinarily rare circumstances.

It matched a man holding a monkey full clothed as a CSAM image, direct from NCMEC's database.

He encountered a 20% false positive rate while running a moderately popular image service, with an admittedly low sample size. It's still evidence, and given NCMEC is immune from oversight, FOIA and accountability, it's concerning.

Also, the fact I know there are false positives does not stem from that post. I know it independently, but since you asked for a source stronger than "just trust a random internet guy", I gave you one.

He's not the only person making the claim though, others throughout these threads with industry knowledge have confirmed what I already knew. If you're asking me to reveal how I know, I'm afraid you'll be disappointed. I'd rather be accused of lying than elaborate.

Post reply on HN