Live data from Hacker News

Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

wired.com

291–300 of 336 posts

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#291

Earlier quoted context omitted.

Also, IIRC, it wasn’t the raw photos. It was small thumbnails of them.

"reduced-quality copy" was the wording in the whitepaper IIRC. So the resolution most likely would've been the same, but the detail blurred so that the poor human agent wouldn't have to see actual CSAM, just enough to make a call whether it is or isn't a likely match.

It was called a “visual derivative”.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#292

Earlier quoted context omitted.

Apple was concerned about governments using the excuse of CSAM to pass laws which would force Apple to weaken encryption across the board. Whether this was the right response to such concern is something I’m not unsympathetic towards. Certainly I think it’s reasonable to say that Apple was trying to thread a needle in a way which was never going to please everyone, even if it somehow turns out to have been the least-…

Yes, but to OP's point: this was patently obvious from the onset. Even here the comments at the time [1] pointed to all sorts of potential misuse, political or religious prosecution, dystopian cases of false positives, and that this would leave the door open to future government escalation beyond CSAM. How could they not see that they would have a giant backlash on their hands? Did they overestimate their ability to…

So how come there is no public backlash against Microsoft and Google who scan your cloud pictures in the cloud, enabling political and religious persecution, dystopian cases of false positives, and future government escalation?

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#293

Earlier quoted context omitted.

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> so that even they can't access it. > scanning stuff on-device What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.

No. A small thumbnail “visual derivative” is included with the neural hash, which is unlocked (only for matches) only once the number of matches exceeds a threshold.

This was all outlined in the first two pages of the white paper, and explained in more detail further down.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#294
post #52

Earlier quoted context omitted.

> I’m not sure I understand apples logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? I don’t see the problem with this status quo. There is a clear demarcation between my device and their server. Each serving the interests of their owner. If I have…

Precisely! The software running on the phone should be representing the owner of the phone, period. We begrudgingly accept cloud scanning because that ship has already sailed, despite it being a violation of the analog of fiduciary duty. But setting the precedent that software on a user's device should be running actions that betray the user is from the same authoritarian vein as remote attestation. The option ignore…

1. What you’re asking for (“The option … where the device encrypts files before uploading them to iCloud, iCloud may scan the encrypted bits anyway to do their legal duty, and that's the end of the story.”) is impossible.

2. The division you envisage (“The software running on the phone should be representing the owner of the phone, period.”) is wishful thinking. Do you think the JavaScript in your browser does only things in your interest?

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#295
post #264

Earlier quoted context omitted.

I want to be clear that I agree with you and I am not providing this explanation as an excuse for Apple, but merely as an explanation for what might have happened with the timeline: first they announced they were doing this, a bunch of us said "no this is the first step towards breaking e2e entirely", and THEN this year there was the high-profile issue--note that I am not saying it is a new issue, but merely that it…

> and THEN this year there was the high-profile issue with the laws in the UK and/or Australia or whatever that showed we were all correct Do you have a link to that issue? I didn't heard about it (I'm from a third world country)

No, as I didn't even know which country it was from for sure, right? ;P But I went ahead and typed "UK encryption" into the Hacker News search and it all popped up from the last few months; the first hit, from a month ago, was even about Apple actively scrambling to push back on it (lending even more support to my point).

https://news.ycombinator.com/item?id=36800297

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#296

Earlier quoted context omitted.

People DID get into a huge fuss and started building false positive generators in a huge wave. Like they were proving something or pwning apple. Nobody read the bit about an actual human verifying results before any law enforcement would be called in. And outsourcing checking is a huge industry even today[0]. How do you think the huge social media companies keep CSAM, gore etc out of their systems? They're not using…

And outsourcing checking is a huge industry even today[0]. How do you think the huge social media companies keep CSAM, gore etc out of their systems? They're not using Pied Piper's hotdog or not algorithm, that's for sure. - is this done for private data or public? Public data check can be outsourced no prob, not sure abt private. Nobody read the bit about an actual human verifying results before any law enforcement…

You can imagine all you want, but you're still wrong.

You can generate a billion false positives and it still won't do anything. You need to get them to people's iCloud photo libraries first. Each library needs to have multiple false positive images before triggering a human check. They intentionally didn't tell the exact number needed, but it's not two.

If you have a way to get enough people to get fake CSAM material on their phones to overwhelm the human checkers, why would you waste in on something stupid like that? Just by forcibly inserting advertisements to peoples Photo Libraries would make you a billionaire. Not an ethical one, but still rich.

Oh, and just for reference. FB gets 350 million photos uploaded every day and they keep it moderated just fine. The amount of people you'd need colluding with you to overwhelm the system Apple had designed would be staggering.

And then you've achieved what? Make it possible to share child pornography because you broke the system? Yay, victory?

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#297
post #289

Earlier quoted context omitted.

This is what I was recalling, this method gives you a clever way to do it using the file itself as the key: > “Convergent encryption solves this problem in a very clever way: “The way to make sure that every unique user with the same file ends up with an encrypted version of that file that is also identical is to ensure they use the same key. However, you can’t share keys between users, because that defeats the entir…

That’s not what’s commonly understood to be a modern cipher. It would be trivial for a government to make a list of undesired messages/images and find everyone that has forwarded it. https://en.wikipedia.org/wiki/Chosen-plaintext_attack

Yeah that’s literally the point in the CSAM case.

For regular people taking photos the government won’t have their plaintext.

For popular media people are uploading the same copy of it saves a lot of bandwidth.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#298
post #294

Earlier quoted context omitted.

Precisely! The software running on the phone should be representing the owner of the phone, period. We begrudgingly accept cloud scanning because that ship has already sailed, despite it being a violation of the analog of fiduciary duty. But setting the precedent that software on a user's device should be running actions that betray the user is from the same authoritarian vein as remote attestation. The option ignore…

1. What you’re asking for (“The option … where the device encrypts files before uploading them to iCloud, iCloud may scan the encrypted bits anyway to do their legal duty, and that's the end of the story.”) is impossible. 2. The division you envisage (“The software running on the phone should be representing the owner of the phone, period.”) is wishful thinking. Do you think the JavaScript in your browser does only t…

A state of affairs where users' devices encrypt files, and then iCloud scans the stored blobs to perform a perfunctory compliance check is clearly not impossible. So please describe what you mean.

Web javascript is one of the places the battle is being fought. Users are being pushed into running javascript (and HTML) that acts directly against our own interests (eg ads, surveillance, etc). Many of the capabilities exploited by the hostile code should be considered browser security vulnerabilities, but the dynamic is not helped by one of the main surveillance companies also making one of the main browsers.

But regardless of the regime the authoritarians are trying to push, the computer-represents-user model is what we should aspire to - the alternative is computational disenfranchisement.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#299
post #294

Earlier quoted context omitted.

Precisely! The software running on the phone should be representing the owner of the phone, period. We begrudgingly accept cloud scanning because that ship has already sailed, despite it being a violation of the analog of fiduciary duty. But setting the precedent that software on a user's device should be running actions that betray the user is from the same authoritarian vein as remote attestation. The option ignore…

1. What you’re asking for (“The option … where the device encrypts files before uploading them to iCloud, iCloud may scan the encrypted bits anyway to do their legal duty, and that's the end of the story.”) is impossible. 2. The division you envisage (“The software running on the phone should be representing the owner of the phone, period.”) is wishful thinking. Do you think the JavaScript in your browser does only t…

> The division you envisage (“The software running on the phone should be representing the owner of the phone, period.”) is wishful thinking.

In this specific case it is not wishful thinking.

The feature got scrapped. Users and people who support privacy won.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#300

Earlier quoted context omitted.

Section 230 removes liability for restricting good faith attempts to combat CSAM. > (2) Civil liability > No provider or user of an interactive computer service shall be held liable on account of— > (A) any action voluntarily taken in good faith to restrict access to or availability of material that the provider or user considers to be obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise ob…

I don't see any reference to child porn there. Who decides what's obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable? Especially otherwise objectionable?

> Who decides

Judges and juries.

And I can assure you that every single judge in the USA, and almost every single member of a jury would decide that CSAM is obscene.

Thats how the law works. We have tons of laws that use general words like this, and trying to be "clever" usually just results in a lost court case or prison time for the person who thinks they found a loophole.

Post reply on HN