Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

wired.com

Post reply on HN
Print view View original

211–220 of 336 posts

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#211

Post by judge2020 »

no_time wrote: post #52
_m8fo wrote: post #16

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? This is not to say they should scan locally, but my understanding of CSAM was that it would only be scanned on its way to the cloud anyways, so users who didn’t use…

> I’m not sure I understand apples logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? I don’t see the problem with this status quo. There is a clear demarcation between my device and their server. Each serving the interests of their owner. If I have…

This status quo is that a lot of countries want to use the CSAM argument to push privacy-invasive technology (cough UK) like e.g. forcing companies to allow the government to break E2EE to catch CSAM distributors. Apple made this feature while planning to move iCloud Photos to E2EE so that they could argue "look, we still catch x CSAM distributors with n < 0.x% false positive rate, even with E2EE photos. therefore you don't need to pass these laws that break E2EE."

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#212

Post by nemothekid »

Moldoteck wrote: post #108
Gigachad wrote: post #102

Earlier quoted context omitted.

Because the idea is that the iCloud data would be encrypted so their servers couldn’t scan it. With the plan being they would do on device scanning of photos that were marked as being stored on iCloud. It’s objectively better than what google does but I’m glad we somehow ended up with no scanning at all.

that sounds strange, I mean i'm not sure what's the big difference. If data is scanned on icloud, this means it's not encrypted, got it, if scanned on devices, data is fully encrypted on icloud, but apple has access by scanning it on devices and can send unencrypted matches, so it behaves as an unencrypted system, that can be altered at apple's will, just like icloud... but still, why scanning locally only if icloud…

Apple doesn’t want to scan period. However if Apple does E2ee icloud, the biggest political issue will be that of CSAM. So in order to reserve CSAM, they came up with this scheme.

Apple doesn’t want to expand their power which is why they don’t scan locally. They weren’t doing it before and they don’t want to offer it now.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#213

Post by judge2020 »

bryan_w wrote: post #126
theshrike79 wrote: post #42

Earlier quoted context omitted.

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> so that even they can't access it. > scanning stuff on-device What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.

> What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.

In the whitepaper, the cryptography required that Apple have multiple different photodna (or whatever the name was for the on-device one) matches before they could unwrap the user's message containing these suspected CSAM photos and to then send them to NCMEC.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#214

Post by janalsncm »

version_five wrote: post #22
gnfargbl wrote: post #9

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.” Both of th…

Mass surveillance is never an appropriate solution, let's start with that. I don't belive tech has an over weighted responsibility to solve society's problems, and in fact it's generally better if we don't try and pretend more tech is the answer. Advocating for more money and more prioritization for this area of law enforcement is still the way to go if it's a priority area. Policing seems to be drifting towards "mal…

> Mass surveillance is never an appropriate solution, let's start with that.

I think we’re well beyond that point now. Whether or not encryption is allowed or not and however private you believe your virtual life to be, in the physical world surveillance is the norm. Your physical location, biometric information, and relationships can and will be monitored and recorded.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#215

Post by MBCook »

bryan_w wrote: post #126
theshrike79 wrote: post #42

Earlier quoted context omitted.

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> so that even they can't access it. > scanning stuff on-device What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.

That was explained in the original design. Each possible match would count, let’s call it a “point”.

Once you reached a certain threshold (the number was not given) it would trigger an alert in a system at Apple.

Each report contained a bit of data that wasn’t enough to identify someone. Once enough “points” from one account accumulated they’d have enough to identify who you were, which files matched, and presumably the full decryption key.

I believe the plan was the suspect files would be decrypted and compared against the real CSAM signatures. If a close match was found it would be sent to NCMEC for confirmation and law enforcement actions.

The threshold was to prevent false positives from the perceptual hashes, like the Google AI scanning incident. Reportedly nobody has one or two pictures. People with CSAM tend to have a lot, so they’d show up “bright red”. They probably didn’t want to reveal the number so people wouldn’t try to keep only that many pictures on their phone to avoid detection.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#216

Post by jasonlfunk »

Gigachad wrote: post #105
rafale wrote: post #11

False positives would constitute a huge invasion of privacy. Even actual positives would be, a mom taking a private picture of her naked baby, how can you report that. They did well dropping this insane plan. The slippery slope argument is also a solid one.

The apple one was only matching against known images, not trying to detect new ones. The google one actually does try to detect new ones and there are reported instances of Google sending the police on normal parents for photos they took for the doctor.

And it would only notify someone for human review if a certain threshold was reached; just having one or two violating images would have tripped the system.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#217

Post by MBCook »

turquoisevar wrote: post #145
_m8fo wrote: post #16

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? This is not to say they should scan locally, but my understanding of CSAM was that it would only be scanned on its way to the cloud anyways, so users who didn’t use…

> Are iCloud Photos in their data centers not scanned? No outright statement confirming or denying this has ever made to my knowledge, but the implication, based both on Apple's statements and the statement of stakeholders, is that this isn't currently the case. This might come as a surprise to some, because many companies scan for CSAM, but that's done voluntarily because the government can't force companies to scan…

It WASN’T the case. Photos are listed on their page of stuff that’s not end to end encrypted.

Since it all went down they added the advanced security option that encrypts photos, messages, and even more.

But that option is opt-in since if you mess it up they can’t help you recover.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#218

Post by krisoft »

gonehome wrote: post #205
matwood wrote: post #163

Earlier quoted context omitted.

You are correct, the original method would only have scanned items destined to iCloud and only transmitted some hash of matching hashes. And yes, similar slippery arguments exist with any providers that store images unencrypted. They are all scanned today, and we have no idea what they are matched against. I speculated (and now we know) when this new scanning announced, that it was in preparation for full E2EE. Apple…

There are also ways to detect matches even with e2ee iirc and I suspect they found doing that instead easier than dealing with the previous approach. At the time I also thought it was obvious it was in preparation for e2ee (despite loud people on HN who disagreed). I do wonder if they had intended to have it be default on though, maybe not since probably better for most users to have a recovery option.

> At the time I also thought it was obvious it was in preparation for e2ee

I thought the same.

> despite loud people on HN who disagreed

Yeah, loud people be like that, but this is really Apple’s communication fault. They could have started with that “hey we want to provide e2e encrypted storage, the price of it will be that we need to scan what you upload for csam”.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#219

Post by clnq »

bigDinosaur wrote: post #32
gnfargbl wrote: post #9

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.” Both of th…

The extreme hysteria created by anything related to children often seems to be carte blanche to destroy privacy and implement backdoors in applications. Most child abuse comes from family members (which must be solved at the source), and the ultra extreme cases simply make awful law (doing away with E2EE or instituting mass surveillance to catch an incredibly small minority is absurd). Much like other 'tough on crime…

> The extreme hysteria created by anything related to children often seems to be carte blanche to destroy privacy and implement backdoors in applications.

Yes, and you can tell because the proposed solutions attack privacy when alternative solutions exist.

For example, simply deleting CSAM material from devices locally without involving any other parties could have achieved the goals without privacy violations.

It makes me somewhat uncomfortable to argue for not involving other parties (like the police) in cases where real CSAM is found on someone’s device. Same as most people, I think that CSAM is morally reprehensible and really harmful to society. But just deleting it en-masse would have been an effective and privacy-respecting solution.

I think it’s important to see nuance even in things we don’t like to think about. Not everything that has a price tag has a price. We were told we needed to give up privacy, but that wasn’t necessary to take CSAM out of circulation.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#220

Post by figlett »

chatmasta wrote: post #199
figlett wrote: post #84

Earlier quoted context omitted.

Mass surveillance isn't necessarily bad. It depends how it's implemented. The solution you describe is basically how it works with the intelligence agencies, in that only a miniscule fraction of the data collected in bulk ever reaches human eyes. The rest ends up being discarded after the retention period. In terms of outcomes, almost nobody is actually surveilled, as the overall effect is the same as no data having…

If your ex-spouse was a contractor for a government agency with access to the mass surveillance machine, would you still feel comfortable "that only a miniscule fraction of the data collected in bulk ever reaches human eyes?" What if you were a candidate for political office, pushing opinions that angered large swaths of the Intelligence Comminity? The "minuscule fraction" of content is not surfaced by some random ro…

What leads you to believe that access to search these datasets is some sort of unregulated, unmonitored free-for-all for anyone allowed to wander into an intelligence agency building?

The scenarios you invented sound very far-fetched to me, if these did happen I very much doubt the perpetrator would be able to get away with it.

Post reply on HN