Apple clarifies why it abandoned plan to detect CSAM in iCloud photos • Hacker News Board

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#121

Post by danpalmer » Fri, Sep 01, 2023, 2:30 PM UTC

Moldoteck wrote: post #51

danpalmer wrote: post #26

Earlier quoted context omitted.

Mass surveillance is bad, but I think there are versions of it that are far less bad than others. Apple's proposed solution would have theoretically only reported cases that were much more than likely to be already known instances of CSAM (i.e. not pictures of your kids), and if nothing else is reported, can we say that they were really surveilled? In some very strict sense, yes, but in terms of outcomes, no.

ok, and in theory, with new generative algorithms, do you think it's still ok? Suppose apple implements this, suppose someone finds a way to generate meme images that can trigger apple's algorithm(but human can't see anything wrong), suppose that someone wants to harm you and sends you a bunch of memes and you save them. What will happen? Or what does happen if somebody is using generative algorithm to create csam li…

These criticisms are reasonable criticisms of a system in general, but Apple's design featured ways to mitigate these issues.

I agree that the basic idea of scanning on device for CSAM has a lot of issues and should not be implemented. What I think was missing from the discourse was an actual look at what Apple were suggesting, in terms of technical specifics, and why that would be well designed to not suffer from these problems.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#122

Post by Shank » Fri, Sep 01, 2023, 2:49 PM UTC

theshrike79 wrote: post #42

_m8fo wrote: post #16

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? This is not to say they should scan locally, but my understanding of CSAM was that it would only be scanned on its way to the cloud anyways, so users who didn’t use…

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it.

They basically did. If you turn on Advanced Data Protection, you get all of the encryption benefits, sans scanning. The interesting thing is that if you turn on ADP though, binary file hashes are unencrypted on iCloud, which would theoretically allow someone to ask for those hashes in a legal request. But it's obviously not as useful for CSAM detection, as, say, PhotoDNA hashes. See: https://support.apple.com/en-us/HT202303

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#123

Post by tomjen3 » Fri, Sep 01, 2023, 2:55 PM UTC

gnfargbl wrote: post #9

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.” Both of th…

The real issue is that FBI/NSA/CIA has abused our trust in the so completely that we have to make E2E communication. From assasinating people like Fred Hampton to national security letters, the government has completely lost the trust of tech.

That is a bigger problem and it will take a long time to fix. So long that I suspect that anybody reading this is long dead, but its like the saying with planting trees.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#124

Post by danaris » Fri, Sep 01, 2023, 2:55 PM UTC

rob74 wrote: post #65

PrimeMcFly wrote: post #12

Earlier quoted context omitted.

It's an incredibly bad thing. It's also an incredibly poor excuse to justify backdooring phones. Cops need to investigate the same way they always have, look for clues, go undercover, infiltrate, find where this stuff is actually being made, etc. Scanning everyone's phones would make their jobs significantly easier, no doubt, but it simply isn't worth the cost to us as a society and there is simply no good counter-ar…

If CSAM was still done the way it "always has been", then "cops" relying on the methods they always had would be a valid answer. But since tech has enabled the distribution of CSAM at unprecedented scales, I think the requests by law enforcement to also make their job a bit easier have some merit...

The technology has changed distribution, yes.

It hasn't particularly changed production, which is where the actual abuse happens. There are still actual people abusing and filming actual children, and those can be found by the police by the same old-fashioned methods they've always had available. (Plus many new ones that don't violate everyone's civil liberties or destroy the security of every networked device.)

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#125

Post by tomjen3 » Fri, Sep 01, 2023, 3:05 PM UTC

figlett wrote: post #10

formerly_proven wrote: post #8

The who is often interesting with these stories. > a new child safety group known as Heat Initiative Doesn't even have a website or any kind of social media presence; it literally doesn't appear to exist apart from the reporting on Apple's response to them, which is entirely based on Apple sharing their response with media, not the group interacting with media. > Sarah Gardner on the other hand previously appeared as…

> So despite looking a bit fishy at first, this doesn't seem to come from a christofascist group. Why would you assume this in the first place?

Because when they couldn't win the war on porn, some right Christians decided to cloak their attack in "concerns" of "abuse". See project Excedus. Of course it has nothing to do with abuse and everything to do with their attempts to keep people from seeing pixels of other people having sex. Backpage was shut down despite being good at removing underage and trafficed women - which meant that sex workers had to find other places that didn't have nearly as good protections.

So yeah. When these things pop up I assume malicious intent.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#126

Post by bryan_w » Fri, Sep 01, 2023, 3:10 PM UTC

theshrike79 wrote: post #42

_m8fo wrote: post #16

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? This is not to say they should scan locally, but my understanding of CSAM was that it would only be scanned on its way to the cloud anyways, so users who didn’t use…

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> so that even they can't access it.

> scanning stuff on-device

What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#127

Post by jtbayly » Fri, Sep 01, 2023, 3:21 PM UTC

flir wrote: post #119

jtbayly wrote: post #115

Earlier quoted context omitted.

It’s guaranteed by our constitution, among other reasons. Search of my communications for no reason is, by definition “unreasonable search.”

That document with 27 amendments?

Yes?

If you want to get it amended, then by all means, make a case for why it should be amended.

In the meantime you wanted to know why mass surveillance isn’t an option. The answer “because it’s against the law” is a simple, good answer.

If you want to know why we decided as a nation to make that such a fundamental law that it is in our constitution, you could do worse than reading about what prompted the writing of the Bill of Rights.

I agree with a lot of the original reasoning.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#128

Post by danbruc » Fri, Sep 01, 2023, 3:31 PM UTC

jtbayly wrote: post #127

flir wrote: post #119

Earlier quoted context omitted.

That document with 27 amendments?

Yes? If you want to get it amended, then by all means, make a case for why it should be amended. In the meantime you wanted to know why mass surveillance isn’t an option. The answer “because it’s against the law” is a simple, good answer. If you want to know why we decided as a nation to make that such a fundamental law that it is in our constitution, you could do worse than reading about what prompted the writing of…

The answer “because it’s against the law” is a simple, good answer.

While often true, at all times there have also been morally wrong laws, so it would not be unreasonable to counter that being written into law on itself means nothing. So you should always be prepared to pull out and defend the reasoning behind a law, which you also hinted at in your following sentences.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#129

Post by traceroute66 » Fri, Sep 01, 2023, 3:45 PM UTC

ben_w wrote: post #83

traceroute66 wrote: post #71

Earlier quoted context omitted.

> criminals are using E2EE communication systems to share sexual abuse material Blah blah blah, the same old argument given by the "think of the children" people. There are many ways to counter that old chestnut, but really, we only need to remember the most basic fundamental facts: 1) Encryption is mathematics 2) Criminals are criminals Can you ban mathematics ? No. Can you stop criminals being criminals ? No. So, l…

> But the criminals ? Do you honestly think they'll think "oh no, game over" ? > No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual. I used to think this, I changed my mind: just as it's difficult to do security correctly even when it's a legal requirement, only the most competent criminal organisations will do this correctly. Unfortunatel…

> only the most competent criminal organisations will do this correctly.

All it takes is for one criminal to write a one-page guide to using GPG and circulate it to the group ....

I know I mentioned paying a cryptographer earlier, but in reality downloading and using GPG is a crude and effective way of defeating an E2EE backdoor.

Are the GPG devs going to backdoor GPG to satisfy governments ? Probably not.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#130

Post by figlett » Fri, Sep 01, 2023, 3:51 PM UTC

jtbayly wrote: post #116

figlett wrote: post #109

Earlier quoted context omitted.

Why is it not okay at all? That's what our intelligence agencies do with their bulk data collection capabilities, and they have an immense positive impact on society.

If you want to argue that they can scan people outside the country and not US citizens, and that that has a benefit, go ahead and make that argument. You might even convince me. But it’s just begging the question to say there’s immense benefit to them searching US citizens’ communications without a reason. That’s the whole question. Show me why we should change the constitution which guarantees us freedom from this s…

I'm writing from a UK perspective so there's no underlying constitutional issue here like there might be in the US. Bulk data collection is restricted by specific laws and this mandates regular operational oversight by an independent body, to ensure that both the collection and each individual use of the data is necessary and proportionate.

Some of this will include data of British citizens, but the thing is, we have a significant home-grown terrorism problem and serious organised criminal gang activity, happening within the country. If intelligence analysts need to look at, for example, which phone number contacted which other phone number on a specific date in the recent past, there's no other way to do this other than bulk collect all phone call metadata from the various telecom operators, and store it ready for searching.

The vast majority of that data will never be seen by human eyes, only indexed and searched by automated systems. All my phone calls and internet activity will be in there somewhere, I'm sure, but I don't consider that in itself to be government oppression. Only if it's used for oppressive purposes, would it become oppressive.