Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

wired.com

Post reply on HN
Print view View original

231–240 of 336 posts

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#231

Post by simondotau »

gruturo wrote: post #23

> "Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit" > "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types." Yes, and it was patently obviou…

Apple was concerned about governments using the excuse of CSAM to pass laws which would force Apple to weaken encryption across the board.

Whether this was the right response to such concern is something I’m not unsympathetic towards. Certainly I think it’s reasonable to say that Apple was trying to thread a needle in a way which was never going to please everyone, even if it somehow turns out to have been the least-worst outcome.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#232

Post by kobalsky »

theshrike79 wrote: post #42
_m8fo wrote: post #16

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? This is not to say they should scan locally, but my understanding of CSAM was that it would only be scanned on its way to the cloud anyways, so users who didn’t use…

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> but the internet got a collective hissy-fit of intentionally misunderstanding the feature

how was it misunderstood? your device would scan your photos and notify apple or whoever if something evil was found. wasn't that what they were trying to do?

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#233

Post by robertoandred »

jmyeet wrote: post #68

Part of the reason why this was (and is) a terrible idea is how these companies operate and the cost and stigma of a false negative. Companies don't want to employ people. People are annoying. They make annoying demands like wanting time off and having enough money to not be homeless or starving. AI should be a tool that enhances the productivity of a worker rather than replacing them. Fully automated "safety" system…

Good thing this wasn't fully automated and there would've been human review.

The whole uproar about this system was made by people who didn't know the most basic things about it.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#234

Post by robertoandred »

Simulacra wrote: post #15

I haven't forgot about the guy that sent photos of his child to his doctor and was investigated for child pornography. With these systems, in my humble opinion, you are just one innocent photo at the beach away from your life turned upside down.

That scenario would've been impossible with Apple's system.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#235

Post by salawat »

tzs wrote: post #188
FredPret wrote: post #151

Earlier quoted context omitted.

No, no, no. There is no parallel to be drawn between better encryption and worse outcomes for kids. Should we also outlaw high-performance cars because these sometimes serve as effective getaway vehicles for criminals? CSAM producers and consumers should be found and punished via old-fashioned methods. How was this done in the past? Did we just never catch any human traffickers / rapists? No, we had detectives who we…

> CSAM producers and consumers should be found and punished via old-fashioned methods. How was this done in the past? The "old-fashioned methods" that they used in the past included intercepting communications of people that were suspected of crimes, such as by getting a warrant allowing them to force the person's phone company to record and turn over the person's calls, or by getting a warrant to intercept and inspe…

>The laws are outlawing some applications that make use of mathematics.

Stop equivocating. You're banning the mathematics. The mechanism is literally the mechanical implementation of the mathematics.

>Calling that outlawing mathematics is as absurd as saying that building codes that won't let me use asbestos insulation in new construction are banning sections of thermodynamics.

...Except that's not even an analogous comparison? The asbestos is forbidden not because it's too good an insulator/foiler of thermodynamics, but because of it's danger to the health of everyone.

Trying to ban applications that use encryption is exactly banning asbestos because it's too good an insulator, and you're interested in seeing whatever is wrapped in it burn.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#236

Post by robertoandred »

olliej wrote: post #4
_boffin_ wrote: post #2

So… they’re just doing on device scanning instead of icloud and just calling it a different name?

No, the terrible misfeature that this group wants is “government provides a bunch of opaque hashes that are ‘CSAM’, all images are compared with those hashes, and if the hashes match then the user details are given to police” Note that by design the hashes cannot be audited (though in the legitimate case I don’t imagine doing so would be pleasant), so there’s nothing stopping a malicious party inserting hashes of any…

How exactly would you be able to "filter" LGBT content? I don't think you understand how this system would've worked.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#237

Post by dnissley »

nyolfen wrote: post #226
Shank wrote: post #196

Earlier quoted context omitted.

The issue is that it’s predicated on an age field that can be set separately. It’s easy to use parental controls to control non-children by setting a lower age internally. Think victims of human trafficking or adults in odd relationship situations. Not quite the same but see: https://www.forbes.com/sites/thomasbrewster/2023/04/06/sex-t... Apple’s updated system allows children to ask for help from an adult using the…

> to control non-children[...] Think victims of human trafficking or adults in odd relationship situations. uh, by preventing them from seeing sexual content?

Yeah this doesn't make any sense

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#238

Post by guiambros »

simondotau wrote: post #231
gruturo wrote: post #23

> "Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit" > "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types." Yes, and it was patently obviou…

Apple was concerned about governments using the excuse of CSAM to pass laws which would force Apple to weaken encryption across the board. Whether this was the right response to such concern is something I’m not unsympathetic towards. Certainly I think it’s reasonable to say that Apple was trying to thread a needle in a way which was never going to please everyone, even if it somehow turns out to have been the least-…

Yes, but to OP's point: this was patently obvious from the onset. Even here the comments at the time [1] pointed to all sorts of potential misuse, political or religious prosecution, dystopian cases of false positives, and that this would leave the door open to future government escalation beyond CSAM.

How could they not see that they would have a giant backlash on their hands? Did they overestimate their ability to get away with the "it's for our children" excuse this badly?"

[1] https://news.ycombinator.com/item?id=28068741

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#239

Post by MBCook »

turquoisevar wrote: post #222
MBCook wrote: post #217

Earlier quoted context omitted.

It WASN’T the case. Photos are listed on their page of stuff that’s not end to end encrypted. Since it all went down they added the advanced security option that encrypts photos, messages, and even more. But that option is opt-in since if you mess it up they can’t help you recover.

Non-encryption ≠ CSAM scanning That said, I could be wrong about them not scanning currently, I simply don’t have anything authoritative saying either way. Only statements that imply that they currently don’t, nothing more.

I don’t know if they do or not, but like everyone else I assume they are. Seems like it would be a massive legal (and PR!) liability if it was discovered they weren’t.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#240

Post by barsonme »

judge2020 wrote: post #213
bryan_w wrote: post #126

Earlier quoted context omitted.

> so that even they can't access it. > scanning stuff on-device What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.

> What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement. In the whitepaper, the cryptography required that Apple have multiple different photodna (or whatever the name was for the on-device one) matches before they could unwrap the user's message containing these suspected CSAM photos and to then send them to NCMEC.

Also, IIRC, it wasn’t the raw photos. It was small thumbnails of them.
Post reply on HN