Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
1–10 of 336 posts
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#2So… they’re just doing on device scanning instead of icloud and just calling it a different name?
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#3So… they’re just doing on device scanning instead of icloud and just calling it a different name?
Not really? It looks like the nudity detection features are all on device, aren't CSAM specific, and seem to be mostly geared towards blocking stuff like unsolicited dick pics.
The earlier design was a hybrid model that scanned for CSAM on device, then flagged files were reviewed on upload.
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#4So… they’re just doing on device scanning instead of icloud and just calling it a different name?
No, the terrible misfeature that this group wants is “government provides a bunch of opaque hashes that are ‘CSAM’, all images are compared with those hashes, and if the hashes match then the user details are given to police”
Note that by design the hashes cannot be audited (though in the legitimate case I don’t imagine doing so would be pleasant), so there’s nothing stopping a malicious party inserting hashes of anything they want - and then the news report will be “person x bought in for questioning after CSAM detector flagged them”.
That’s before countries just pass explicit laws saying that the filter must includE LGBT content (in the US several states consider books with lgbt characters to be sexual content, so a lgbt teenager would be de facto CSAM), in the UK the IPA is used to catch people not collecting dog poop so trusting them not to expand scope is laughable, in Iran a picture of a woman without a hijab would obviously be reportable, etc
What Apple has done is add the ability to filter content (eg block dick picks) and for child accounts to place extra steps (incl providing contact numbers I think?) if a child attempts to send pics with nudity, etc
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#5So… they’re just doing on device scanning instead of icloud and just calling it a different name?
Yes - and there’s a huge difference between the two.
In a word, decentralization.
By detecting unsafe material on-device / while it is being created, they can prevent it from being shared. And because this happens on individual devices, Apple doesn’t need to know what’s on people’s iCloud. So they can offer end-to-end encryption, where even the data on their servers is encrypted. Only your devices can “see” it (it’s a black box for Apple servers, gibberish - without the correct decryption key).
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#6So… they’re just doing on device scanning instead of icloud and just calling it a different name?
No, the terrible misfeature that this group wants is “government provides a bunch of opaque hashes that are ‘CSAM’, all images are compared with those hashes, and if the hashes match then the user details are given to police” Note that by design the hashes cannot be audited (though in the legitimate case I don’t imagine doing so would be pleasant), so there’s nothing stopping a malicious party inserting hashes of any…
>in the UK the IPA is used to catch people not collecting dog poop
What does this mean? What is IPA? I tried Googling for it but I’m not finding much. I would love to learn more about that
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#7Earlier quoted context omitted.
No, the terrible misfeature that this group wants is “government provides a bunch of opaque hashes that are ‘CSAM’, all images are compared with those hashes, and if the hashes match then the user details are given to police” Note that by design the hashes cannot be audited (though in the legitimate case I don’t imagine doing so would be pleasant), so there’s nothing stopping a malicious party inserting hashes of any…
>in the UK the IPA is used to catch people not collecting dog poop What does this mean? What is IPA? I tried Googling for it but I’m not finding much. I would love to learn more about that
The investigatory powers act.
It was passed to stop terrorism, because previously they found that having multiple people (friends and family etc) report that someone was planning a terrorist attack failed to stop a terrorist attack.
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#8The who is often interesting with these stories.
> a new child safety group known as Heat Initiative
Doesn't even have a website or any kind of social media presence; it literally doesn't appear to exist apart from the reporting on Apple's response to them, which is entirely based on Apple sharing their response with media, not the group interacting with media.
> Sarah Gardner
on the other hand previously appeared as the VP of External Affairs (i.e. Marketing) of Thorn (formerly DNA Foundation): https://www.thorn.org/blog/searching-for-a-child-in-a-privat...
So despite looking a bit fishy at first, this doesn't seem to come from a christofascist group.
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#9> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.”
Both of these arguments are absolutely, unambiguously, correct.
The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. This is, I argue, a bad thing. Is is bad for the individuals who are re-victimised on every share. It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it.
Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Does the industry feel that it has any responsibility at all to do so? Or do we all just shout "yay, individual freedom wins again!" and forget about the actual problem that this (misguided) initiative was originally aimed at?
Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos
#10The who is often interesting with these stories. > a new child safety group known as Heat Initiative Doesn't even have a website or any kind of social media presence; it literally doesn't appear to exist apart from the reporting on Apple's response to them, which is entirely based on Apple sharing their response with media, not the group interacting with media. > Sarah Gardner on the other hand previously appeared as…
> So despite looking a bit fishy at first, this doesn't seem to come from a christofascist group.
Why would you assume this in the first place?