Apple’s device surveillance plan is a threat to user privacy – and press freedom

Post by intricatedetail » Wed, Aug 18, 2021, 10:57 PM UTC

kemayo wrote: post #17

shadowfacts wrote: post #14

While the precedent this sets is indeed concerning, the specific hypotheticals this article give are nonsensical. > an adversary could trick Apple’s algorithm into erroneously matching an existing image In which case the malicious, adversary-controlled images are sent to Apple. After which—the implication is—they can be re-obtained by... the adversary that created them. So what? An adversary could conceivably lower t…

> An adversary would either have to: More importantly, they'd need to suborn Apple's human-review process, because the people doing the review would need to know what not-CSAM they're looking for. Could Apple be coerced into (or willingly) do this? I have no idea. But it's a very different threat-model than the article suggests, that boils down to the "do you trust your OS vendor?" question.

Post by cyanite » Wed, Aug 18, 2021, 10:59 PM UTC

35803288 wrote: post #19

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ acti…

Post by intricatedetail » Wed, Aug 18, 2021, 11:02 PM UTC

kemayo wrote: post #15

mattnewton wrote: post #8

Earlier quoted context omitted.

For b), I can't see how creating spurious icloud accounts and spoofing it will be hard at it's face. I have made dozens of icloud accounts personally for testing in the past. They may take steps to address this attack vector, it's not an unsolvable problem but it isn't solved by requiring a device or icloud account as far as I can tell. For c), the problem isn't just trusting Apple, for whom the hashes are somewhat o…

For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.

Post by sparker72678 » Wed, Aug 18, 2021, 11:04 PM UTC

35803288 wrote: post #22

35803288 wrote: post #19

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ acti…

Ten years from now we will read a leaked document stating that US authorities requested FISA Courts warrants (or similar) and made Apple scan for things other than csam. It already happened. Court orders are easier than hacks once you iron out the legal opinions.

Post by kemayo » Wed, Aug 18, 2021, 11:07 PM UTC

intricatedetail wrote: post #25

kemayo wrote: post #15

Earlier quoted context omitted.

For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.

How would they know it is a false positive and not a CSAM image concealed in a document image that NeuralHash was able to "see"? It will be impossible to tell from a lowres picture with certainty.