101–110 of 145 posts
Of course. Politicians and officials hate whistleblowers far more than pedophiles, and it's obvious this mechanism will be used to find future Chelsea Mannings and Reality Winners with better opsec to make examples out of them.
Earlier quoted context omitted.
How? I'd love to be convinced that this mechanism could be easily extended to any content at any time without it being totally obvious to the rest of us that it's being used that way.
Not OP and not speaking with any authority on this more just curious: If alongside this Apple rolled out the ability for them to initiate a scan on any phone for any thing (not a tool for mass surveillance but targeted surveillance) how would we know given Apples closed ecosystem? This doesn't necessarily speak to the issue here but more to the Apple ethos in general, but I am curious.
https://twitter.com/pwnallthethings/status/14248736290037022...
Earlier quoted context omitted.
For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.
> you still need to imagine a scenario where Apple is deliberately complicit Which is very easy to imagine. All I have to do is imagine a national security letter.
(As distinct from Apple voluntarily searching for CSAM, which will be part of the terms of service. And distinct from being compelled to search cloud servers, which is exempt from 4A under the "third party doctrine".)
This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ acti…
Earlier quoted context omitted.
For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.
Constantly flagging my account to Apple is an attack of its own. I don't need Apple to be complicit, just make a mistake by erring on the side of caution when reporting.
What does it take to have 30 or more images in your photo library that have been all been manipulated to be a perceptual hash match to 30 or more A1-classified images, while also being able to pass human review ("erring on the side of caution") with respect to them being maybe fitting the A1 classification.
There's no ambiguity in prepubescent minors engaged in sex acts.
While the precedent this sets is indeed concerning, the specific hypotheticals this article give are nonsensical. > an adversary could trick Apple’s algorithm into erroneously matching an existing image In which case the malicious, adversary-controlled images are sent to Apple. After which—the implication is—they can be re-obtained by... the adversary that created them. So what? An adversary could conceivably lower t…
> an adversary could trick Apple’s algorithm into erroneously matching an existing image This is a very real, possible attack. Apple ships its CSAM model on device so any attacker can have a copy of the model. Then the attacker creates an image that triggers CSAM but looks like a panda [1]. Now the attacker sends tons of triggering photos to the unsuspecting victim, who now gets questioned by the FBI. 1: https://medi…
That's glossing over the middle part where a human from Apple (before it even gets to law enforcement) actually look at the images and goes "oh, these are actually pandas" and realizes they were erroneously detected.
Earlier quoted context omitted.
My understanding is that NeuralHash is only supposed to be looking at visible pixels. Granted, I'm taking their word on that one, but I've not seen any evidence indicating that it does look for stenographically concealed images.
Not necessarily steganography. For example if the image has drastically lowered contrast, the lowres image will look like a false positive, but it will not be.
Here's what I don't get: who are these people importing CSAM into their camera roll? I for one have never felt the urge to import regular, legal porn into my camera roll. So why would anyone do that with stuff they know could land them in prison? Who the hell co-mingles their deepest darkest dirtiest secret amongst pictures of their family and last night’s dinner?
If someone wants to conceal their CSAM library, I'm sure there's probably dozens of apps in the App Store that can store photos securely behind an additional layer of encryption.
Earlier quoted context omitted.
> you still need to imagine a scenario where Apple is deliberately complicit Which is very easy to imagine. All I have to do is imagine a national security letter.
A national security letter can do a lot, but it cannot supersede the United States Constitution. Any attempt to co-opt the on-device search mechanism would be an unequivocal violation of the 4th Amendment of the United States Constitution—it's a search of private property being compelled by the Government. (As distinct from Apple voluntarily searching for CSAM, which will be part of the terms of service. And distinct…
Earlier quoted context omitted.
I'm not arguing that these aren't real problems, but neither are unique problems of the the client side scanning solution. It's the same or maybe even worse with server side scanning. I'd assume FB, Google & co have some solution to b), so Apple should be able to figure out something. For c), at least Apple takes extra precautions by requiring the photos to be in two separate database provided by different government…
Yeah I agree that B is likely a non issue, at least not an issue that affects the users directly. But C doesn't make me feel any better because the fundamental issue is where the scanning happens. If it happens off my device it cannot happen to photos I don't send off my device. I understand they have policy governing this, but that's not addressing the core conceptual problem of crossing the network boundary onto wh…
But the second you enable iCloud Photos sync, Apple has the right to not allow CSAM on THEIR servers.
Earlier quoted context omitted.
A national security letter can do a lot, but it cannot supersede the United States Constitution. Any attempt to co-opt the on-device search mechanism would be an unequivocal violation of the 4th Amendment of the United States Constitution—it's a search of private property being compelled by the Government. (As distinct from Apple voluntarily searching for CSAM, which will be part of the terms of service. And distinct…
If you receive an unconstitutional National Security Letter, you can't just ignore it. You must endure a lengthy, expensive, and stressful legal battle. And due to the nondisclosure requirement in the NSL, you also have to do this in secret, without going to the public for support. Or you can fold and comply.
So the letter goes to Apple. They have ample time and resources to push back indefinitely. Demanding that Apple implement a Government dragnet across tens of millions of private devices is so far beyond unconstitutional that complying wouldn't even be fleetingly contemplated as an option. In fact, Apple is the sort of company that would move heaven and earth to ensure that this unconstitutional NSL becomes public. If nothing else, their defiance of it would be fantastic PR.
Therefore it would be a massively stupid-ass move for the Government to try. They would have zero prospect of a positive outcome and they know it.