Apple says it'll remove iMessage and FaceTime in UK rather than break encryption

Apple's solution was to run every photo through a perceptual hash algorithm with a neural network inside of it and then compare that against a list of known hashes. This is actually not materially different from the system that every image host has used for a while now. The main difference was that they were going to use a bunch of cryptography to ensure that...

- Images could be scanned only after a threshold number of illegal images had been flagged

- Law enforcement agencies could not use the system to flag non-CSAM images

Part of the way they achieve the former - and what makes the system unworkable - is that it's client-sided.

To be clear, every service already scans for CSAM. But the naive way of doing this - comparing SHA-2 message digests - can be trivially manipulated into producing false negatives. Just flip a bit in the image and it becomes a new image. Perceptual hash algorithms instead use notions of image similarity to protect against this, but it also introduces the possibility of false positives - i.e. images that look innocuous but match against CSAM and get you flagged. This is not merely a problem to be solved, but a consequence of such systems being differentiable, which is necessary for them to actually work.

For remote scans, you can at least keep the perceptual hash algorithm secret and avoid leaking information about what images do and don't get flagged. This will frustrate attempts at adversarial training. However, when you put the perceptual hash on everyone's phone, it can be extracted, and people who don't like the idea of running a spying dragnet will deliberately break the hash just to tell you to fuck off. So people were making tools that would modify one image to look like another, which could be used to either hide CSAM or, worse, making cat pictures that get your iCloud flagged.

There's also a bit of moral insult involved with having your phone do things that aren't in your benefit, even if this ostensibly were to keep the feds from demanding explicit backdoors[0]. This system is laughably easy to defeat if you're jailbroken - just turn the daemon off. Hence why Apple made no attempt to integrate it into macOS where users have root access[1].

Law enforcement would never firewall their capabilities to "just CSAM." That's not how the law works. You can't sue your drug dealer for not giving you the weed you paid for, and for similar reasons, law enforcement is never going to only wiretap pedophiles and not drug dealers. There's a few exceptions to this[2], but generally, you should assume that if you're about to give the feds the legal capability to spy, it will be used for every crime down to and potentially including routine traffic stops.

Apple's bulwark against that was to have the system only scan for images that were flagged by multiple independent child protection agencies. However, this isn't a guarantee; the western world routinely collaborates in very not independent ways. There's no guarantee that, say, Europe's pedo-fighters wouldn't have had their arms twisted by their countries' intelligence apparatus, operating in concert with the American CIA who was arm-twisting NCMEC. Apple promised they'd be reviewing all reports before forwarding them to law enforcement, but in this situation there's nothing preventing them from having their arms twisted here, too[3].

[0] For the record, there was no evidence that Apple was using this system to enable iCloud end-to-end encryption. They'd already turned that off because of a UX problem: too many people were permanently locking themselves out of their own data and Apple couldn't break into it for them.

[1] Root access on macOS is complicated by the fact that SIP and boot security exists. They could at least theoretically have made the kernel refuse to terminate the CSAM scanner process at all, but that would be a speed bump at best. Users absolutely can turn off SIP or boot modified kernels on Macs that would remove whatever protection Apple added to the CSAM scanner. And Apple currently has no interest in locking owners out of macOS like they did with iOS.

[2] The NSA actually does try to keep their spying tools to themselves, because they have to fight nation-states, not criminals. Even then, they'll still slip hints under the door and tell the FBI they can only use them if they can make up a story that doesn't expose how the NSA's spying apparatus works.

[3] National Security Letters are one hell of a drug.

Apple could also have their arm twisted to add backdoors, but they've already shown that they'll yell loudly if that happens. Furthermore, the existence of software signatures means that such a backdoor would generate irrefutable paper trails, which is the sort of thing that intelligence agencies hate. It's much easier for them to just quietly extend an existing quasi-backdoor than to create an explicit one that loudly screams "DO NOT USE YOUR PHONE TO DO CRIMES"

Yea, and even if it is actionable and "real" - they're .. i think, correct me if i'm wrong, by law a for-profit company. Meaning none of this is noble, this is just aligned with our interests because they believe that to be the most profitable image for them currently.

Which is all well a good for now, but profit directions can change, and i suspect it's easier to change a direction guided by profit than a direction guided by morals. Ie i suspect their current "good direction" is less stable than some would suggest.

> The UK keeps trying to throw their weight around only to find out they aren’t that big.

Their (former) empire is a part of their cultural identity. It's going to take a few more generations to shed the expectations of what was.

The days of two washed-up subjects of the crown taking Kafiristan all on their lonesome are dead and gone.

So what you're really complaining about is that other people will not follow your choices. If your relations have such a hard line that a single app is "the only way to interact with certain groups of people", they are the ones to blame, not the maker of the app.

While technically true, it's also a complete BS. It's in the same vain as Torrent websites are not hosting copyrighted content - technically true but complete BS.