Apple’s new abuse prevention system: an antritust/competition point of view

Some website reporting you to the police for doing something illegal is not the same as your hardware/software being stuffed with snakeoil spyware that slows down the UI all for some made up cause.

I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In(!).

HN likes to downplay the actual reality as if it's always some trojan horse, but the issue is real. It's worth talking to people that work on combatting it if you get the chance. I'm not really commenting on Apple's approach here (which I haven't thought enough about), but I know enough that an immediate dismissal based on it 'not helping' is not really appreciating the real tradeoffs you're making.

You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. Maybe that's the proper tradeoff, but we shouldn't pretend it's not a tradeoff at all.

"Robin Hanson proposed stores where banned products could be sold. There are a number of excellent arguments for such a policy—an inherent right of individual liberty, the career incentive of bureaucrats to prohibit everything, legislators being just as biased as individuals. But even so (I replied), some poor, honest, not overwhelmingly educated mother of five children is going to go into these stores and buy a “Dr. Snakeoil’s Sulfuric Acid Drink” for her arthritis and die, leaving her orphans to weep on national television" [0]

[0]: https://www.lesswrong.com/posts/PeSzc9JTBxhaYRp9b/policy-deb...

Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.

For iMessage child account ML scanning it’s on device and just generic sexual material restriction - more similar to standard parental controls than anything else (alerts only go to the parent, and only happen on child accounts)

My initial impression is this is a good approach. The risk mostly comes from future applications (like the CCP adding hashes of other stuff that isn’t CSAM like tankman or something).

The frankly ignorant knee-jerk responses from technical HN readers do a disservice and weaken the ability of technical people to push back when necessary.

Problem is that this scanning is necessarily fuzzy and there is going to be a false positive rate to it. And the way that you'll find out that you've tripped a false positive is that the SWAT team will knock your door down and kill your dog (at a minimum). Then you'll be stuck in a Kafkaesque nightmare trying to prove your innocence where you've been accused by a quasi Governmental agency that hides its methods so the "bad guys" can't work around them.

It isn't just "authoritarian regimes" abusing it, it is the stochastic domestic terrorism that our own government currently carries out against its own citizens every time there's a beaurocratic fuckup in how it manages its monopoly on violence.

This is the "Apple/Google cancelled my account and I don't know why" problem combined with SWATing.