Live data from Hacker News

Apple defeats liability for not scanning iCloud for CSAM

blog.ericgoldman.org

131–140 of 597 posts

Re: Apple defeats liability for not scanning iCloud for CSAM

#131
As the creator of mediaden.ca[1] I’ve thought about this. Client side scanning is maybe marginally better than server side scanning, but both paths lead to privacy rot.

Governments need to catch criminals, but they shouldn’t do it at everyone else’s expense.

1. https://mediaden.ca

Re: Apple defeats liability for not scanning iCloud for CSAM

#132
post #123

Earlier quoted context omitted.

I still also totally don't get their policy. Trying to avoid false positives by not firing until a threshold was hit (was it 20 images?) seemed insane from a PR position... rightly or wrongly, all it would take would be the wrong court case and you can see the headlines: "Apple says users can have up to 20 CSAM images on their phone before they'll tell police"

Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM? I think the program would have to say use but morally I'd say no. The issue with client scanning is it has to assume the worst, or they are than liable. If its the person has 20+ different baptism of nude babys well huh that actually might be CSAM because the context of how that concentrated photos implies but even than its hard what if t…

> America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures

The US has the largest pornography industry in the world by a massive margin, and the largest consumption of online pornography per capita

Meanwhile should we be surprised that CSAM production is higher in countries like the Philippines that have very weak digital policing, abject poverty, high numbers of street children etc?

Re: Apple defeats liability for not scanning iCloud for CSAM

#133
post #14

Earlier quoted context omitted.

> It is crazy people think apple isnt on the side of privacy. > It also ensured pressure from governments and plaintiffs, including CSAM victims, who preferred Apple’s more interventionist approaches, which Apple had voluntarily demonstrated it was willing to do. I feel that Apple open pandora's box with the client-side scanning. It proved that it was technically feasible, and was "privacy preserving". I use scare qu…

I thought the client side scanning was to protect children? If it suspects an image is bad, it blurs it and pops up a warning including a link to resources to go to for help. Very different than trying to narc out users to the authorities.

The original proposal was to use a visual hash designed to identify known bad CSAM images even if cropped or otherwise edited. Your computer would scan all your stuff for these images and report you to apple who would report you to the cops. This presented a number of issues.

Accidental false positives could lead to horrific outcomes up to and including oh look bob got shot by the cops for resisting.

It was possible to produce apparently matching innocuous images and then poison people's machines with them.Oops did you click on that picture of a tree have fun with the cops. Like an advanced form of swatting.

Although inspired by a desire to find CSAM Apple could be forced to scan for ANYTHING by repressive regimes including America and China.

Although initially targeting images client side scanning of messages is a pretty obvious next step. Again obvious good motivation exists and is completely justifiable who doesn't want to stop the next mass shooting or terrorist attack... and then we can basically use it to find people critical of the regime. Do remember we are presently prosecuting a political figure for a picture of sea shells and a guy in texas is rotting in prison for distributing political literature.

Re: Apple defeats liability for not scanning iCloud for CSAM

#134

Earlier quoted context omitted.

> It proved that it was technically feasible, and was "privacy preserving". Didn't their paper disproved by reversing the perceptual hashes to reveal blurred version of the images being hashed, and Apple basically said "that's fair, it's not as robust as we wanted, let's visit this later"? If not, I'll happily stand corrected, but please share sources. Addenda: - Apple's original paper: https://web.archive.org/web/20…

> to reveal blurred version of the images being hashed Skimmed your linked paper. It seems they were able to classify hashes up to ~8% top-1 accuracy and ~30% top-10. Not exactly a blurred version, or any images at all. So for example, they can say that you probably have images of trees, or images of buildings, but without much other data & very low accuracy. I'd still be a lot more concerned about them simply flaggi…

It starts at a broad understanding and ends with people permanently giving up their right and ability to keep rogue corporate governments in check.

Re: Apple defeats liability for not scanning iCloud for CSAM

#135
post #57

Earlier quoted context omitted.

Apple is very much like WhatsApp. Yes you cant perfectly trust their E2EE against state actors, but both in fact put some effort into making world have little bit more privacy. At least on Desktop we have usable Linux, but on the phones there is literally nothing usable because thanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months. Yes its possible to make And…

> Apple is very much like WhatsApp. Yes you cant perfectly trust their E2EE against state actors, but both in fact put some effort into making world have little bit more privacy. They probably use E2EE just so they don't have to respond to court orders and such.

Like when Google got tired of handling geofencing warrants.

Re: Apple defeats liability for not scanning iCloud for CSAM

#137
post #64

Earlier quoted context omitted.

> most people don't need cloud backups What world do you live in?

The world where the operating system on my phone (GrapheneOS) isn't conspiring against me or uploading my files to someone else's computer.

Is your phone also made of indestructible materials so it can't break, and magnetically linked to your person so it can't be stolen? Backups exist so you don't lose precious photos/data if you lose the device. Most People™ do not have the technical knowledge to set up a self-owned backup system, and/or will not realize how badly their future selves will wish they had backups if the setup friction for a self-owned solution is too high to conveniently do it right this second.

Re: Apple defeats liability for not scanning iCloud for CSAM

#138
post #123

Earlier quoted context omitted.

I still also totally don't get their policy. Trying to avoid false positives by not firing until a threshold was hit (was it 20 images?) seemed insane from a PR position... rightly or wrongly, all it would take would be the wrong court case and you can see the headlines: "Apple says users can have up to 20 CSAM images on their phone before they'll tell police"

Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM? I think the program would have to say use but morally I'd say no. The issue with client scanning is it has to assume the worst, or they are than liable. If its the person has 20+ different baptism of nude babys well huh that actually might be CSAM because the context of how that concentrated photos implies but even than its hard what if t…

> Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM?

In some countries it is as far as I’m aware

Re: Apple defeats liability for not scanning iCloud for CSAM

#139
post #36

Earlier quoted context omitted.

This is based on a faulty understanding of the underlying systems. The risk with this sort of E2E encryption is not that the service provider pinky promises not to decrypt what they have, it's that they promise they will not insert a new key into your circle of trust to subsequently start decrypting things .

I think you've imagined this faulty understanding. There are many mechanisms by which Apple could actually decrypt the data despite pinky promises not to. You listed one. There are others.

You're suggesting they purposely put a backdoor into all their custom methods? Why?

From a liability standpoint that implies a security breach could result in massive loss of customer data and if it did occur would destroy their privacy image to their customers.

I agree with the point that what you actually trust is the company to not insert maliscous code or keys into your protected path but modern systems actually contain ways to truly lock out the company itself from seeing your data.

Security wise it's amazing. If a company's admin cannot take your data it's excedingly hard for a hacker to do so.

Re: Apple defeats liability for not scanning iCloud for CSAM

#140
post #123

Earlier quoted context omitted.

Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM? I think the program would have to say use but morally I'd say no. The issue with client scanning is it has to assume the worst, or they are than liable. If its the person has 20+ different baptism of nude babys well huh that actually might be CSAM because the context of how that concentrated photos implies but even than its hard what if t…

> America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures The US has the largest pornography industry in the world by a massive margin, and the largest consumption of online pornography per capita Meanwhile should we be surprised that CSAM production is higher in countries like the Philippines that have very weak digital policing, abject poverty, high numb…

The US is pretty extreme, you have at the same time easy access to all the pornography you want (unless you’re in states that require age verification), and also a very prude culture. It’s not the only place with such contradictions though
Post reply on HN