An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology • Hacker News Board

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#571

Post by zibzab » Fri, Aug 06, 2021, 7:11 PM UTC

snowwrestler wrote: post #404

zibzab wrote: post #46

Earlier quoted context omitted.

Are you sure? Here is a true story for you: Company X claimed users voice commands never left their devices. Then someone leaked recordings of people having sex, commiting crimes, etc recorded from X devices. This was brought up by media. For every concerned user there were ten apologist trying to justify this behaviour and a week later everyone forgot this ever happened.

If you’re talking about Amazon, those stories did affect Echo sales, and the trajectory of the category overall. When was the last time you read a breathless article about how smart speakers will change everything?

I was thinking of another company, but I am very happy that Amazon bottom line was affected (?)

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#572

Post by mirkules » Fri, Aug 06, 2021, 7:11 PM UTC

vineyardmike wrote: post #490

snowwrestler wrote: post #396

Earlier quoted context omitted.

I get this sentiment but the known-bad-image-scanning technology is not new. That’s not what Apple announced. Many tech services already do that, which already enables the slippery slope you’re illustrating here. I’m not trying to minimize the danger of that slope. But as someone who is interested in the particulars of what Apple announced specifically, it is getting tiresome to wade through all the comments from peo…

But now its on your device. "Its just when you upload to icloud or recieve an iMessage" they say. BUT the software's on your device forever. What about next year? What about after an authoritarian goverment approaches them? By 2023 it might be searching non-uploaded photos. You can always not use cloud tech like PhotoDNA but you can't not use software BUILT IN to your device. Especially when its not transparent.

It’s also not inconceivable to access the camera live and perform recognition live on feeds. It’s not even that expensive to do that anymore.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#573

Post by davidham » Fri, Aug 06, 2021, 7:13 PM UTC

sathackr wrote: post #42

US Government: We suspect the person in this photo of committing a crime. Here is your subpoena, Apple. You are directed to scan all iPhone and iCloud storage for any pictures matching this NeuralHash and report to us where you find them. Chinese Government: Here is the NeuralHash for Tienanmen square. Delete all photos you find matching this or we will bar you from China. Apple has at this point already admitted thi…

Even if you don’t use iOS, seems likely Google could do/is doing this also.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#574

Post by danShumway » Fri, Aug 06, 2021, 7:14 PM UTC

zepto wrote: post #565

danShumway wrote: post #552

Earlier quoted context omitted.

> ‘Think of the children’ works for a reason. "Think of the children" will always work, no matter what the context is, no matter what the stats are, and no matter what we do. That does not mean that we should not care about the children, and it does not mean that we shouldn't care about blocking CSAM. We should care about these issues purely because we care about protecting children. If there are ways for us to reduc…

> So in the real world we know that the majority of child abuse comes from people that children already know. Historically this has been regarded as true, but according to the FBI, online predation is a growing problem. https://www.fbi.gov/news/stories/child-predators > Your mistake here is assuming that anything that technologists can build will ever change those people's minds or make them ease up on calls to ban e…

> Arguing that concerns about children are overblown or being exploited for nefarious means may be ‘true’, but it does nothing to provide an alternative that Apple could use

- If the stats don't justify their fears

- And I come up with a technological solution that will make the stats even lower

- Their fears will not be reduced

- Because their fears are not based on the stats

----

> Apple’s customers do include many people who are worried about sexual predators reaching their children via their phones though

Are they worried about this because of a rational fear based on real-world data? If so, then I want to talk to them about that data and I want to see what basis their fears have. I'm totally willing to try and come up with solutions that reduce the real world problem as long as we're all considering the benefits and tradeoffs of each approach. We definitely should try to reduce the problem of CSAM even further.

But if they're not basing their fear on data, then I can't help them using technology and I can't have that conversation with them, because their fear isn't based on the real world: it's based on either their cultural upbringing, or their preconceptions about technology, or what media they consume, or their past traumas, or whatever phobias that might be causing that fear.

Their fear is real, but it can not be solved by any technological invention or policy change, including Apple's current system. Because you're telling me that they're scared regardless of what the reality of the situation is, you're telling me they're scared regardless of what the stats are.

That problem can't be solved with technology, it can only be solved with education, or emotional support, or cultural norms. If they're scared right now without knowing anything about how bad the problem actually is, then attacking the problem itself will do nothing to help them -- because that's not the source of their fear.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#575

Post by ZekeSulastin » Fri, Aug 06, 2021, 7:15 PM UTC

todd3834 wrote: post #160

I appreciate the privacy concerns, I really do. But please, can we suggest an alternative that still benefits children the way that this will? Imagine being Tim Cook and learning that technology that you control is being used for things that everyone agrees is horrific to children. Would you be able to turn a blind eye in matter of principle? I’m struggling with my opinion on this one because I care so much about the…

That’s why this sort of thing is first used “for the children” (or “to prevent terrorism” back in 2001 or 2015[0]) - because when you argue in favor of privacy you then also implied to support whatever is being used as the impetus.

The alternative is to not start scanning your device in the first place and getting the foot in the door, because once it’s open it’s really hard to close.

[0] https://en.wikipedia.org/wiki/FBI–Apple_encryption_dispute

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#576

Post by VBprogrammer » Fri, Aug 06, 2021, 7:18 PM UTC

Seems to me that a 1x1 pixel image tag pointing at an image matching one of the signatures would be enough to trigger some kind of action, even if the end user doesn't see that single pixel. What then, does your phone get seized automatically?

Of course the civil disobedience way of dealing with this would be to find an entirely safe image which matches the hash so that every website can safely embed such an image and show the utter futility of the idea.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#577

Post by webmobdev » Fri, Aug 06, 2021, 7:19 PM UTC

zepto wrote: post #416

sathackr wrote: post #42

US Government: We suspect the person in this photo of committing a crime. Here is your subpoena, Apple. You are directed to scan all iPhone and iCloud storage for any pictures matching this NeuralHash and report to us where you find them. Chinese Government: Here is the NeuralHash for Tienanmen square. Delete all photos you find matching this or we will bar you from China. Apple has at this point already admitted thi…

As far as I can see: 1. This is a serious attempt to build a privacy preserving solution to child pornography. 2. The complaints are all slippery slope arguments that governments will force Apple to abuse the mechanism. These are clearly real concerns and even Tim Cook admits that if you build a back door, bad people will use it. However: Child pornography and the related abuse is widely thought of as a massive probl…

The intention to prevent child sexual abuse material is indeed laudable. But the "solution" is not. Especially when we all know that this "solution" gives a legal backing to corporates to do even more data mining on its users, and can easily be extended (to scan even more "illegal" content) and abused into a pervasive surveillance network desired by a lot of government around the world.

For those wondering what's wrong with this, two hard-earned rights in a democracy go for a toss here:

    1. The law presumes we are all innocent until proven guilty.
    2. We have the right against self-incrimination.

Pervasive surveillance like this starts with the presumption that we are all guilty of something ("if you are innocent, why are you scared of such surveillance?" or "what do you have to hide if you are innocent?"). The right against self-incrimination is linked to the first doctrine because compelling an accused to testify transfers the burden of proving innocence to the accused, instead of requiring the government to prove his guilt.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#578

Post by osmarks » Fri, Aug 06, 2021, 7:24 PM UTC

mirkules wrote: post #572

vineyardmike wrote: post #490

Earlier quoted context omitted.

But now its on your device. "Its just when you upload to icloud or recieve an iMessage" they say. BUT the software's on your device forever. What about next year? What about after an authoritarian goverment approaches them? By 2023 it might be searching non-uploaded photos. You can always not use cloud tech like PhotoDNA but you can't not use software BUILT IN to your device. Especially when its not transparent.

It’s also not inconceivable to access the camera live and perform recognition live on feeds. It’s not even that expensive to do that anymore.

It would probably kill the battery to do that constantly, as the processor would not be able to sleep ever. Although relying on energy efficiency to not improve is not a good long term situation.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#579

Post by zepto » Fri, Aug 06, 2021, 7:28 PM UTC

babesh wrote: post #569

zepto wrote: post #416

Earlier quoted context omitted.

As far as I can see: 1. This is a serious attempt to build a privacy preserving solution to child pornography. 2. The complaints are all slippery slope arguments that governments will force Apple to abuse the mechanism. These are clearly real concerns and even Tim Cook admits that if you build a back door, bad people will use it. However: Child pornography and the related abuse is widely thought of as a massive probl…

There is no end to the loss of privacy in the name of safety if your bogeyman is increasingly sophisticated actors. The more sophisticated child molesters out there will find out about what Apple is doing and quickly avoid it. Isn’t that what the FBI is complaining about, that they have grown increasingly sophisticated? The more sophisticated privacy adherents will also avoid Apple and resort to end to encryption and…

> The more sophisticated child molesters out there will find out about what Apple is doing and quickly avoid it.

I think this is exactly what Apple wants to he the result of their their iMessage scanning.

They are not in this to make arrests. They just want parents to feel safe letting children use their products. Driving predators to use different tools is fine with them.

As far as the FBI goes, this is presumably not their preference, but it’s still good for them if it makes predation a little harder.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#580

Post by FabHK » Fri, Aug 06, 2021, 7:29 PM UTC

brasscodemonkey wrote: post #227

FabHK wrote: post #194

Earlier quoted context omitted.

> direct access to everyone's data and messages. From what I understand, only to Chinese customers 's data and messages (bad enough, sure, but not as bad as you say).

Depends on the company. If it’s a Chinese company like TikTok, data is stored in China and therefore property of the state. Things are about to get worse and they crack down on private companies in China. And it’s not just Americans who worry about this. When word got out that Line was storing user data in China servers, it blew up in the news in Japan and Taiwan and went into immediate investigation. Line ended up m…

> Depends on the company

I was referring specifically to Apple & iCloud (and I thought GP was as well).