An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology • Hacker News Board

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#461

Post by _rdvw » Fri, Aug 06, 2021, 4:59 PM UTC

fragileone wrote: post #354

chopin wrote: post #273

Earlier quoted context omitted.

> Phone: Pixel with GrapheneOS or CalyxOS I've seen this recommendation very often lately. As I am shopping for a new phone: Why is it that hardware directly from Google is recommended for putting another OS onto it (I've seen recommendations for LineageOS as well). What makes it better than any stock phone supported by LineageOS?

See the GrapheneOS or CalyxOS websites for more details, they are significantly hardened for security compared to LineageOS. Currently those two projects only support Pixels, mainly because they're all bootloader unlockable. If these projects had as many volunteers as LOS then more devices could be officially supported. LOS on a supported Android phone is still a better option than a stock Android or iPhone at least.

My https://divestos.org project, while not as secure as GrapheneOS, provides lots of security to many older devices.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#462

Post by randcraw » Fri, Aug 06, 2021, 5:00 PM UTC

toxik wrote: post #9

donkeyd wrote: post #4

I recently listened to a Darknet Diaries episode on messaging app Kik. This app is apparently being used by many people to trade child pornography. In this episode, there was some criticism expressed on how Kik doesn't scan all the images on their platform for child pornography. I would really like to hear from people who sign this open letter, how they think about this. Should the internet be a free for all place wi…

I think there is a very deep and troublesome philosophical issue here. Ceci n'est pas une pipe. A picture of child abuse /is not/ child abuse. Let me ask you a counter-question. If I am able to draw child pornography so realistically that you couldn't easily tell, am I committing a crime by drawing?

Or if you make a film about child abuse in which a child actor pretends to be abused, can you arrest the actor who abuses? If any depiction of the act is a crime, then you can.

This issue came before the US Supreme Court about a decade ago and they ruled that the depiction of a crime is not a crime so long as the depiction can in any way be called "art". In effect, any synthetic depiction of a crime is permitted.

However that ruling predated the rise of deep fakes. Would the SC reverse that decision now that fakes are essentially indistinguishable from the real thing? Frankly I think the current SC would flip since it's 67% conservative and has shown a willingness to reconsider limits on the first Amendment (esp. speech and religion).

But how would we re-draw the line between art and crime? Will all depictions of nudes have to be reassessed for whether the subject even might be perceived as underage? What about films like "Pan's Labyrinth" in which a child is tortured and murdered off-screen?

Do we really want to go there? This enters the realm of thought-crime, since the infraction was solely virtual and no one in the real world was harmed. If we choose this path, the freedom to share ideas will be changed forever.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#463

Post by zepto » Fri, Aug 06, 2021, 5:00 PM UTC

MikeUt wrote: post #429

zepto wrote: post #416

Earlier quoted context omitted.

As far as I can see: 1. This is a serious attempt to build a privacy preserving solution to child pornography. 2. The complaints are all slippery slope arguments that governments will force Apple to abuse the mechanism. These are clearly real concerns and even Tim Cook admits that if you build a back door, bad people will use it. However: Child pornography and the related abuse is widely thought of as a massive probl…

> So: where are the proposals for a better solution? Better policing and child protective services to catch child abuse at the root, instead of panicking about the digital files it produces? If you'd been paying attention, you'd have noticed that real-world surveillance has massively increased, which should enable the police to catch predators more easily. Why count only privacy-betraying technology as a "solution",…

How do you police people reaching out to children via messaging with sexual content?

> Why count only privacy-betraying technology as a "solution", while ignoring the rise of police and surveillance capabilities?

I don’t. But if your solution is ‘just police more’, you need to explain how the police should detect people who are grooming children by sending images to them.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#464

Post by defaultname » Fri, Aug 06, 2021, 5:01 PM UTC

rdedev wrote: post #435

defaultname wrote: post #418

Earlier quoted context omitted.

"Do keep in mind that at least one govt has successfully pressured apple to give up on its privacy" No company can defend you from your government. "All it would take"... That is the slippery slope. If a government is going to say "that's a nice looking hashing system you have there, now we need you to..." they could as easily -- more easily -- have said "that's a nice filesystem you have there, we need you to...". H…

Sure no company can completely defend me from my govt but atleast they can not build tools that make it easier. Also while it could be easy for a college graduate to build such a system, only Apple has the capability of rolling out this system to all of their phones. Otherwise we would have already seen such a system implemented in other countries

"only Apple has the capability of rolling out this system"

Right. Exactly. Any country in the world can mandate that Apple do anything they want (any of the slippery slope mandates), and Apple can comply or withdraw from the market. If any country wanted to demand that Apple compare all files against a ban list, they could have done that in 2007, or any year since. There is zero technical barrier and it would be a trivial task.

The point is that this development moves the bar infinitesimally. I would argue not at all. Fearmongering that depends upon "Well what if..." didn't actually think it through very well.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#465

Post by bisRepetita » Fri, Aug 06, 2021, 5:01 PM UTC

zepto wrote: post #416

sathackr wrote: post #42

US Government: We suspect the person in this photo of committing a crime. Here is your subpoena, Apple. You are directed to scan all iPhone and iCloud storage for any pictures matching this NeuralHash and report to us where you find them. Chinese Government: Here is the NeuralHash for Tienanmen square. Delete all photos you find matching this or we will bar you from China. Apple has at this point already admitted thi…

As far as I can see: 1. This is a serious attempt to build a privacy preserving solution to child pornography. 2. The complaints are all slippery slope arguments that governments will force Apple to abuse the mechanism. These are clearly real concerns and even Tim Cook admits that if you build a back door, bad people will use it. However: Child pornography and the related abuse is widely thought of as a massive probl…

>Child pornography and the related abuse is a massive problem

>proposals for better solution

How do you define "massive"? What makes you think that current approach is not working well enough?

Of course, it's really bad as soon as the very first case.

But we need to go beyond the sensationalist articles which talk about millions of pictures outthere. Even though this is, yes, a problem.

In the US, it looks like the number of child pornography cases is going down each year, from 1,593 in 2016 to 1,023 cases in 2020:

https://www.ussc.gov/sites/default/files/pdf/research-and-pu...

So yes, there is a problem, but I would love some to see some justification that the problem is actually as massive as people imply, or getting worse. Or if they have a political goal in mind.

And it feels to me like we talk about it a lot more, there is less taboo or social protections for perpetrators about it, which must help to fight this more efficiently than in the past.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#466

Post by fragileone » Fri, Aug 06, 2021, 5:02 PM UTC

Engineering-MD wrote: post #380

fragileone wrote: post #172

Earlier quoted context omitted.

Calyx is a degoogled Android ROM. It's probably the best choice until mobile Linux improves.

What are the benefits over graphene OS?

microG support mainly, it's needed for some Play Services APIs like push notifications and maps, though the choice all depends on what apps you use. GrapheneOS is great also and even better for those with very high security and privacy requirements.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#467

Post by magicloop » Fri, Aug 06, 2021, 5:03 PM UTC

echelon wrote: post #390

websites2023 wrote: post #381

Earlier quoted context omitted.

>It's fine to be worried about the second one, but it's wrong to conflate the two. Agreed, and just to be clear, I'm worried about that too. It just appears that we (myself and the objectors) have different lines. If Apple were to scan devices in the US and prevent them from sharing memes over iMessage, that would cross a line for me and I'd jump ship. But preventing CSAM stuff from getting on their servers seems fin…

> "preventing CSAM stuff from getting on their servers seems fine to me" You're either naive or holding your fingers in your ears if you think this is the objective. Let me repeat this again: this is a tool for the CCP, FBI, intelligence, and regimes.

I think the situation is clear when we think of this development from a threat modelling perspective.

Consider a back-door (subdivided into code-backdoors and data-backdoors) placed either on-device or on-cloud. (4 possibilities)

Scanning for CP is available to Apple on-cloud (in most countries). Scanning for CP is available to the other countries on-cloud (e.g. China users have iCloud run by a Chinese on shore provider). Scanning for CP is not available to Apple on-device (until now)

This is where the threat model comes in. Intelligence agencies would like a back door (ideally both Code and Data).

This development creates an on-device data-backdoor because scanning for CP is done via a neural network algorithm plus the use of a database of hashes supplied by a third party.

If the intelligence service poisons the hashes database then it won't work because the neural network scans for human flesh and things like that, not other kinds of content. So the attack works for other sexual content but not political memes. It is scope-limited back door.

For it to be a general back door, the intelligence agency would need the neural network (part of apple's on-device code) and well as the hashes database to be modified. So that is both requiring a new code back door (Apple has resisted this), and a data back door both on-device.

Currently Apple has resisted:

Code back doors (on device) Data back doors on device (until now)

and Apple has allowed Data back doors in cloud (in certain countries) Code back doors in cloud (in certain countries)

In reality the option to not place your photos in iCloud is a euphemism for "don't allow any data backdoor". That is because iCloud is a data-backdoor due to it being able to be scanned (either by Apple or an on-shore data provider).

My analysis is that the on-device scanning does not improve Apple's ability to identify CP since it does so on iCloud anyway. But if my analysis is incorrect, I'd be genuinely interested if anyone can correct me on this point.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#468

Post by RegnisGnaw » Fri, Aug 06, 2021, 5:05 PM UTC

rantwasp wrote: post #444

RegnisGnaw wrote: post #311

Note that: 1) its only scanned on upload to iCloud, so if you don't upload then its not scanned 2) (per another article, https://techcrunch.com/2021/08/05/apple-icloud-photos-scanni ...): Most cloud services — Dropbox, Google, and Microsoft to name a few — already scan user files for content that might violate their terms of service or be potentially illegal, like CSAM. So you really can't opt out unless you avoid al…

yes it’s only icloud photos for now. what’s going to be next? are you going to have a huge scandal every time they turn on a new “feature”?

Its interesting that its only brought up when Apple does it. Google, MS, Dropbox, etc gets a pass..

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#469

Post by minton » Fri, Aug 06, 2021, 5:05 PM UTC

SirensOfTitan wrote: post #105

I know the privacy approach Apple has been pushing was just marketing, but I didn't care too much because I enjoyed my iPhone and M1 macbook. Because of this decision (and the fact that my iPhone more-or-less facilitates poor habits throughout my life), I'm considering moving completely out of the Apple ecosystem. Does anyone have any recommendations for replacements? * On laptops: I have an X1 carbon extreme running…

I understand the impulse to turn away from Apple. However, it’s not a practical solution. How long until this tech or worse is mandated in all products? I think the real answer is for the people to stop bickering about primary colors and work together toward passing legislation that limits the pervasive invasion of privacy by governments and corporations.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#470

Post by zepto » Fri, Aug 06, 2021, 5:06 PM UTC

bisRepetita wrote: post #465

zepto wrote: post #416

Earlier quoted context omitted.

As far as I can see: 1. This is a serious attempt to build a privacy preserving solution to child pornography. 2. The complaints are all slippery slope arguments that governments will force Apple to abuse the mechanism. These are clearly real concerns and even Tim Cook admits that if you build a back door, bad people will use it. However: Child pornography and the related abuse is widely thought of as a massive probl…

>Child pornography and the related abuse is a massive problem >proposals for better solution How do you define "massive"? What makes you think that current approach is not working well enough? Of course, it's really bad as soon as the very first case. But we need to go beyond the sensationalist articles which talk about millions of pictures outthere. Even though this is, yes, a problem. In the US, it looks like the n…

> child pornography cases is going down each year, from 1,593 in 2016 to 1,023 cases in 2020

Sam Harris (and the FBI) would likely say this is because detection is getting harder thanks to encryption.

The only authority on this is actually the FBI, but I presume you wouldn’t trust them.

That distrust is reasonable, but irrelevant. What matters is that many people will trust them and see the fear as legitimate.

Arguing over numbers with me is irrelevant. I’m saying both positions are reasonable fears.

You cannot win by denying that.

One winning move would be to take the problem seriously and work out a better technical solution. There may be others.