Live data from Hacker News

An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

appleprivacyletter.com

641–650 of 713 posts

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#641
post #617

Earlier quoted context omitted.

> This is essential to science. Okay, fine. Are you claiming that people who are calling to ban encryption are doing so on a scientific basis? Come on, be serious here. People call to ban encryption because it scares them, not because they have a model of the world based on real data or real science that they're using to reinforce that belief. If they did, we could argue with them. But we can't, because they don't. >…

> This is essential to science. >> Okay, fine. Are you claiming that people who are calling to ban encryption are doing so on a scientific basis? No. Did I say something to that effect? > Come on, be serious here. People call to ban encryption because it scares them, not because they have a model of the world based on real data or real science that they're using to reinforce that belief. You say this as if you are ar…

> Neither of us disagree that such people exist. Indeed we both believe that they do.

> Why does changing such a person’s mind matter?

Okay, finally! I think I understand why we're disagreeing. Please tell me if I'm misunderstanding your views below.

> You’ll need to explain what the contradiction is.

I kept getting confused because you would agree with me right up to your conclusion, and then suddenly we'd both go in opposite directions. But here's why I think that's happening:

You agree with me that there are irrational actors that will not be convinced by any kind of reason or debate that their fears are irrational. You agree with me that those people will never stop calling to ban encryption, and that they will not be satisfied by any alternative you or I propose. But you also believe there's another category of people who are "semi-rational" about child abuse. They're scared of it, maybe not for any rational reason. But they would be willing to compromise, they would be willing to accept a "solution" that targeted some of their fears, and they might be convinced than an alternative to banning encryption is better.

Where we disagree is that I don't believe those people exist -- or at least if they do exist, I don't believe they are a large enough or engaged enough demographic to have any political clout, and I don't think it's worth trying to court them.

My belief is that by definition, a fear that is not based on any kind of rational basis is an irrational fear. I don't believe there is a separate category of people who are irrationally scared of child predators, but fully willing to listen to alternative solutions instead of banning encryption.

So when you and I both say that we can't convince the irrational people with alternative solutions, my immediate thought is, "okay, so the alternative solutions are useless." But of course you think the alternative solutions are a good idea, because you think those people will listen to your alternatives, and you think they'll sway the encryption debate if they're given an alternative. I don't believe those people exist, so the idea of trying to sway the encryption debate by appealing to them is nonsense to me.

In my mind, anyone who is rational enough to listen to your arguments about why an alternative to breaking encryption is a good idea, is also rational enough to just be taught why banning encryption is bad. So for people who are on the fence or uninformed, but who are not fundamentally irrationally afraid of encryption, I would much rather try gently reaching out to them using education and traditional advocacy techniques.

----

Maybe you're right and I'm wrong, and maybe there is a political group of "semi-rational" people who are

A) scared about child abuse

B) unwilling to be educated about child abuse or to back up their beliefs

C) but willing to consider alternatives to breaking encryption and compromising devices.

If that group does exist, then yeah, I get where you're coming from. BUT personally, I believe the history of encryption/privacy/freedom debates on the Internet backs up my view.

Let's start with SESTA/FOSTA:

First, Backpage did work with the FBI, to the point that the FBI even commented that Backpage was going beyond any legal requirement to try and help identify child traffickers and victims. Second, both sex worker advocates and sex workers themselves openly argued that not only would SESTA/FOSTA be problematic for freedom on the Internet, but that the bills would also make trafficking worse and make their jobs even more dangerous.

Did Backpage's 'compromise' sway anyone? Was there a group of semi-reasonable people who opposed sites like Backpage but were willing to listen to arguments that the bills would actively make sex trafficking worse? No, those people never showed up. The bills passed with broad bipartisan support. Later, several Senators called to reexamine the bills not because alternatives were proposed to them, but because they put in the work to educate themselves about the stats, and realized the bills were harmful.

Okay, now let's look at the San Bernardino case with Apple. Apple gave the FBI access to the suspect's iCloud account, literally everything they asked for except access to decrypt the phone itself. Advocates argued that the phone was unlikely to aid in the investigation, and also suggested using an exploit to get into the phone, rather than requiring Apple to break encryption. Note that in this case the alternative solution worked, the FBI was able to get into the phone using an exploit rather than by compelling Apple to break encryption. The best case scenario.

Did any of that help? Was there a group of semi-reasonable people who were willing to listen to the alternative solution? Did the debate cool because of it? No, it changed nothing about the FBI's demands or about the political debate. What did help was Apple very publicly and forcefully telling the FBI that any demand at all to force them to install any code for any reason would be a violation of the 1st Amendment. So minus another point from compromise as an effective political strategy in encryption debates, and plus one point to obstinance.

Okay, now let's jump back to early debates about encryption: the clipper chip. Was that solved by presenting the government and concerned citizens with an alternative that would better solve the problem? No, it wasn't -- even though there were plenty of people who argued at the time for encryption experts to work with the government instead of against it. Instead the clipper chip problem was solved both when encryption experts broke the clipper chip so publicly and thoroughly that it destroyed any credibility the government had in claiming it was secure, and it was solved by the wide dissemination of strong encryption techniques that made the government's demands impossible, over the objections of people who called for compromise or understanding of the government's position.

----

I do not see any strong evidence for a group of people who can't be educated about encryption/abuse, but who can be convinced to support alternative strategies to reduce child abuse. If that group does exist, it does a very good job of hiding, and a very bad job of intervening during policy debates.

I do think that people exist who are skeptical about encryption but who are not so irrational that they would fall into our category of "impossible to convince." However, I believe they can be educated, and that it is better to try and educate them than it is to reinforce their fears.

Because of that, I see no political value in trying to come up with alternative solutions to assuage people's fears. I think those people should either be educated, or ignored.

It is possible I'm wrong, and maybe you could come up with an alternative solution that reduced CSAM without violating human rights to privacy and communication. If so, I would happily support it, I have no reason to oppose a solution that reduces CSAM if it doesn't have negative effects for the Internet and free culture overall, a solution like that would be great. However, I very much doubt that you can come up with a solution like that, and if you can, I very much doubt that outside of technical communities anyone will be very interested in what you propose. I personally think you would be very disappointed by how few people arguing for weakening encryption right now are actually interested in any of the alternative solutions you can come up with.

And it's my opinion, based on the history of privacy/encryption, that traditional advocacy and education techniques will be more politically effective than what you propose.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#642

Earlier quoted context omitted.

As someone else mentioned, for a NAS using TrueNAS (used to be FreeNAS) is easy enough and quite satisfying. You can find plenty of guides. In general, you need to balance budget, capacity requirements, and form factor. Old servers are often great. Big disks seem like a good idea, but for rebuild times going over 4TB is horrible. However, unfortunately HDD prices right now are horrible...

define horrible? i have a 8x18tb raid-6 array and it rebuilds in 2-3 days.

I guess it depends. Personally I don't like having rebuilds longer than a day.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#643

There has to be some incentive somewhere for Apple to do this. They know it's wrong, they know it will be abused. Tim Cook himself, if he wasn't rich and powerful, would be executed in a number of countries that Apple operates in for his sexual/romantic identity. Apple also removes LGBT based applications in countries where they're illigal, to continue doing business. This demonstrates that Apple complies with the de…

I'm not sure such a big conspiracy is needed. After all, the reason "nothing to hide" memes are so common is that a lot of people believe them enthusiastically. It seems entirely plausible to me that a core team of passionate crusaders could have driven this project to completion by just making it too awkward for anyone to object. For a sample of outside perspectives ( https://www.npr.org/2021/08/06/1025402725/apple-…

This is possible, but the decision is coming from higher up. I can certainly see apple engineers thinking this would really work, by focusing entirely upon the singular problem and getting a chance to use fancy tech. But at the end of the day Timmy is the head cook, and decisions like this go against apples "mission", so, there's incentive higher than just the engineers.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#644
post #604

Earlier quoted context omitted.

If you had the option of buying this hardware stack through an integrated third-party brand/storefront that offered support for the products and their integration would that make you feel differently?

That's an interesting idea. The amount of work even for a techie to maintain all this, is considerable. Could it be set up for a "user". Tech support, would be; Interesting.

I made an example storefront a few months ago and jotted down a plan in a bout of procrastination. I have a few thoughts on a pragmatic lazy (in the cs sense) approach.

If you or anyone one else is interested email me: ipodopt@pm.me

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#645
post #625

Earlier quoted context omitted.

They are options that almost nobody can or will use, so they won’t have any impact. If your goal is to inform a small minority of expert users that they should protect themselves against corporate/government encroachment, by the looks of the comments here, I’d say you’ve already succeeded.

I think you're missing my point. You do your best to inform the majority not because the majority will enact change based on it, but so that the minute and disparate slices of the population for whom that information is relevant but might not otherwise have been exposed to it can access it and perform or investigate whatever actions they deem necessary and economical to mediate the potential threat. This forum is too…

I notice you edited your top level comment, and it seems like several others to change the meaning of our thread.

I’m not missing your point if you are changing your point after I have replied.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#646
post #639
post #626

Earlier quoted context omitted.

Can you describe the middle ground that I have ignored? What disingenuous argument have I made? The only thing I have argued for is for hackers to attempt technical solutions that are more to their liking than Apple’s, because arguments are not preventing the slide.

I am saying that you are promoting a slippery ground falsely as middle ground. Basically the argument I hear from you is “If you build a back door, then people will use it. So let’s build it anyway because it is a middle ground.” The problem I have with it is the “let’s build it anyway”. That seems as clear as day. Why do I have to keep repeating myself? Don’t be an apologist.

[deleted]

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#647
post #639
post #626

Earlier quoted context omitted.

Can you describe the middle ground that I have ignored? What disingenuous argument have I made? The only thing I have argued for is for hackers to attempt technical solutions that are more to their liking than Apple’s, because arguments are not preventing the slide.

I am saying that you are promoting a slippery ground falsely as middle ground. Basically the argument I hear from you is “If you build a back door, then people will use it. So let’s build it anyway because it is a middle ground.” The problem I have with it is the “let’s build it anyway”. That seems as clear as day. Why do I have to keep repeating myself? Don’t be an apologist.

> If you build a back door, then people will use it. So let’s build it anyway because it is a middle ground.

This looks a completely made up position that has nothing to do with anything I have said. If you can find a comment where I am advocating building back doors, I invite you to quote it.

> That seems as clear as day. Why do I have to keep repeating myself?

If it was clear you’d be able to support it with a quote. I’m pretty sure you can’t.

> Don’t be an apologist.

It doesn’t seem like you have been following my argument, so it’s unclear why you’d stoop to a personal attack.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#648
post #625

Earlier quoted context omitted.

They are options that almost nobody can or will use, so they won’t have any impact. If your goal is to inform a small minority of expert users that they should protect themselves against corporate/government encroachment, by the looks of the comments here, I’d say you’ve already succeeded.

I think you're missing my point. You do your best to inform the majority not because the majority will enact change based on it, but so that the minute and disparate slices of the population for whom that information is relevant but might not otherwise have been exposed to it can access it and perform or investigate whatever actions they deem necessary and economical to mediate the potential threat. This forum is too…

Zepto, the Thread won't let me respond directly to yoy, but I haven't touch led any of my comments since they first received a reply. I often touch my comments within a couple minutes of posting in order to correct typos or clarify my intentions, but don't do so without remark once they've become canonical in a thread.

Your replies to my threads are as they were written.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#649
post #194
post #158

Earlier quoted context omitted.

The Chinese government already has direct access to everyone's data and messages. There is no encryption. This is mandated. [0]: https://www.nytimes.com/2021/05/17/technology/apple-china-ce...

> direct access to everyone's data and messages. From what I understand, only to Chinese customers 's data and messages (bad enough, sure, but not as bad as you say).

That is what I meant. The point is that everyone talking about how this policy introduces a backdoor which can be exploited by totalitarian states are wrong.

There's no need. Apple willfully obliges local laws.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#650

Earlier quoted context omitted.

> ‘Think of the children’ works for a reason. "Think of the children" will always work, no matter what the context is, no matter what the stats are, and no matter what we do. That does not mean that we should not care about the children, and it does not mean that we shouldn't care about blocking CSAM. We should care about these issues purely because we care about protecting children. If there are ways for us to reduc…

> " Reducing the real-world occurrences for irrational fears doesn't make those fears go away. " " You're saying it yourself, these people aren't motivated by statistics about abuse, they're frightened of the idea of abuse " We could say the same thing the other way - people up in arms are not frightened by statistics of abuse of a surveillance system, but frightened of the idea of a company or government abusing it.…

> This thread is not full of statistics and data about existing content filtering and surveillance systems and how often they are actually being abused.

It is filled with explanations about why the systems you mention are tangibly different from what Apple is proposing. There is a huge difference between scanning content on-device and scanning content in a cloud. That doesn't mean that scanning content in the cloud can't be dangerous, but it is still tangibly different. There is also a huge difference between a user-inspectable list of malware being blocked in a website and an opaque list of content matches that users can not inspect or debate. There is also a huge difference between a user-inspectable static list and an AI system with questionable accuracy guarantees. And there is a huge difference between a user-controlled malware list that is blocked locally without informing anyone, and a required content list that sends notifications to other people/governments/companies when it is bypassed.

That being said, if you want to look at stats about how accurate AI filters are for explicit material in the examples you mention, there are a ton of stats online about that, and they're mostly all quite bad.

> nobody has any data or facts about how often systems do slide down slippery slopes, or get dragged back up them

There's a lot to unpack in this one sentence, and it would take more time than I'm willing to give, but are you really implying that government surveillance doesn't count as a real slippery slope because sometimes activists reverse the trend?

> Minors will see the prompt "if you do this, your parents will find out" and can choose not to and the parents don't find out. There's an example of the message in the Apple announcement[1]

You misunderstand the concern. The risk is not the child themselves clicking through to the photo (although it would be easy for them to accidentally do so), it's the risk of that data being leaked from other phones because a friend thoughtlessly clicks through the prompt.

> The open letter itself has multiple inaccurate descriptions of how the thing works by the second paragraph to present it as maximally-scary.

Where? Here's the second paragraph:

> Apple's proposed technology works by continuously monitoring photos saved or shared on the user's iPhone, iPad, or Mac. One system detects if a certain number of objectionable photos is detected in iCloud storage and alerts the authorities. Another notifies a child's parents if iMessage is used to send or receive photos that a machine learning algorithm considers to contain nudity.

The only thing I can think of is the word "continuously" which is not strictly inaccurate but could be misinterpreted as saying that the scanning will be constantly running on the same set of photos, and the subtle implication that this scanning will happen to photos "saved", which might be misinterpreted as implying that this scan will happen to photos that aren't uploaded to iCloud. But given that the second sentence immediately clarifies that this is referring to photos uploaded to iCloud, it seems like a bit of a stretch to me to call this misinformation.

> people are jumping straight to "horrible governments will be able to disappear critics" and ignoring that horrible governments already do that and have many much easier ways of doing that.

Hang on a sec. A little while ago you were calling my fears theoretical, now you're admitting that governments routinely abuse this kind of power. You really think it's unreasonable to be cautious about giving them more of this power?

> Does that change your opinion either way?

It does not change my opinion, but at least it's real data, so more of that in these debates please. I'm not denying or rejecting the numbers that the UK lists.

> A less panicky "Open Letter to Apple" might encourage them to make that data public

Holy crud, I would hope this is the bare minimum. Are we really having a debate over whether or not Apple will make that data public? I thought that was just assumed that they would. If that's up in the air right now, then we've sunk really low in the overall conversation about public accountability and human rights.

> which seems an important distinction that you're glossing over in your position "it won't reduce abuse so it shouldn't be built" when the builders are not claiming it will reduce abuse.

I realize this is branching out on in a different direction, but it sure as heck better reduce abuse or its not worth building. CSAM is disgusting, but the primary reason to target it is to reduce abuse. If reducing CSAM doesn't reduce abuse, it's not worth doing and we should focus our efforts elsewhere.

I know this is something that might sound abhorrent to people, but we are having this debate because we care about children. We have to center the debate on the reduction of the creation of CSAM, the reduction of child abuse, and the reduction of gateways into child abuse. Reducing child abuse is the point. We absolutely should demand evidence that these measures reduce child abuse, because reducing child abuse and reducing the creation of CSAM is a really stinking important thing to do.

Which leads back to your other note:

> does it make any difference if they involved people the child knew? If so, what difference do you think that makes?

Yes, it makes a massive difference, because knowing more about where child abusers are coming from and how they interact with their victims makes it easier to target them and will make our efforts more effective. We should care about this stuff.

> It's not the case that "think of the children" leads to political universal agreement of any system, as you're stating.

I think you misunderstand. Nobody who's willing to bring out "think of the children" as a debate killer has ever dropped the argument because they got a concession. That there are some entities (like the EFF) who are willing to reject the argument as a debate killer and look at it through a risk analysis lens does not mean the unquestioned argument of "one child is too many" is any less toxic to real substantive political debate.

> without prohibiting or weakening encryption and in full respect of privacy

I don't want to bash on the EU too hard here, but it has this habit of just kind of tacking onto the end of its laws "but make sure no unintended bad things happen" and then acting like that solves all of their problems. It doesn't mean anything when they add these clauses. This is the same EU that argued for copyright filters and then put on the end of their laws, "but also this shouldn't harm free expression or fair use."

It means very little to me that the EU says they care about privacy. What real, tangible measures did they include to make sure that in practice encryption would not be weakened?

Look, I can do the same thing. Apple should not implement this system, but they should also reduce CSAM. See, I just proved I care about both, exactly as convincingly as the EU! So you know I'm serious about CSAM now, I said that Apple should reduce it. But I predict that you'll "dismiss with something that amounts to 'I won't change my opinion when presented with this fact', yes?"

> There are real things to criticise about this system, the chilling effect of surveillance, the chance of slippery slope progression, the nature of proprietary systems, the chance of mistakes and bugs in code or human interception, the blurred line between "things you own" and "things you own which are closely tied to the manufacturer's storage and messaging systems"

Wait, hold on. Forget literally everything that we were talking about above. This is, like, 90% of what people are criticizing! What else are people criticizing? These are really big concerns! You got to the end of your comment, then suddenly listed out 6 extremely good reasons to oppose this system, and then finished by saying, "but other than those, what's the problem?"

Post reply on HN