Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

appleprivacyletter.com

Post reply on HN
Print view View original

611–620 of 713 posts

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#611

Post by aidenn0 »

donkeyd wrote: post #68
aidenn0 wrote: post #54

Earlier quoted context omitted.

I won't use a service that performs dragnet surveillance on my communication. The US Postal service does not open every single letter to check if it contains illegal material. If I rent a storage unit, the storage company does not open up my boxes to see if I have albums of child pornography. This move by Apple is equivalent. I'm going to turn this around and say that those in favor of Apple's move: "Should the inter…

> The US Postal service does not open every single letter to check if it contains illegal material. You'd be surprised about the amount of packages that gets x-rayed in order to find drugs. But yes, you're 100% right that it's not all of them.

I forgot about that. I think it's nearly all stamped mail over a certain weight? I don't remember if labeled mail is similarly scanned and my google skills are failing me today.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#612

Post by heavyset_go »

croes wrote: post #581
heavyset_go wrote: post #516

Earlier quoted context omitted.

> This only finds known pictures of child abuse not new ones No, it finds images that have similar perceptual hashes, typically using the hamming distance or another metric to calculate differences between hashes. I've built products that do similar things. False positives are abound with perceptual hashes, especially when you start fuzzy matching them.

But these are just variations of known pictures to recognize them if they cropped or scaled. The hash of a really new picture isn't similar to the hash of a known picture.

Perceptual hashes have collisions, and it is entirely possible for two pictures that have nothing to do with each other to have similar perceptual hashes. It's actually quite common.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#613

Post by zepto »

crocodiletears wrote: post #601
zepto wrote: post #568

Earlier quoted context omitted.

It doesn’t matter whether they are aware. They can’t take precautions. There are no technical solutions that people can use, and technologists seem to be uninterested in working on them. Apple’s solution is the best on offer. Raising awareness about Facebook’s problems hasn’t harmed Facebook.

They can take precautions by not using iOS devices. The goal isn't to harm Apple, it's to make the knowledge available to those who need it, and have the will to avoid it. That could mean anything from watching what they put on their iPhone, to putting them down the track of using degoogled android variants on select phones, to ditching cell phones all together depending on their needs. Knowing that your phone is wat…

Android has been doing this all along, so not using iOS devices will not help.

As I said, there are no good solutions.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#614

Post by belter »

defaultname wrote: post #557
belter wrote: post #528

Earlier quoted context omitted.

What would the two different systems be then? I am certainly willing to agree I conflated them if you clarify.

One system optionally checks iMessages (to be sent or received) against one or more "nudity" neural networks if the user is identified as a child in the iCloud family sharing group. If it triggered, the child is given information/opt out, optionally choosing to go ahead with sending or viewing, with the caveat that their parent(s) will be notified (optionally, I presume). Nothing about this event is sent off device.…

There are two systems and I will comment first on the second one. The second system you were referring to, as in the 'other', is the CSAM. And by the funcionality description, already sounds terrifying enough.

You will be flagged, reported to the authorities under technical argumentation the Algorithm has a "one in a trillion" chance of failure. Account blocked, and you start your "guilty until proven innocent process" from there.

Due to scale at what Apple works its also clear to see, if you think about it, that the additional human process will be on a random basis. The volume would be too high for a human chain to validate each flagged account.

In any case at multiple occasions, it is clear that the current model of privacy with Apple is that there is no privacy. It is a tripartite between you, the other person you interact with, and Apple algorithms and human reviewers. Is there any difference between this, and having a permanent video stream from what is happening inside each division in your house, analyzed by a "one in a trillion neural net algorithm", and additionally reviewed by a human on a need to do basis ?

CSAM

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

"Using another technology called threshold secret sharing, the system ensures that the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content. Only when the threshold is exceeded does the cryptographic technology allow Apple to interpret the contents of the safety vouchers associated with the matching CSAM images."

"The threshold is selected to provide an extremely low (1 in 1 trillion) probability of incorrectly flagging a given account. This is further mitigated by a manual review process wherein Apple reviews each report to confirm there is a match, disables the user’s account, and sends a report to NCMEC. If a user feels their account has been mistakenly flagged they can file an appeal to have their account reinstated."

So in other words, there is no doubt for this one there will be intervention, of Apple employees when required. For training purposes, for system testing, for law enforcement purposes etc...

I guess these and other functionality was the reason the decided not to encrypt the icloud backups

"Apple dropped plan for encrypting backups after FBI complained" https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...

Now concerning the second feature/system called: "Expanded Protections for Children"

"https://www.apple.com/child-safety/pdf/Expanded_Protections_..."

After reviewing what I believe to be every single document published so far by Apple on this, I found only one phrase and nothing more detailing how it works. Looking at the amount of detail published for CSAM, the lack info on this one already looks like a redflag to me. So I am not really sure how you can be so certain of the implementation details.

The phrase is only this: "Messages uses on-device machine learning to analyze image attachments and determine if a photo is sexually explicit. The feature is designed so that Apple does not get access to the messages."

Nothing more I could find. If you have additional details please let me know.

This is a feature that will be added in a future update there some conclusions from my part. If we just stay with the phrase "The feature is designed so that Apple does not get access to the messages." I have to note its missing the same level of detail that was published for CSAM.

I can only conclude that:

- They do not get access to the messages currently due to the current platform configuration. Note they did not say the images will stay on the phone currently or in the future. Just that uses local neural nets technology and feature is designed, so that Apple does not get access to the messages.

- They did not say they will not update the feature in the future for leveraging the icloud compute capabilities

- They did not say if there is any opt in or opt out for data for their training purposes

- They do not say if they can update locally the functionality at the request of law enforcement.

I agree with you that it looks like it stays locally by the description, but the phrase as written looks like weasel words.

They also mention one of the scenarios: Where the child will agree to send the photo to the parent before viewing, would that be device to device communication or via their icloud/CSAM system ? Unclear, at least for what I could gather so far.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#615

Post by bisRepetita »

zepto wrote: post #519
bisRepetita wrote: post #499

Earlier quoted context omitted.

Agree. It may well be a bad proxy. But it does not mean the problem is getting worse either, one would need to back it up with some arguments/estimates. The number of pictures on the internet does not seems a good one to me (cats population has not grown that big). I am thinking that people don't seem to know if the problem is really getting bigger. They just say it. If we don't know that, we cannot use growing child…

> They just say it. If we don't know that, we cannot use growing child pornography as an argument to reducing civil liberties. https://www.fbi.gov/news/stories/child-predators The FBI says it. People can and do use that argument whether we like it or not.

Yes, they said it in 2011. With just one data point and no trend. Nothing shows here it exploded since. So the FBI and the whole society may have found somewhat effective ways without scanning phones.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#616

Post by babesh »

zepto wrote: post #586
babesh wrote: post #584

Earlier quoted context omitted.

My point is that the FBI will just use that as a pretext for greater intrusion into privacy. Why stop at users who have iCloud photos turned on? Why not scan all photos? Why limit it to child predators? Why not use this too for terrorists and anyone the FBI or any other government deems as subversive? In fact, if you just look at what the FBI has been saying over the years, that is exactly what they intend to do. Peo…

Where is anyone arguing against you? The point is that the demands for solutions may be neverending, but that doesn’t mean the problems are non-existent. If we want to limit the damage caused by the dynamic, we need to offer better solutions, not just complain about the FBI.

The problem of child molesters is a fixed quantity. The loss of privacy is ever increasing. When you see this dynamic, you know that a con is afoot.

The solution for child porn isn’t found in snooping on everyone. It is in infiltrating the networks. Go have the FBI infiltrate the rings. Here is an example of why the FBI is disingenuous. This child porn ring wasn’t using phones. Guess what they were using?

https://abc7.com/archive/7844451/

Computers in public libraries

Like I said, sophisticated actors aren’t the targets.

Another example. Osama bin Laden. He was air gapped. No cell phones or computers for him. No one even calling on a phone nearby. Osama bin Laden was found via an informant.

The next actor will be even more sophisticated. Probably single use dumb paired cell phones with call locations randomized. Probably plastic surgery. Locations that spy satellites cannot see.

Did snooping on cell phone data help find Osama? When that wasn’t enough, did grabbing all online data help? How about grabbing data straight from smart phones? Nope. Nope. Nope. Yet governments want more, more, more. Why do you think snooping helps against people who don’t even use phones for their most private communications?

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#617

Post by zepto »

danShumway wrote: post #609
zepto wrote: post #608

Earlier quoted context omitted.

> Statistics are a reflection of reality. No, they are the output of a process. Whether a process reflects ‘reality’ is dependent on the process and how people understand it. This is essential to science. Even when statistics are the result of the best scientific processes available, they are typically narrow and reflect only a small portion of reality. This is why they are insufficient. > When you say that the stats…

> This is essential to science. Okay, fine. Are you claiming that people who are calling to ban encryption are doing so on a scientific basis? Come on, be serious here. People call to ban encryption because it scares them, not because they have a model of the world based on real data or real science that they're using to reinforce that belief. If they did, we could argue with them. But we can't, because they don't. >…

> This is essential to science.

>> Okay, fine. Are you claiming that people who are calling to ban encryption are doing so on a scientific basis?

No. Did I say something to that effect?

> Come on, be serious here. People call to ban encryption because it scares them, not because they have a model of the world based on real data or real science that they're using to reinforce that belief.

You say this as if you are arguing against something I have said. Why?

> If they did, we could argue with them. But we can't, because they don't.

We can still argue with them, just not with science.

> Can you find a place where this contradicts something I’ve said?

> Yes, see below:

You’ll need to explain what the contradiction is. You have said you don’t understand it, but you not understanding doesn’t make it a contradiction.

>> such that when calls to weaken encryption or insert back doors are made as they always will be, we don’t have to

> I'm open to some kind of clarification that makes this comment make sense.

It makes sense to have solutions that don’t weaken privacy. Wouldn’t you agree?

> How are your "solutions" going to make people less afraid?

They won’t.

> On what basis are you going to argue with these people that your solution is better than banning encryption?

Which people? The parents, the nefarious actors, apple’s customers?

> Pretend that I'm a concerned parent right now. I want to ban encryption. What can you tell me now to convince me that any other solution will be better?

Of course not because you are going to play the role of an irrational parent who cannot be convinced.

Neither of us disagree that such people exist. Indeed we both believe that they do.

Why does changing such a person’s mind matter?

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#618

Post by mousepilot »

mrtranscendence wrote: post #545
noasaservice wrote: post #167

Earlier quoted context omitted.

This on-device scanning is even worse. They cant even tell what was violating, or what hash, or what image. Just that the computer said you are violating. We have no idea about the hash collisions. When talking about whole world, 2^256 isn't a big enough space.... even if they're using 256 bits. And how dare anybody criticize this - criticism is tantamount to being for child porn. (Then again, that's why it was chose…

2*256 is a huge space. If everyone in the whole world had 50,000 images stored in iCloud, the probability of a collision would still be infinitesimally, unimaginably small.

2*256 is 512. you should probably type 2^256.

lol picky I know

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#619

Post by crocodiletears »

zepto wrote: post #613
crocodiletears wrote: post #601

Earlier quoted context omitted.

They can take precautions by not using iOS devices. The goal isn't to harm Apple, it's to make the knowledge available to those who need it, and have the will to avoid it. That could mean anything from watching what they put on their iPhone, to putting them down the track of using degoogled android variants on select phones, to ditching cell phones all together depending on their needs. Knowing that your phone is wat…

Android has been doing this all along, so not using iOS devices will not help. As I said, there are no good solutions.

* drop your phone

* don't store images on your phone

* use a rom like Graphene, don't install a cloud storage or Google Play Services

Those aren't great solutions, but they're options.

Re: An Open Letter Against Apple's Privacy-Invasive Content Scanning Technology

#620

Post by gordon_gee123 »

ianhawes wrote: post #20
donkeyd wrote: post #4

I recently listened to a Darknet Diaries episode on messaging app Kik. This app is apparently being used by many people to trade child pornography. In this episode, there was some criticism expressed on how Kik doesn't scan all the images on their platform for child pornography. I would really like to hear from people who sign this open letter, how they think about this. Should the internet be a free for all place wi…

Many people share photos of child pornography via the mail. There has been criticism that the USPS does not open all mail and scan them for child pornography. I would really like to hear from people who do not sign this letter how they think about that.

They already scan for bombs and hazardous materials, so yes the line is drawn somewhere between ‘let anything get sent’ and ‘track everything’
Post reply on HN