Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple wants to redefine what it means to violate your privacy. We mustn’t let it

ar.al

Post reply on HN
Print view View original

71–80 of 83 posts

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#71

Post by heavyset_go »

foxyv wrote: post #2

The thing that scares me isn't so much the violation of privacy. It's the idea that some computer algorithm can accuse me of a crime automatically with no evidence and generate an investigation. Judging by how police respond to these leads, you can end up in jail based on this "Evidence." While you wait for a 6 month investigation to be completed you lose your job and get an arrest record. Even if your photo is just…

This is why algorithmic and predictive policing is so terrifying. Everyone in the justice system now has a great way to shirk liability for their actions and decisions: they can just defer to the infallible black box system that tells them who is a criminal or not. Want to side step accusations of bias or targeting? They were just going after whoever the black box said was guilty, and computers purportedly are not biased.

The corollary to this shirking of responsibility is how it lets people, and not just law enforcement, justify abuse of whoever the system says is guilty of a crime, because after all, the company that made the black box says that there is a one in a trillion chance of encountering a false positive. Judges will throw the book at defendants because, statistically, the black box system is almost never wrong, and the system says the defendants are monsters.

But the most Kafkaesque part is that people will never get an answer for how these systems determined they were guilty. It's a trade secret, it's part of on-going investigations, it's critical to national security, or in Apple's case, it's literally illegal to look and find out how their system came to conclusion because viewing the data itself is a crime. These systems often inscrutable ML models, as well, and we all know just how buggy and error prone computers and software can be.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#72

Post by heavyset_go »

api wrote: post #26
aNoob7000 wrote: post #9

Earlier quoted context omitted.

I want to add that they are scanning your property without your permission. The crazy part is that the government can't do this with a warrant or probable cause, but Apple decided that it can and will. Amazing that Apple destroyed their image about "Privacy" in minutes.

I really wonder about their motives for this. It really has destroyed their image to the point that I'm thinking about abandoning the platform if nothing changes. This will not just be about CP. In many countries this will instantly be used to detect any kind of dissent. In the US it will take longer, but eventually I can imagine "misinformation" as determined by some stupid algorithm flagging you for quiet social cr…

> This will not just be about CP. In many countries this will instantly be used to detect any kind of dissent. In the US it will take longer, but eventually I can imagine "misinformation" as determined by some stupid algorithm flagging you for quiet social credit style lists. What constitutes misinformation will change with the political winds.

In the US, there are over 70,000 overdose deaths a year[1], and that number has increased dramatically each year.

If Apple is already scanning photos and messages, why not save 70,000+ lives while they're at it by detecting heroin and fentanyl dealers, too?

[1] https://www.cdc.gov/drugoverdose/deaths/index.html

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#73

Post by weikju »

system16 wrote: post #22

I can't understand how anyone - even Apple's usual cheerleaders like Gruber - can justify defending this with a straight face. It's scanning your content on your device, without your consent. Full stop. Apple's FAQ to try and quell some of the backlash for this just makes it sound even worse in my opinion with gems like this: _Could governments force Apple to add non-CSAM images to the hash list?_ _Apple will refuse…

> Maybe there will be more of an uproar when this inevitably comes to macOS.

It's coming to the next macOS release this fall.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#74

Post by heavyset_go »

bogwog wrote: post #39

Serious question: why did Apple bother making that announcement at all? I can't imagine they're naive enough to think it would be good press for them? They could have done this quietly without telling anyone, maybe with a vaguely-worded update to the terms of service for the next mandatory iOS update that nobody reads anyways.

> Serious question: why did Apple bother making that announcement at all? I can't imagine they're naive enough to think it would be good press for them?

Because they are proud of it.

I think for some people, this was a feel good project. I know I'd feel better about working for a company that profits from forced labor[1] and lobbies to water down legislation against forced labor[2] if I was working to help children there.

Also, who wouldn't want to stop CSAM distribution? They probably thought dissent would be shamed and outnumbered.

[1] https://www.theverge.com/2021/5/10/22428899/apple-suppliers-...

[2] https://www.washingtonpost.com/technology/2020/11/20/apple-u...

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#75

Post by heavyset_go »

bogwog wrote: post #42
vxNsr wrote: post #5

If you wanna screw someone over and you know they have an iPhone with icloud backup set up, you can whatsapp them a pic that matches CP signature.

This could actually have some very serious consequences. Quote from the apple announcement: > Apple then manually reviews each report to confirm there is a match, disables the user’s account, and sends a report to NCMEC. So a match doesn't just get you reported to the authorities, it disables your account. Even if the feds do investigate you and find you innocent due to being swatted or whatever, you still need to fi…

> This could also lead to large spam campaigns of CP being sent to random people, either to just fuck with them, or to undermine the system/overwhelm investigators with a bunch of false positives.

If you're a nation-state that wants to sow discord and distrust in another nation's society, what better way than to frame random, respected or important people?

The attacker wouldn't need to worry about being arrested, due to being in a foreign country and most likely associated with foreign intelligence agencies. Those same agencies would have the evidence they could plant on people's devices or cloud storage, and the resources not to get caught.

They wouldn't even have to burn zero days or accounts/numbers to send messages, just look for database dumps of services that the target country uses, and then log in and upload the evidence.

It could be cheap, easy and scalable.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#76

Post by jrm4 »

zepto wrote: post #69
jrm4 wrote: post #67

Earlier quoted context omitted.

No. I am saying "my, or anyone's evaluation of whether or not this thing today is bad is utterly meaningless, because history shows that law enforcement type powers literally never respect limits like these." Again, your "tech knowledge" will not help you here. The lines you percieve between this "not-bad" thing and a future actually bad thing don't meaningfully exist. Better to resort to simpler principles, go with…

> Again, your "tech knowledge" will not help you here. The lines you percieve between this "not-bad" thing and a future actually bad thing don't meaningfully exist. Are you saying you don’t understand the technology? That you don’t have knowledge about this subject? > Better to resort to simpler principles, go with the 4th amendment, slightly modified to include the tech companies. If the FBI wants to be in my stuff,…

No, I'm saying stop being a nerd. I do understand the technology just fine -- but that's entirely beside the point. A deep understanding of the technology is not necessary. A shallow understanding is sufficient.

Let's go back to a real case, Kyllo v. US. They used a thermal imaging camera to "look inside" a building, from the unusual heat signature, they correctly presumed a marijuana grow operation, and went in without a warrant.

Doesn't matter though. The court said they needed a warrant because people should be able to presume a level of privacy that the camera violated. Anything that one reasonably believes is private should be protected like that, regardless of whether you are using technology to "look" at it or not. The authorities observed a thing that a reasonable person would think of as private because they treated it that way.

Same applies here, even moreso, because Apple has previously guaranteed privacy, and already- this is not privacy on its face.

The only really important gap in knowledge is the one I mentioned before, that this does also end up in a slippery slope.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#77

Post by zepto »

jrm4 wrote: post #76
zepto wrote: post #69

Earlier quoted context omitted.

> Again, your "tech knowledge" will not help you here. The lines you percieve between this "not-bad" thing and a future actually bad thing don't meaningfully exist. Are you saying you don’t understand the technology? That you don’t have knowledge about this subject? > Better to resort to simpler principles, go with the 4th amendment, slightly modified to include the tech companies. If the FBI wants to be in my stuff,…

No, I'm saying stop being a nerd. I do understand the technology just fine -- but that's entirely beside the point. A deep understanding of the technology is not necessary. A shallow understanding is sufficient. Let's go back to a real case, Kyllo v. US. They used a thermal imaging camera to "look inside" a building, from the unusual heat signature, they correctly presumed a marijuana grow operation, and went in with…

> No, I'm saying stop being a nerd.

That seems like a pretty empty thing to say at the best of times. It’s not clear how it helps.

> I do understand the technology just fine -- but that's entirely beside the point.

Do you? That remains to be seen. A shallow understanding is only sufficient if it is correct and supports your other ideas.

As for Kyllo v US, that doesn’t obviously explain anything about lumping this in with biased facial rec. We seem to have moved away from that remark.

It’s unclear what you mean when you say Apple has ‘guaranteed’ privacy.

Do you mean, they have said they will only use this technology to check for CSAM, and they won’t use it for anything else?

If so, then I agree. Now that this has been publicized, and Apple has released detailed answers to the privacy concerns, people can reasonably expect it to be used for only the purpose of preventing CSAM being uploaded to iCloud, and not for anything else. That is a publicly documented committment and seems like would stand up well in court.

By your reasoning, there is no slippery slope, and this feature is exactly what Apple says it is, because they have made such public commitments. You say it isn’t privacy on its face but why do you think that?

If the system only reports already publicly known CSAM, and is well known to do so, and is part of an opt-in service, how is that a privacy violation?

If on the other hand you are claiming that Apple has made some other blanket ‘guarantee’ of privacy and this new feature contradicts that, I’d be curious to know what guarantee you are referring to.

It’s worth noting that once detected, CSAM must be reported by statute, and other cloud providers report tens of millions of images per year. I don’t know what the status of these reports are in the courts, of whether they have been tested.

I am not a lawyer, but perhaps you are.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#78

Post by tzs »

jaywalk wrote: post #27

A lot of people seem to be forgetting/not know about the other aspect of what Apple will be doing, which is scanning iMessage pics sent or received by minors for nudity. If it's detected, their parents will be notified and have the ability to view the pic in question.

> If it's detected, their parents will be notified and have the ability to view the pic in question.

There are some other steps in there. If such a photo is detected the minor is notified that it may be intended to harm them and the subject of the photo may not have consented to sharing it. They are asked if they are sure they want to view the photo.

If they say they are sure and they are between 13 and 17 they are shown a blurred version of the photo. Their parents are not notified.

If they say they are sure and they are under 13 they are told it is their choice but their parents want to make sure they are OK and will be notified so they can check.

If they then elect to view the photo they are shown a blurred version of the photo and their parents are notified.

This is all off by default. Parents have to expressly opt in to it when they set up a child's device with Family Sharing.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#79

Post by onepunchedman »

mypastself wrote: post #38
onepunchedman wrote: post #16

Earlier quoted context omitted.

It's the same excuse the EU are currently using to infringe upon its citizens' privacy and require messaging application providers to install backdoors. It's an appeal to emotion, and since we have to assume that these legislators are intelligent, it's a disgusting overreach.

They are indeed overreaches, but even a valuable child protection law or service might presumably be pushed with an appeal to emotion. It’s better to familiarize yourself with the situation before dismissing it out of hand. While preserving a healthy dose of skepticism, of course.

I agree that one should look into what is being proposed and its implementation, but in both these instances backdoors are being introduced. Once backdoors are in place any government can petition access, malicious actors have attack vectors, and all of this for one proposed quasi-legitimate use-case.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#80

Post by jrm4 »

zepto wrote: post #77
jrm4 wrote: post #76

Earlier quoted context omitted.

No, I'm saying stop being a nerd. I do understand the technology just fine -- but that's entirely beside the point. A deep understanding of the technology is not necessary. A shallow understanding is sufficient. Let's go back to a real case, Kyllo v. US. They used a thermal imaging camera to "look inside" a building, from the unusual heat signature, they correctly presumed a marijuana grow operation, and went in with…

> No, I'm saying stop being a nerd. That seems like a pretty empty thing to say at the best of times. It’s not clear how it helps. > I do understand the technology just fine -- but that's entirely beside the point. Do you? That remains to be seen. A shallow understanding is only sufficient if it is correct and supports your other ideas. As for Kyllo v US, that doesn’t obviously explain anything about lumping this in…

I am, but that's beside the point.

Look, you're a huge sucker if you think that the boundaries of the tech and the stated policy today are 1) not fluid and 2) here's the bigger part -- aren't there primarily for the purpose of laying the groundwork for more intrusive spying. That's the "nerd" charge. If you follow the words they are saying and treat those as gospel and limiting, you're a nerd and a sucker.

And to take it further, if I seem paranoid or whatnot -- that's fine; it's better and smarter to be wrong in my direction than it is in the sucker direction, where you can't put the toothpaste back in the proverbial tube.

Post reply on HN