Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple wants to redefine what it means to violate your privacy. We mustn’t let it

ar.al

Post reply on HN
Print view View original

51–60 of 83 posts

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#51

Post by zepto »

jrm4 wrote: post #50
zepto wrote: post #46

Earlier quoted context omitted.

What does any of this have to do with facial recognition tech or race?

I get that questions like these are perhaps meant to be "more precise" about the specific tech in question, but honestly, they do more to display the naivete of the questioner in terms of understanding how this sort of thing plays out in real life, when it's not just tech folks, but other policymakers and stakeholders are involved and have the power to make decisions. Even if the thing Apple is talking about right no…

> it would be much MORE of a mistake to NOT lump them in together if we're trying to bring this debate to the general public, which we, of course, we should.

Frankly this suggests that you think it’s a good idea to mislead the general public. I think that is one of the ways we harm our public discourse. I could be missing a connection between the two that is obvious to you but not to me, in which case I apologize.

I am not naïve about facial recognition. I am not white, and I have known about ML technology since the 90s. Long before it became a known problem it was obvious to me that ML models would end up simply reflecting the biases of the corpus they were trained on, and this would lead to them embodying discrimination of one kind or another, some of which would not be obvious in advance. The perceived ‘neutrality’ of algorithms would be and in fact is invoked to minimize this problem, when of course the problem is not with the algorithms but with what people feed into them.

Please let me know if that doesn’t capture the problem with facial recognition adequately.

So, given that I’m not naïve about facial recognition, I ask again - what is the connection you see here between racially biased facial recognition and Apple’s CSAM countermeasures?

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#52

Post by oceanghost »

gjsman-1000 wrote: post #33
josh_today wrote: post #20

What Apple is doing is the equivalent of the police one day deciding to search everyone’s physical home photo albums just in case there’s a picture of an illegal activity.

Not really, actually. They only do the scanning if you use iCloud Photos backup. If you use, say, local storage instead of iCloud, you don't get scanned (which would be a more apt comparison to a physical photo book). Also, they aren't scanning for new illegal activity, but images that you somehow obtained of past illegal activity that the government already knows about.

This to me is the most unbelievable aspect of the whole thing.

Apple is basically saying:

"So hey, if you have anything you don't want us to know about, here's how to get around our scans (wink, wink)."

What would be the point of implementing a system and then telling people how to avoid it? I think at some point in the near future they will flip a switch and start scanning private photos.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#53

Post by josh_today »

gjsman-1000 wrote: post #33
josh_today wrote: post #20

What Apple is doing is the equivalent of the police one day deciding to search everyone’s physical home photo albums just in case there’s a picture of an illegal activity.

Not really, actually. They only do the scanning if you use iCloud Photos backup. If you use, say, local storage instead of iCloud, you don't get scanned (which would be a more apt comparison to a physical photo book). Also, they aren't scanning for new illegal activity, but images that you somehow obtained of past illegal activity that the government already knows about.

I see your distinction and it makes sense- don’t want to be subject to Apple’s CSAM checks, don’t upload to iCloud.

I’d say we can do better. We need better tools for individualized data ownership and interoperability.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#54

Post by y7 »

falcolas wrote: post #13
throw03172019 wrote: post #7

Earlier quoted context omitted.

There is a human review (ugh.) as well. But still, your example is very concerning.

From Apple's announcements, there is a human comparison of the visual hashes . They don't appear to be forcing humans to view the images themselves.

The technical assessments speak about "visual derivatives". I'm guessing they mean a low resolution thumbnail.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#55

Post by foxyv »

zepto wrote: post #44
foxyv wrote: post #2

The thing that scares me isn't so much the violation of privacy. It's the idea that some computer algorithm can accuse me of a crime automatically with no evidence and generate an investigation. Judging by how police respond to these leads, you can end up in jail based on this "Evidence." While you wait for a 6 month investigation to be completed you lose your job and get an arrest record. Even if your photo is just…

> The thing that scares me isn't so much the violation of privacy. It's the idea that some computer algorithm can accuse me of a crime automatically with no evidence and generate an investigation. Nothing about apple’s proposal involves algorithms accusing people with a crime without evidence. All actual alerts are done by humans checking the photos.

Predictive policing models require a human in the loop as well. However people tend to trust these algorithms far beyond their reliability without knowing how they work. As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that.

A positive result may be enough to see all of your devices in evidence bags for the next 6 months and serve as probable cause for warrants. In addition, how hard is it to subpoena Apple for a report on how many times a whistleblower's device has been flagged then use it as probable cause. Or, cherry pick which reports Apple makes to target undesirables.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#56

Post by foxyv »

throw03172019 wrote: post #32
foxyv wrote: post #23

Earlier quoted context omitted.

They say there is a human review, but fail to explain the parameters of the review. How many photos is a reviewer expected to review in an hour? What are their incentives and performance metrics? Do they have a 3 party voting system to confirm? For positive human review is there additional scrutiny? Is data anonymized properly? Does law enforcement receive a copy? Is this going to be limited to child exploitation? Wi…

I would hope they aren’t incentivized to incriminate innocent people.

I would hope so as well. However, my hopes for these policing algorithms have been dashed too many times. The potential for abuse is just too great without effective laws and appropriate procedures. We've seen the results at YouTube, Pasco County Florida, the UK, and Facebook.

There is a reason that a key part of justice is being able to face your accusers. Secret algorithms are being used to abrogate this right. You can't cross examine an algorithm. Expert witnesses to contradict its expertise are too expensive for most defendants. Then the developers of the algorithm can claim with plausible deniability that the algorithm is 1 in a trillion accurate.

It's essentially the new bite mark analysis and polygraph pseudoscience used to elicit false admissions of guilt or convince gullible juries. "We have the hash of your photo George, we KNOW you are a pedophile. Just admit it and you can go home. We'll go easy on you. We promise!"

https://www.forbes.com/sites/nicksibilla/2021/04/26/lawsuit-...

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#57

Post by ksec »

>I will not write another line of code for their platforms ever again.

Surprised that this is still a thing. Apple has made it very clear in their App Store case, they do not need developers and Apps on their platform. And Apple operating their App Store has been a benefits, or more like a gift to developers for access to their users.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#58

Post by salawat »

zepto wrote: post #45
aNoob7000 wrote: post #9

Earlier quoted context omitted.

I want to add that they are scanning your property without your permission. The crazy part is that the government can't do this with a warrant or probable cause, but Apple decided that it can and will. Amazing that Apple destroyed their image about "Privacy" in minutes.

> I want to add that they are scanning your property without your permission. This is a false statement. Scanning is opt-in.

Allow it or don't buy our product is a fairly disingenuous take on "opt-in".

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#59

Post by zepto »

foxyv wrote: post #55
zepto wrote: post #44

Earlier quoted context omitted.

> The thing that scares me isn't so much the violation of privacy. It's the idea that some computer algorithm can accuse me of a crime automatically with no evidence and generate an investigation. Nothing about apple’s proposal involves algorithms accusing people with a crime without evidence. All actual alerts are done by humans checking the photos.

Predictive policing models require a human in the loop as well. However people tend to trust these algorithms far beyond their reliability without knowing how they work. As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. A positive result may be enough to see all of your devices in evidence bags for the next 6 months and serve…

> As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that.

No, this is not correct. Human reviewers see a visual derivative which is separate from the hash. It’s basically a blurred thumbnail - enough to visually confirm that the image is not a false positive, buy not enough that the reviewers are constantly exposed fo child porn.

Also remember that multiple matches are required to even get to the human review.

The rest of your comment really doesn’t seem to match the system being described. It’s not predictive policing or anything like it, and it is obviously very much against Apple’s interest for it to generate false positives.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#60

Post by zepto »

salawat wrote: post #58
zepto wrote: post #45

Earlier quoted context omitted.

> I want to add that they are scanning your property without your permission. This is a false statement. Scanning is opt-in.

Allow it or don't buy our product is a fairly disingenuous take on "opt-in".

> Allow it or don't buy our product is a fairly disingenuous take on "opt-in".

That’s not how it works.

The only thing it affects is iCloud Photo Library. There are numerous competitors you can choose from if you want a cloud photo library, and nothing prevents you from just keeping your photos on device or exporting them to a PC or Mac.

Post reply on HN