Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple wants to redefine what it means to violate your privacy. We mustn’t let it

ar.al

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#61

Post by smsm42 »

zepto wrote: post #59
foxyv wrote: post #55

Earlier quoted context omitted.

Predictive policing models require a human in the loop as well. However people tend to trust these algorithms far beyond their reliability without knowing how they work. As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. A positive result may be enough to see all of your devices in evidence bags for the next 6 months and serve…

> As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. No, this is not correct. Human reviewers see a visual derivative which is separate from the hash. It’s basically a blurred thumbnail - enough to visually confirm that the image is not a false positive, buy not enough that the reviewers are constantly exposed fo child porn. A…

I'm not buying it. If you can recognize what it is, enough to be able to be sure, you're exposed to it. If not, you're just guessing - and your guess is heavily biased towards confirming what the machine said, because Apple spend $MASSIVE_AMOUNT on this technology, and who are you to question it based on a blurry picture? Also, you'd be saving children, and if you're wrong (which you're likely not - remember, industry-leading AI!) - well, the police surely will find that out very quickly and everything will be fine.

It's not like it's the first time such systems are built. We have a cases of Big Social banning people for random stuff and then saying "it was a technical error" when the noise in the media is strong enough. We have chess channels banned for racist hatespeech. Only this time the question is not whether you will be denied the opportunity to post cat pictures on facewitter for a week. It's pretty much the most shameful accusation one can be subjected to in our society. Once the press gets hold of it - and it'd get hold of it the minute the police does - there's no coming back from it for the person affected (well, maybe if they are Hunter Biden, but not otherwise). And all that will be hinged on an anonymous drone looking at a blurry picture?

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#62

Post by foxyv »

zepto wrote: post #59
foxyv wrote: post #55

Earlier quoted context omitted.

Predictive policing models require a human in the loop as well. However people tend to trust these algorithms far beyond their reliability without knowing how they work. As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. A positive result may be enough to see all of your devices in evidence bags for the next 6 months and serve…

> As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. No, this is not correct. Human reviewers see a visual derivative which is separate from the hash. It’s basically a blurred thumbnail - enough to visually confirm that the image is not a false positive, buy not enough that the reviewers are constantly exposed fo child porn. A…

> It’s not predictive policing or anything like it, and it is obviously very much against Apple’s interest for it to generate false positives.

It is not predictive policing. However it's a pretty close cousin. Automated policing. I'm also skeptical that a blurred image will be enough to confirm/deny CP. I'm pretty sure that it's a system similar to YouTube's content ID and will work out in a very similar fashion. Also they have a very good incentive to err on the side of false positives in order to reduce liability for hosting CP on their servers.

It worked for the DMCA, now law enforcement are trying something similar for CP.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#63

Post by zepto »

foxyv wrote: post #62
zepto wrote: post #59

Earlier quoted context omitted.

> As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. No, this is not correct. Human reviewers see a visual derivative which is separate from the hash. It’s basically a blurred thumbnail - enough to visually confirm that the image is not a false positive, buy not enough that the reviewers are constantly exposed fo child porn. A…

> It’s not predictive policing or anything like it, and it is obviously very much against Apple’s interest for it to generate false positives. It is not predictive policing. However it's a pretty close cousin. Automated policing. I'm also skeptical that a blurred image will be enough to confirm/deny CP. I'm pretty sure that it's a system similar to YouTube's content ID and will work out in a very similar fashion. Als…

> It is not predictive policing. However it's a pretty close cousin. Automated policing.

It’s not policing. This is a mechanism to detect if people are uploading known child pornography images to Apple’s servers without giving Apple access to your photos. That is the only use case for this system.

Yes, if you try to upload such a collection, a police report may be filed, but only if you do this specific thing and it is verified by humans.

> I'm also skeptical that a blurred image will be enough to confirm/deny CP

It doesn’t have to confirm CP - it only has to confirm that the image matches the known CP from the database.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#64

Post by jrm4 »

zepto wrote: post #51
jrm4 wrote: post #50

Earlier quoted context omitted.

I get that questions like these are perhaps meant to be "more precise" about the specific tech in question, but honestly, they do more to display the naivete of the questioner in terms of understanding how this sort of thing plays out in real life, when it's not just tech folks, but other policymakers and stakeholders are involved and have the power to make decisions. Even if the thing Apple is talking about right no…

> it would be much MORE of a mistake to NOT lump them in together if we're trying to bring this debate to the general public, which we, of course, we should. Frankly this suggests that you think it’s a good idea to mislead the general public. I think that is one of the ways we harm our public discourse. I could be missing a connection between the two that is obvious to you but not to me, in which case I apologize. I…

Not at all -- what I'm saying is that tech people end up misleading themselves in terms of likely outcomes by having these discussions and focusing on the hard discrete lines around this or that particular technology.

What very reliably happens is -- thhe people who make the decisions (who, unfortunately, are very rarely tech people) will lump them in anyway LATER, when it actually matters and is too late.

So your point is technically correct, and simultaneously absolutely does not capture the problem with facial recognition adequately, because it doesn't factor in "if you get people to sign off on Apples specific thing today, you'll basically be able to sign off on just about anything that sort of looks like it to the layperson" tomorrow.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#65

Post by zepto »

jrm4 wrote: post #64
zepto wrote: post #51

Earlier quoted context omitted.

> it would be much MORE of a mistake to NOT lump them in together if we're trying to bring this debate to the general public, which we, of course, we should. Frankly this suggests that you think it’s a good idea to mislead the general public. I think that is one of the ways we harm our public discourse. I could be missing a connection between the two that is obvious to you but not to me, in which case I apologize. I…

Not at all -- what I'm saying is that tech people end up misleading themselves in terms of likely outcomes by having these discussions and focusing on the hard discrete lines around this or that particular technology. What very reliably happens is -- thhe people who make the decisions (who, unfortunately, are very rarely tech people) will lump them in anyway LATER, when it actually matters and is too late. So your po…

Well the ‘tech people’ line is an ad-hominem. I don’t see anyone doing a great job of knowing the right move in terms of communicating about complex issue to the public. Aren’t you a tech person too?

Who do you see as ‘the people who make the decisions’ in this case? I.e. who do you imagine will lump these together? I could see the FBI saying ‘you did CSAM detection, so surely you can do ‘child predator detection via facial recognition’. Is that what you mean?

As for getting people to sign off on stuff - I think it’s not so obvious what we do and don’t want people to sign off on. Not implementing something like this now could easily mean they are forced to scan in the cloud, and that really would be a slippery slope.

It seems like you are saying even though this thing isn’t bad, we should persuade people into not signing off on it because we want to make sure they don’t later sign off on some facial recognition thing that actually is bad. Is that an accurate enough paraphrase?

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#66

Post by zepto »

smsm42 wrote: post #61
zepto wrote: post #59

Earlier quoted context omitted.

> As I understand it, human reviewers for this program do not see the photo itself, but instead see the hash and make a determination from that. No, this is not correct. Human reviewers see a visual derivative which is separate from the hash. It’s basically a blurred thumbnail - enough to visually confirm that the image is not a false positive, buy not enough that the reviewers are constantly exposed fo child porn. A…

I'm not buying it. If you can recognize what it is, enough to be able to be sure, you're exposed to it. If not, you're just guessing - and your guess is heavily biased towards confirming what the machine said, because Apple spend $MASSIVE_AMOUNT on this technology, and who are you to question it based on a blurry picture? Also, you'd be saving children, and if you're wrong (which you're likely not - remember, industr…

> I'm not buying it. If you can recognize what it is, enough to be able to be sure, you're exposed to it. If not, you're just guessing

That’s a misunderstanding of what is happening. The reviewer doesn’t have to see whether the visual derivative looks like child porn. They only have to see whether it matches the visual derivative of the child porn image that the hash matched.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#67

Post by jrm4 »

zepto wrote: post #65
jrm4 wrote: post #64

Earlier quoted context omitted.

Not at all -- what I'm saying is that tech people end up misleading themselves in terms of likely outcomes by having these discussions and focusing on the hard discrete lines around this or that particular technology. What very reliably happens is -- thhe people who make the decisions (who, unfortunately, are very rarely tech people) will lump them in anyway LATER, when it actually matters and is too late. So your po…

Well the ‘tech people’ line is an ad-hominem. I don’t see anyone doing a great job of knowing the right move in terms of communicating about complex issue to the public. Aren’t you a tech person too? Who do you see as ‘the people who make the decisions’ in this case? I.e. who do you imagine will lump these together? I could see the FBI saying ‘you did CSAM detection, so surely you can do ‘child predator detection via…

No. I am saying "my, or anyone's evaluation of whether or not this thing today is bad is utterly meaningless, because history shows that law enforcement type powers literally never respect limits like these."

Again, your "tech knowledge" will not help you here. The lines you percieve between this "not-bad" thing and a future actually bad thing don't meaningfully exist.

Better to resort to simpler principles, go with the 4th amendment, slightly modified to include the tech companies. If the FBI wants to be in my stuff, they need a warrant, and that's it.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#68

Post by smsm42 »

zepto wrote: post #66
smsm42 wrote: post #61

Earlier quoted context omitted.

I'm not buying it. If you can recognize what it is, enough to be able to be sure, you're exposed to it. If not, you're just guessing - and your guess is heavily biased towards confirming what the machine said, because Apple spend $MASSIVE_AMOUNT on this technology, and who are you to question it based on a blurry picture? Also, you'd be saving children, and if you're wrong (which you're likely not - remember, industr…

> I'm not buying it. If you can recognize what it is, enough to be able to be sure, you're exposed to it. If not, you're just guessing That’s a misunderstanding of what is happening. The reviewer doesn’t have to see whether the visual derivative looks like child porn. They only have to see whether it matches the visual derivative of the child porn image that the hash matched.

So they'd just compare two small very blurry thumbnails? Not sure that makes it any better.

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#69

Post by zepto »

jrm4 wrote: post #67
zepto wrote: post #65

Earlier quoted context omitted.

Well the ‘tech people’ line is an ad-hominem. I don’t see anyone doing a great job of knowing the right move in terms of communicating about complex issue to the public. Aren’t you a tech person too? Who do you see as ‘the people who make the decisions’ in this case? I.e. who do you imagine will lump these together? I could see the FBI saying ‘you did CSAM detection, so surely you can do ‘child predator detection via…

No. I am saying "my, or anyone's evaluation of whether or not this thing today is bad is utterly meaningless, because history shows that law enforcement type powers literally never respect limits like these." Again, your "tech knowledge" will not help you here. The lines you percieve between this "not-bad" thing and a future actually bad thing don't meaningfully exist. Better to resort to simpler principles, go with…

> Again, your "tech knowledge" will not help you here. The lines you percieve between this "not-bad" thing and a future actually bad thing don't meaningfully exist.

Are you saying you don’t understand the technology? That you don’t have knowledge about this subject?

> Better to resort to simpler principles, go with the 4th amendment, slightly modified to include the tech companies. If the FBI wants to be in my stuff, they need a warrant, and that's it.

If you don’t understand the technology how will you know whether it violates the 4th amendment or not?

Re: Apple wants to redefine what it means to violate your privacy. We mustn’t let it

#70

Post by zepto »

smsm42 wrote: post #68
zepto wrote: post #66

Earlier quoted context omitted.

> I'm not buying it. If you can recognize what it is, enough to be able to be sure, you're exposed to it. If not, you're just guessing That’s a misunderstanding of what is happening. The reviewer doesn’t have to see whether the visual derivative looks like child porn. They only have to see whether it matches the visual derivative of the child porn image that the hash matched.

So they'd just compare two small very blurry thumbnails? Not sure that makes it any better.

It’s not literal blurring - it’s a transform that means you don’t see the actual image, but you do see features and detail that lets you easily tell two source images apart.

The point is that you aren’t looking to see if there is porn in the image. You are looking to seem if the image matches the porn image.

It’s a good mechanism.

Post reply on HN