The deceptive PR behind Apple’s “expanded protections for children” • Hacker News Board

Re: The deceptive PR behind Apple’s “expanded protections for children”

#291

Post by samstave » Thu, Aug 12, 2021, 11:32 PM UTC

ummonk wrote: post #215

fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

To be clear, it's 1 in 1 trillion per account. 1 in 1 trillion per photo would potentially be a more realistic risk, since some people take tens of thousands of photos.

except for the time that its 100% for an account...

But the truly sad thing is that there are trillions of instances, photos, moments that will never even see the light of day. and plenty of human children get abused, raped, murdered every single day.

Child abuse should be a capital crime.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#292

Post by XorNot » Thu, Aug 12, 2021, 11:33 PM UTC

FabHK wrote: post #223

joe_the_user wrote: post #106

Earlier quoted context omitted.

The Daring Fireball article actually is a little deceptive here. It goes over a bunch of that won't get parents in trouble and gives a further couched justification of the finger printing example. The question is whether an ordinary baby photo is likely to collide with the one of the CSAM hashes Apple will be scanning for. I don't think Apple can give a definite no here (Edit: how could give a guarantee that a system…

> how could give a guarantee that a system that finds any disguised/distorted CSAM won't tag a random baby picture with a similar appearance. Cannot guarantee, but by choosing a sufficiently high threshold, you can make the probability of that happening arbitrarily small. And then you have human review. > And given such collision, the picture might be looked at by Apple and maybe law enforcement No, not "the picture"…

> "visual derivative".

Do you have any idea that means? Because I certainly don't - how could you possibly identify whether an image is CSAM without looking at something which is reasonably the same image?

What is a visual derivative? Take that algorithm and run it over some normal images and show me what they look like.

All of this is being aggressively talked around because everyone knows it's not going to stand up to any reasonable scrutiny (i.e. plenty of big image datasets out there - does Apple's implementation flag on any of those? Who knows - they're not going to refer to anything specific about how they got "1 in a trillion").

Re: The deceptive PR behind Apple’s “expanded protections for children”

#293

Post by vineyardmike » Thu, Aug 12, 2021, 11:35 PM UTC

fortenforge wrote: post #137

querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

That 1/1t rate apple gave is post human review according to a recent interview

Re: The deceptive PR behind Apple’s “expanded protections for children”

#294

Post by drenvuk » Thu, Aug 12, 2021, 11:35 PM UTC

gonehome wrote: post #284

drenvuk wrote: post #271

Earlier quoted context omitted.

They've given you two options: 1. Encrypt everything in the cloud but upload the hashes of these items as well on the device. Also notify us so we can notify law enforcement if they're doing some illegal stuff. 2. Everything is unencrypted in the cloud. No actions are taken on the device. No notifications to authorities. With option one the sanctity of the device ownership is breached. With option two it's maintained…

For #2 actions are taken in the cloud, authorities are contacted, and all of the photos are unencrypted. Option #1 doesn’t really breach the ‘sanctity of device ownership’ because it only occurs if you’ve enabled iCloud photos on the device. Option #1 seems better to me in its current implementation. I understand the fear of abusing the hash matching. I just think that’s a separate thing.

>Option #1 doesn’t really breach the ‘sanctity of device ownership’ because it only occurs if you’ve enabled iCloud photos on the device.

I'm done. You'll rationalize anything.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#295

Post by kristofferR » Thu, Aug 12, 2021, 11:35 PM UTC

stickfigure wrote: post #213

fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

The false positive rate for any given image is not 1 in a trillion. Perceptual hashing just does not work like that. It also suffers from the birthday paradox problem - as the database expands, and the total number of pictures expands, collisions become more likely. The parent poster does make the mistake of assuming that other pictures of kids will likely cause false positives. Anything could trigger a false positiv…

> Like, say, the naughty pictures you've been taking of your (consenting) adult partner. I'm sure Apple's outsourced low-wage-country verification team will enjoy those.

I'm not sure they'll be able to after looking at CSAM all day...

Re: The deceptive PR behind Apple’s “expanded protections for children”

#296

Post by jjcon » Thu, Aug 12, 2021, 11:35 PM UTC

AlexandrB wrote: post #20

seph-reed wrote: post #9

I really don't see why the scanning would ever be done on the phone instead of on iCloud if it only affects iCloud images. But I do have guesses why.

That's the crux of it. Why bother with on-device identification, unless one of: a. Apple intends to E2E encrypt iCloud data. b. This is intended to extend to all photos on the device in the future. I'm hoping it's (a), but it's probably (b). And in either case it sets a bad precedent for other companies to follow. Edit: This also turns every jailbreak into a possible CSAM detection avoidance mechanism, giving the gov…

Even if it is A though, what is the point of E2E encryption if your endpoints are compromised (there is none). The only purpose at that point would be marketing which does seem like something Apple would do but I sincerely doubt that will be the case

Re: The deceptive PR behind Apple’s “expanded protections for children”

#297

Post by FabHK » Thu, Aug 12, 2021, 11:36 PM UTC

gambiting wrote: post #272

FabHK wrote: post #254

Earlier quoted context omitted.

> because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) No. If the number of matches to known CSAM in your library exceeds a threshold, then a person will look at a "visual derivative" of only those pictures whose perceptual hatch match that of known CSAM. Note that, if I understand correctly, pictures that Android users sync to Google have already been s…

>>the number of matches to known CSAM in your library exceeds a threshold, then a person will look at a "visual derivative" of only those pictures whose perceptual hatch match that of known CSAM Apple very specifically said that their employees will look at the suspected picture before sending it through to authorities. Where do you see the bit about visual derivatives? What would that even mean or look like? Also wh…

> Where do you see the bit about visual derivatives?

In Apple's white paper about the proposed feature:

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

See the links at the bottom here for more:

https://www.apple.com/child-safety/

> Also what is this threshold?

The "perceptual hash" is supposed to match a specific image (though possibly cropped, or otherwise altered a bit, such as through a filter), not "toddlers" per se.

> Google doesn't use perceptual hashing, or at least haven't said they do.

I don't know what the other cloud providers are doing, but I'd be very surprised if they use (trivially circumventable) cryptographic hashes.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#298

Post by Hackbraten » Thu, Aug 12, 2021, 11:36 PM UTC

aczerepinski wrote: post #260

benlivengood wrote: post #174

Earlier quoted context omitted.

No chaos. The photos would be reported, reviewers would say "that's weird" since the false positive was obviously harmless and the industry would eventually switch to a different hash method while ignoring the false positives generated by the collision. If there were a flood of false positive images being produced the agencies would work faster to come up with a new solution, not perform mass arrests.

I thought the images were encrypted after the hashing was done locally. Reviewers can still view them?

Any local match causes a “safety voucher” to be uploaded along with the encrypted image. The voucher contains a (fragment of a) decryption key. If that fragment is combined with enough of its buddies from other vouchers, Apple gets to decrypt the image.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#299

Post by nicce » Thu, Aug 12, 2021, 11:36 PM UTC

1vuio0pswjnm7 wrote: post #206

"The hypothesis that I have is that Apple wishes to distance itself from checking users' data. They've been fighting with the FBI and the federal government for years , they've been struggling with not reporting CSAM content to the NCMEC, they don't want to be involved in any of this anymore." However there is close to zero evidence to support this idea. I was just reading something the other day that directly contra…

> However there is close to zero evidence to support this idea

But there is. For example Apple used a lot of effort in this area, when they built their hardware security module (HSM), which is basically on every iPhone and iPad.

This module is built in such a way, that nobody, not even Apple can access security keys from this device, or reprogram it again. Locked iPhone or password vault stays locked or gets cleaned. One blog about this matter: https://blog.cryptographyengineering.com/2016/08/13/is-apple...

How about user tracking? Apple is one of the few companies which is not caught yet by selling data to third parties, nor even collecting more than needed to develop their products.

Our new fancy iCloud is maybe the biggest evidence? It is maybe the cleverest way to this date to enable somekind of E2EE while getting limited info about the content. Highly recommed reading that PSI paper.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#300

Post by XorNot » Thu, Aug 12, 2021, 11:38 PM UTC

FabHK wrote: post #243

shapefrog wrote: post #99

Earlier quoted context omitted.

If icloud backup works as advertised - it backs up your device. However, if we consider the slipery slope, under pressure from a shaddow government, the contents of your phone could have been uploaded to the CIA every day, including live recordings 24 hours a day.

Yes, but that has always been the case. It can upload to iCloud, it could also upload to the CIA. What has changed?

Policy-wise Apple have just said "yes, we are going to use our super-admin powers to push updates to turn your phone against you".

Sure, a suitably powerful authoritarian org could do lots of secret things, but that isn't what happens in real life: in real life you publicly change policy in increments and get everyone to go along with it.

"Apple was actually scanning all users photos for CSAM regardless of iCloud usage due to a bug in the most recent firmware" is a headline that is guaranteed in the future.