Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The deceptive PR behind Apple’s “expanded protections for children”

piotr.is

Post reply on HN
Print view View original

361–370 of 595 posts

Re: The deceptive PR behind Apple’s “expanded protections for children”

#361

Post by bonestamp2 »

fortenforge wrote: post #137
querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

> Apple requires a specific threshold of matches before a report is triggered

True, but the wording of that condition was very vague... the threshold could be 1.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#362

Post by [deleted] »

stetrain wrote: post #276
farmerstan wrote: post #227

Earlier quoted context omitted.

How do new hashes get added to this database? How do we know that all the hashes are of CSAM? Who is validating it and is there an audit trail? Or can bad actors inject their own hashes into the database and make innocent people get reported as pedophiles?

There is a manual review step by Apple. You have to first get a “significant” number of photos flagged, then they have to pass Apple’s manual review (ie looking at photo thumbnails), and only then does Apple report the account.

[deleted]

Re: The deceptive PR behind Apple’s “expanded protections for children”

#363

Post by salawat »

jet_32951 wrote: post #210
throwaway212135 wrote: post #182

Earlier quoted context omitted.

I am not sure the right questions are being asked. 1. Who is adding these photos to NCMEC? 2. How often are these photos added? 3. How many people have access to these photos - both adding and viewing? Everyone is focused on Apple and no one is looking at MCMEC. If I wanted to plant a Trojan horse, I would point everyone towards Apple and perform all of the dirty work on the NCMEC end of things.

Exactly. An unknown mechanism adds hashes to a NGO subject to exactly what conditions? This initiative makes me extremely leery of black boxes, to the extent that any algorithm between subject and accusation had damned well better be explainable outside the algorithm; else I as a jury member am bound to render a "not guilty" verdict.

Welcome to "rejected in voire dire".

Re: The deceptive PR behind Apple’s “expanded protections for children”

#364

Post by nomel »

stetrain wrote: post #276
farmerstan wrote: post #227

Earlier quoted context omitted.

How do new hashes get added to this database? How do we know that all the hashes are of CSAM? Who is validating it and is there an audit trail? Or can bad actors inject their own hashes into the database and make innocent people get reported as pedophiles?

There is a manual review step by Apple. You have to first get a “significant” number of photos flagged, then they have to pass Apple’s manual review (ie looking at photo thumbnails), and only then does Apple report the account.

I find how severely this is all being misunderstood, as in very few actually RTFM, on a site like HN, incredibly eye opening.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#365

Post by nicce »

hu3 wrote: post #280
crummy wrote: post #193

Earlier quoted context omitted.

How is that different from how things work now?

Before, Apple would only scan data already in the cloud. Now the pandora box has been opened. They are adding capability to scan files on iPhones before it hits the cloud. Any technical or financial excuse they might have used in the past to not scan files locally is now rendered null. Governments can just say: "you know what? scan these arbitrary sets of hashes as well, they are illegal in my jurisdiction and since…

> Any technical or financial excuse they might have used in the past to not scan files locally is now rendered null.

This just proves that people don’t understand much about technology in depth. Capability has been there already, for very long time.

99% of their work has gone for implementing that perceptual hashing function and their PSI system.

If you want to give excuses, there will be always more. But they are not reasons to prevent this pandora box.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#366

Post by cwkoss »

slownews45 wrote: post #150
still_grokking wrote: post #70

Earlier quoted context omitted.

According to https://rentafounder.com/the-problem-with-perceptual-hashes/ the false-positive rate will be likely high. Given the billions of pictures going through this system there are going to be a lot of false accusations of child porn possession likely (and alone such an accusation can ruin lives). HN discussion of that article from a few days ago: https://news.ycombinator.com/item?id=28091750

This is where the thresholding and manual review come in, but could be a bit scary for sure.

Even with thresholding and manual review, the idea that most iphone users (who don't have any CP) are going to have a non-zero child porn score in apple's database is just super creepy.

Just a minor policy change (lower threshold) away from calling millions of people pedophiles.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#367

Post by ec109685 »

FabHK wrote: post #194
fortenforge wrote: post #137

Earlier quoted context omitted.

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

Probability of a false positive for a given image = p Probability of N false positives (assuming independence) = p^N Threshold N is chosen by Apple such that p^N -12 log(10)/log(p) [since log(p) ETA: Suppose, just for the sake of the argument, that p = 10^-3 (one false positive in 1000, so really quite bad). Then log(p) = -3 log(10), so N > -12 log(10)/(-3 log(10)) = 12/3 = 4. Similarly, if p is one in a million (10^…

If the threshold is 4, then 4 photos needed to have incorrectly matched, meaning the accuracy needs to only be 1/10000 if the average user has 10k images. (1/10000)^4*10k is 1 trillion.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#368

Post by simondotau »

nomel wrote: post #358
stickfigure wrote: post #213

Earlier quoted context omitted.

The false positive rate for any given image is not 1 in a trillion. Perceptual hashing just does not work like that. It also suffers from the birthday paradox problem - as the database expands, and the total number of pictures expands, collisions become more likely. The parent poster does make the mistake of assuming that other pictures of kids will likely cause false positives. Anything could trigger a false positiv…

My guess is that the one in a trillion figure includes the threshold that must be exceeded, requiring multiple false positives, bringing the full chance of your account being flagged to 1 in a trillion.

Correct, the "1 in 1 trillion" does factor in the requirement for multiple images to match. From Apple's technical summary:

"Using another technology called threshold secret sharing, the system ensures that the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content. Only when the threshold is exceeded does the cryptographic technology allow Apple to interpret the contents of the safety vouchers associated with the matching CSAM images."

"The threshold is selected to provide an extremely low (1 in 1 trillion) probability of incorrectly flagging a given account. This is further mitigated by a manual review process wherein Apple reviews each report to confirm there is a match..."

And when the manual review process sees that the images flagged aren't NCMEC classification A1 (A=prepubescent, 1=sex acts) the flag is cleared.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#369

Post by hu3 »

nicce wrote: post #365
hu3 wrote: post #280

Earlier quoted context omitted.

Before, Apple would only scan data already in the cloud. Now the pandora box has been opened. They are adding capability to scan files on iPhones before it hits the cloud. Any technical or financial excuse they might have used in the past to not scan files locally is now rendered null. Governments can just say: "you know what? scan these arbitrary sets of hashes as well, they are illegal in my jurisdiction and since…

> Any technical or financial excuse they might have used in the past to not scan files locally is now rendered null. This just proves that people don’t understand much about technology in depth. Capability has been there already, for very long time. 99% of their work has gone for implementing that perceptual hashing function and their PSI system. If you want to give excuses, there will be always more. But they are no…

> Capability has been there already, for very long time.

Capability was always there but it wasn't implemented. Now arguments like excessive battery drain or processor usage or any argument they could have come up with can no longer be used since they went ahead and implemented software that scans iPhone files.

Perceptual hashing is a mere detail of how they are scanning files TODAY. Same for scanning only files which are to be sent to iCloud, a mere detail that can be changed anytime and surely will be requested by tyrants around the world.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#370

Post by simondotau »

Hackbraten wrote: post #298
aczerepinski wrote: post #260

Earlier quoted context omitted.

I thought the images were encrypted after the hashing was done locally. Reviewers can still view them?

Any local match causes a “safety voucher” to be uploaded along with the encrypted image. The voucher contains a (fragment of a) decryption key. If that fragment is combined with enough of its buddies from other vouchers, Apple gets to decrypt the image.

More precisely, once they have sufficient vouchers, Apple gets to decrypt the contents of the safety vouchers, which contains a low resolution, grayscale copy of the original image. Safety vouchers don't give Apple access to your photo library.
Post reply on HN