ImageNet contains naturally occurring Apple NeuralHash collisions

Post by dont__panic » Thu, Aug 19, 2021, 6:23 PM UTC

breuleux wrote: post #67

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Presumably Apple would be afraid that, say, the EU becomes suspicious, issues a court order to obtain the hashes, notices they cannot audit the CCP hashes, pointedly asks "what is this", becomes absolutely livid that their citizens are spied on by a country that is not them, fines Apple out the wazoo, then extradites whoever is responsible and puts them in prison. I mean, China's not the only player in this. Putting…

Post by slg » Thu, Aug 19, 2021, 6:25 PM UTC

yeldarb wrote: post #92

slg wrote: post #48

Earlier quoted context omitted.

>This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2). Assuming the NCMEC database has more than 20,000 images, this represents a slightly higher rate than Apple had previously reported. But, assuming there are less than a million images in the dataset, it's probably in the right ballpark. Apple reported a pretty similar collision rate so maybe they did.

Their reported collision rate was against their CSAM hash database[1]. > In Apple’s tests against 100 million non-CSAM images, it encountered 3 false positives when compared against NCMEC’s database. In a separate test of 500,000 adult pornography images matched against NCMEC’s database, it found no false positives. [1] https://tidbits.com/2021/08/13/new-csam-detection-details-em...

Post by sennight » Thu, Aug 19, 2021, 6:26 PM UTC

endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

Post by stjohnswarts » Thu, Aug 19, 2021, 6:27 PM UTC

shuckles wrote: post #36

mikepurvis wrote: post #22

Earlier quoted context omitted.

At least part of the concern is that a hash collision is basically "cause" for Apple to then dump and begin manually (like, with humans) reviewing the contents of your device, all of which will be happening behind the closed doors of a private corporation, outside of any of the usual oversight or innocent-presumption mechanisms that come from it happening through the courts. That, combined with a (pretty reasonable)…

That is literally the status quo with every cloud service. Apple, unlike the others, has said that they will evaluate you on the basis of what’s included in the associated data of your safety voucher, and you can inspect those contents because they’re shipped in the client. Facebook, for all I know, might be calculating a child predator likelihood score on my account based on how often I look up my middle school ex-g…

Post by stjohnswarts » Thu, Aug 19, 2021, 6:29 PM UTC

endisneigh wrote: post #85

foerbert wrote: post #79

Earlier quoted context omitted.

I don't think we can just appeal to the status quo here and assume it's acceptable. There's a couple reasons. First, how many people really understood this previously? Did society at large actually knowingly accept the current state of things, or did it just happen without most people realizing it? Even here on HN where we'd expect to find people way more knowledgeable about it than in general I'm not sure how well k…

> But more philosophically, it's your own device being turned against you to check you for criminal behavior. That's very different from somebody else checking up on you after you willingly interact with them. This literally only works once you willing send photos to iCloud.