Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

181–190 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#181

Post by slg »

stjohnswarts wrote: post #127
slg wrote: post #113

Earlier quoted context omitted.

>But how long before it scans everything, and there's no way to opt out? Do we think this is detectable? If yes, then why worry about it if we will know when this switch is made? If not, why did we trust Apple that this wasn't happening already? That is the primary thing I don't understand, this fear rests on an assumption that Apple is a combination of both honest and corrupted. If they are honest, we have no reason…

You're viewing trust in apple as a binary choice. It is not. Trust is a spectrum like most things. You need to get away from that digital thinking. It's the whole reason we have to challenge government and be suspicious of it. It's the same with companies.

I view trust more as a collection of binary choices than one single spectrum. Do I trust Apple to do X? There is only two possible answers to that (or I guess three if we include "I don't know"). If the answer isn't binary, then X is too big.

In this instance the specific question is "Do I trust Apple to be honest about when they scan our files?". I don't know why this news would change the answer to that question.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#182

Post by mrtranscendence »

tgsovlerkhgsel wrote: post #141

> Apple now has over 1.5 billion users so we are talking about a large pool of users at stake which increases the likelihood of even a low probability event manifesting This is an extremely good point. If the whole system, end to end, after all safeguards (e.g. human reviewers which can also make mistakes) has a one-in-a-billion chance to ruin a user's life, then statistically, we can expect 1-2 users to have their l…

> This is an extremely good point. If the whole system, end to end, after all safeguards (e.g. human reviewers which can also make mistakes) has a one-in-a-billion chance to ruin a user's life, then statistically, we can expect 1-2 users to have their lives ruined.

No one will have their lives ruined. If there's a false collision, someone at Apple has to look at the images as a final safeguard. If it's not actually CSAM then it won't ever make it to law enforcement.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#183

Post by ipaddr »

endisneigh wrote: post #39
biesnecker wrote: post #37

Earlier quoted context omitted.

Who is scanning your phone right now with PhotoDNA?

Microsoft, Facebook and Google. More depending on what services you use

None of those are scannig your phone. None of them aside from google even sell a phone.

Everyone suspected they were scanning images for this and a number of other things on their services.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#184

Post by zug_zug »

endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

And what if the FBI demands a list of all Apple users who have matched even 1 CSAM photo for their own private watchlist?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#185

Post by slg »

sigmar wrote: post #143
slg wrote: post #113

Earlier quoted context omitted.

>But how long before it scans everything, and there's no way to opt out? Do we think this is detectable? If yes, then why worry about it if we will know when this switch is made? If not, why did we trust Apple that this wasn't happening already? That is the primary thing I don't understand, this fear rests on an assumption that Apple is a combination of both honest and corrupted. If they are honest, we have no reason…

Are we going to need to reverse engineer every single Apple update to make sure the feature hasn't creeped into non-iCloud uses? Is the inevitable Samsung version of this system going to be as privacy-preserving? How are we sure the hash list isn't tainted? All of these problems are solved by one principle: Don't put the backdoor code in the OS to begin with.

>Are we going to need to reverse engineer every single Apple update to make sure the feature hasn't creeped into non-iCloud uses?

Did we do this already? If not, why did we assume this didn't exist previously?

>Is the inevitable Samsung version of this system going to be as privacy-preserving?

I don't see how this is Apple's fault. Be angry at Samsung for their system. I don't blame Snap if I have a problem with Instagram Stories.

>How are we sure the hash list isn't tainted?

How did we know the hash list wasn't tainted when doing this comparison on Apple's servers?

>All of these problems are solved by one principle: Don't put the backdoor code in the OS to begin with.

Apple controls the hardware, software, and cloud services. The backdoor didn't need to exist in the OS for there to be a backdoor.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#186

Post by cwkoss »

endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

> Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America?

I believe a single image flag could be used as pretext to arrest someone that the government has already deemed an enemy of the state. Ex. Someone like Julian Assange

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#187

Post by fallingknife »

yeldarb wrote: post #98
smithza wrote: post #91

This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2). Assuming the NCMEC database has more than 20,000 images, this represents a slightly higher rate than Apple had previously reported. But, assuming there are less than a million images in the dataset, it's probably in the right ballpark. If the author was comparing 2 trillion pictures of people, or children specifically, I think this false-posi…

The sample set is ImageNet, which is a well-known dataset in Computer Vision and is available for download here: https://www.kaggle.com/c/imagenet-object-localization-challe... I'd love to see this work extended; if you find additional collisions in the wild please submit a PR to the repo (please do not submit artificially generated adversarial images): https://github.com/roboflow-ai/neuralhash-collisions For what it…

500K is not a large enough dataset to determine that. The collision rate could plausibly be 1 in 500K (or even a bit higher) and have no collisions in the sample.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#188

Post by [deleted] »

tines wrote: post #60
slg wrote: post #52

Earlier quoted context omitted.

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?

Tin-foil hat time: Who's to say that they could stand up to them before this system? The system itself could have been proposed by the CCP in the first place. I'll take my hat off now.

[deleted]

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#189

Post by knodi123 »

WesolyKubeczek wrote: post #76
endisneigh wrote: post #43

Earlier quoted context omitted.

That’s the same as what Apple is going to do - scan right before sending to iCloud

The same as what Apple is saying it's going to do , there's a difference.

Man, wouldn't you love to be the developer who gets assigned the feature to commit these horrible secret privacy violations with deeply evil ethical problems?

You don't even have to implement the feature. All you need is really good proof that you were asked to, and now your job at Apple is secure, along with a huge raise, for years if not for life. Something commensurate with what you and they both know they would lose in the PR damage and possible EU fines.

Post reply on HN