Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

31–40 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#31

Post by rm999 »

tldr: this is expected. The article addresses everything I'm about to say, but I think the lede is buried:

>Apple's NeuralHash perceptual hash function performs its job better than I expected

When you are just looking for any two collisions in 100M x 100M comparisons, of course you'll find a small number of positives, Apple said as much. The number of expected collisions will scale linearly with the number of 'bad' images, which is not 100M. Assuming it's 100k, we'd expect 1000x fewer collisions in ImageNet, or ~0.002 collisions, which is effectively 0. It's the artificial images that will potentially sink all this, not a very low rate of naturally occurring hash collisions.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#32

Post by rootusrootus »

minitoar wrote: post #11
endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

The damage is already done by the time it gets to the point of devices being confiscated.

By the time the FBI comes knocking for your devices, they have a lot of evidence, not a list of hash collisions.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#33

Post by stickfigure »

endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

Let's say I get you to click on a link. That link contains a thumbnail gallery of CSAM. With the right CSS, you might not even notice it, but it's in your browser's cache and on your filesystem. Lots of pictures - more than enough for your phone to snitch on you.

All because you clicked on a link.

Phishing attacks can now put you in prison, label you as a pedophile and sex offender, and destroy your life.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#34

Post by biesnecker »

> This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2). Assuming the NCMEC database has more than 20,000 images, this represents a slightly higher rate than Apple had previously reported. But, assuming there are less than a million images in the dataset, it's probably in the right ballpark.

The number of images in that database could well be far in excess of a million. According to NCMEC [1], in 2020 65.4 million files that were reported to them, and "[s]ince the program inception in 2002, CVIP [child victim identification project] has reviewed more than 330 million images and videos."

Of course many of those were duplicate but it would be entirely unsurprised if there were more than a million original files.

[1] https://www.missingkids.org/ourwork/impact

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#35

Post by endisneigh »

stickfigure wrote: post #33
endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

Let's say I get you to click on a link. That link contains a thumbnail gallery of CSAM. With the right CSS, you might not even notice it, but it's in your browser's cache and on your filesystem. Lots of pictures - more than enough for your phone to snitch on you. All because you clicked on a link. Phishing attacks can now put you in prison, label you as a pedophile and sex offender, and destroy your life.

This is already possible with PhotoDNA. When has it happened?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#36

Post by shuckles »

mikepurvis wrote: post #22
endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

At least part of the concern is that a hash collision is basically "cause" for Apple to then dump and begin manually (like, with humans) reviewing the contents of your device, all of which will be happening behind the closed doors of a private corporation, outside of any of the usual oversight or innocent-presumption mechanisms that come from it happening through the courts. That, combined with a (pretty reasonable)…

That is literally the status quo with every cloud service. Apple, unlike the others, has said that they will evaluate you on the basis of what’s included in the associated data of your safety voucher, and you can inspect those contents because they’re shipped in the client. Facebook, for all I know, might be calculating a child predator likelihood score on my account based on how often I look up my middle school ex-girlfriend on Instagram.

In addition, “pretty reasonable” is an opinion not fact. Where is the evidence that PhotoDNA hashes have been compromised in this way in the fifteen years they’ve been used?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#37

Post by biesnecker »

endisneigh wrote: post #26
zbobet2012 wrote: post #16

Earlier quoted context omitted.

That's not the route in which this will be exploited, and used at scale. A corrupt government has to know they want you "to just get you". Instead they will embed a collision in anti-government meme. That collision will flag you, and now they know you harbor doubts and will come get you. This is why it's a privacy concern. It's no the tech (like you said photo dna's been about forver), it's the scanning of the phone.

What you’re saying is already possible. Where are the examples of this happening? PhotoDNA has existed since 2008.

Who is scanning your phone right now with PhotoDNA?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#38

Post by rootusrootus »

pizzaknife wrote: post #21

the world of forensics is something im not familiar with, a couple questions: - When a hash matches (correctly or incorrectly), how is said image reported? is the matching image passed on to a human to verify or? - what is the survey size / who(m) are subject to this CSAM net?

after ~30 matches, only the matching images are passed on to a human for visual verification. only images uploaded to iCloud are subject to matching

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#39

Post by endisneigh »

biesnecker wrote: post #37
endisneigh wrote: post #26

Earlier quoted context omitted.

What you’re saying is already possible. Where are the examples of this happening? PhotoDNA has existed since 2008.

Who is scanning your phone right now with PhotoDNA?

Microsoft, Facebook and Google. More depending on what services you use

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#40

Post by Dah00n »

endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

Lots of things start out like this, then later they start looking for or blocking other things. Pirated movies, files leaked from three letter agencies, a picture of the president with a ** in his mouth, etc. Even if this is 100% bullet-proof it is still enough that Apple should be seen as privacy invading and leaking everything to the government (and others later on) as it will be abused if implemented everywhere. This isn't irrelevant because of other things happening that are also bad. This is added on-top of those broken system, like the ones in the US you mention.
Post reply on HN