Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A catalog of naturally occurring images whose Apple NeuralHash is identical

github.com

Post reply on HN
Print view View original

71–80 of 304 posts

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#71

Post by scotty79 »

Why are exact collisions interesting? They are not intended to be compared exactly.

This algorithm doesn't even give exact matches for the same image on different hardware.

https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX

Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason is that neural networks are based on floating-point calculations. The accuracy is highly dependent on the hardware. For smaller networks it won't make any difference. But NeuralHash has 200+ layers, resulting in significant cumulative errors.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#72

Post by koolhaas »

FabHK wrote: post #68
koolhaas wrote: post #61

Earlier quoted context omitted.

Yes I agree, bit of a stretch. Based on their whitepaper, it's a smaller version of the original image, I guess just large enough to support the human verification step. But I'm unsure that the thumbnail is included with every CSAM "voucher" -- it's likely only included when you pass the 30 image limit. Need to read that section more clearly.

> Based on their whitepaper, it's a smaller version of the original image, I seem to recall that the white paper speaks of a "visual derivative" without specifying it further.

The Technical Summary uses "visual derivative" without clarification, but their Threat Model PDF clarifies it further as thumbnails:

>The decrypted vouchers allow Apple servers to access a visual derivative – such as a low-resolution version – of each matching image.

https://www.apple.com/child-safety/pdf/Security_Threat_Model...

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#73

Post by yeldarb »

scotty79 wrote: post #71

Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on different hardware. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason…

That is a good point; has Apple stated how many bits two images’ NeuralHashes can differ by and still be considered a “match” by their system?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#74

Post by AnonC »

A very relevant point on this entire discourse about Apple’s on-device CSAM scanning:

According to the U.S. law, key snippets of which are quoted on the Stratechery blog (by Ben Thompson), Apple isn’t obligated to scan for CSAM. It’s only obligated to act on CSAM if it finds them.

While it’s good for Apple to scan on its systems (iCloud) like Facebook, Google and other companies do on their servers, it’s inappropriate to do it on individual devices, which starts with the assumption that anyone who has iCloud photos enabled is a potential CSAM hoarder and needs to pay with their device’s battery life and time for the scanning to happen and report back. It’s a sort of micro-robbery that Apple is doing on the devices when there is no legal compulsion to do so.

Everything else on trusting Apple’s NeuralHash or the sanctity of the NCMEC hashes come later, IMO.

I sincerely hope Apple realizes that it’s got a dud solution on hand, eats humble pie (which it’s usually not capable of) and ditches this whole thing. I know a lot of egos at Apple are at stake here. But doing the right thing matters for a company that claims that “privacy is a fundamental human right” and has a CEO who’s a member of a marginalized/discriminated community and understands the risks of these efforts.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#75

Post by StrLght »

I don't really get what this repository is trying to achieve and what's the point of collecting collisions. Collisions will happen, that's just how it is with hashes.

It's already a public knowledge that Apple has 2 more systems (some server-side verification and a manual check later) to prevent false-positives. So what's the point of researching collisions in NeuralHash?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#77

Post by SiempreViernes »

saithound wrote: post #59
slownews45 wrote: post #28

"This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2)" That is not bad. As a tool to filter down what apple human reviewers need to look at this is pretty good. Ultimately these images will make it to a human reviewer who can make a call as they would in any flagging system. Could a backend server side system do a more precise hash (96 bits is not a ton) prior to human review?

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the blog post linked from the repo [2], are both derived without making any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the practical false positive rate is now expected to be much much higher. Imagine that y…

Yeah, of course the collision rate in an adversarial dataset is likely to be much higher.

But I really wonder why you think this is an important objection, do you think a lot of people want to go to the "get flagged for child porn" casino?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#78

Post by simondotau »

nannal wrote: post #41
eesmith wrote: post #18

Earlier quoted context omitted.

A catalog of a thousand pages begins with the first entry.

And a story may start with the first word, but if I present the word "Octopus" and say check out my story, you're going to be well within bounds to question me on it.

Well to be fair there were two collisions, so your story should be "Octopus imploded" and now everyone is captivated by the plight of this unlucky cephalopod.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#79

Post by saynay »

saithound wrote: post #59
slownews45 wrote: post #28

"This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2)" That is not bad. As a tool to filter down what apple human reviewers need to look at this is pretty good. Ultimately these images will make it to a human reviewer who can make a call as they would in any flagging system. Could a backend server side system do a more precise hash (96 bits is not a ton) prior to human review?

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the blog post linked from the repo [2], are both derived without making any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the practical false positive rate is now expected to be much much higher. Imagine that y…

This argument is utterly incoherent. Of course they don't include into the false positive rate images that are intentionally designed to generate positive hits, what would be the point of that? The only interesting metric is the false positive rate for normal images.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#80

Post by woofie11 »

StrLght wrote: post #75

I don't really get what this repository is trying to achieve and what's the point of collecting collisions. Collisions will happen, that's just how it is with hashes. It's already a public knowledge that Apple has 2 more systems (some server-side verification and a manual check later) to prevent false-positives. So what's the point of researching collisions in NeuralHash?

No. Most proper cryptographic hash systems (e.g. used for verifying files, rather than data structures) never have collisions.

Try to find a SHA256 collision.

Anywhere, ever, in the history of mankind.

This isn't for lack of looking. A lot of very smart people have looked for them. If you find one, I bet you'll be eligible for a tenured faculty slot at a good university, if not more. A whole world of secure systems would need to be re-engineered.

Hypothetical collisions of course exist, by the pigeonhole principle, just not in the real world.

Post reply on HN