Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

171–180 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#171

Earlier quoted context omitted.

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

https://www.reuters.com/article/us-china-apple-icloud-insigh... https://support.apple.com/en-us/HT208351

Where do you think your iCloud keys are stored? Switzerland?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#172

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

The answers are probably the same. What different is how hard it is to discover.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#173
I have a suggestion for Apple. Maybe they could use this technique in a way that benefits all their customers by finally solving the problem of duplicate image imports in Photos.app. I have a couple of hundred duplicate images in my Library because of problems during local import from my iPhone into Photos.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#175
post #52

Earlier quoted context omitted.

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?

The difference is the ease with which they can demur. Before, it would be a whole heck of a lot of new, additional work. They also have the problem of actually introducing it without being noticed, or having to come up with some cover for the new behavior. Now? Well now it's real simple. It will even conveniently not expose the actual images it's checking for. Apple now has significantly less ability to rationally re…

> The difference is the ease with which they can demur.

If Apple can be cowed by China into adding fake CSAM hashes by threat of banning iPhone sales, they could be cowed to surveil Chinese citizens in the search for subversive material. It's no skin off China's back if it's harder for Apple -- they'll either make the demand or they won't. This changes basically nothing.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#176
Here's one concern -- Neural Hash is finding false positives with images that look very similar. What if an underage girl takes a selfie (or over-age girl) and the pose or background features are similar enough to trigger a false collision. Then Apple is going to a manual step where they are able to look at a low-res version of somebody's private photos as I understand it.

In my opinion that last step is not okay. I suppose the 30-image threshhold is a mitigating factor, really imo Apple is making their problem into my problem. I want to purchase from a company that offers me the peace of mind not to even have to think about such concerns, price isn't an obstacle. If Apple can't cater to my needs I hope another company will.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#177
post #98
post #91

This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2). Assuming the NCMEC database has more than 20,000 images, this represents a slightly higher rate than Apple had previously reported. But, assuming there are less than a million images in the dataset, it's probably in the right ballpark. If the author was comparing 2 trillion pictures of people, or children specifically, I think this false-posi…

The sample set is ImageNet, which is a well-known dataset in Computer Vision and is available for download here: https://www.kaggle.com/c/imagenet-object-localization-challe... I'd love to see this work extended; if you find additional collisions in the wild please submit a PR to the repo (please do not submit artificially generated adversarial images): https://github.com/roboflow-ai/neuralhash-collisions For what it…

0 out of 0.5 million vs. 3 out of 100 million does not imply with any reasonable confidence that the incidence in porn is lower.

You'd expect the same result even if the incidence in porn was 10x higher than in typical images (30 in 100 million = 0.15 in 0.5 million).

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#178

It didn’t seem to take long for the weights for Apple’s network to be discovered. And I suppose they must send the banned hashes to the client for checking too. So I expect that list will be discovered and published soon too (unless they have some way to keep them secret?) I think one important question is: how reversible is Apple’s perceptual hash? For example, my understanding of Microsoft’s PhotoDNA is that their…

The hashes sent to the client are cryptographically blinded, making it impossible for the client to determine the original CSAM hashes.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#179

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the article, are both derived without any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the false positive rate is expected to be much much higher. Imagine that you play a game of craps against an online casino…

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#180
post #128

> By taking advantage of the birthday paradox, and a collision search algorithm that let me search in n(log n) time instead of the naive n^2, I was able to compare the NeuralHashes of over 2 trillion image pairs in just a few hours. I think you could just do "sort | uniq -c | sort -nr" on the neuralhash values to find the most frequently occurring ones pretty fast?

That’s exactly the O(n log n) algorithm.
Post reply on HN