Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

451–460 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#451
post #307

I am not exactly sure if this is how this works but it appears to me that all the hashes of your photos get uploaded to a server and was wondering if it is possible to reverse the hashes to be able to deduce what's in each photo hashed..

no and no

Your answer without an explanation doesn't amount to anything more than a scant opinion. "no and no" isn't worth commenting on hackernews in my opinion. As I stated above, I am not very familiar with how this bleeding edge technology works but am not learning anything new from your response. If you wanted to expand I'd appreciate it

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#452

Earlier quoted context omitted.

This attack does seem easily defeated, even naively, by downscaling by three different means (bicubic, nearest neighbor, Lanczos, etc.) and rejecting the downscale that most differs from the other two, since the attack is tailored to a specific downscaling algorithm -- the attack seems to only be effective against systems that make no effort at all to safeguard against it. Granted, Apple makes no mention of any safeg…

The objective of being mindful of the thumbnail is to fool the human reviewer responsible for alerting the police to your target's need for a good swatting - the algorithm has already flagged the image by the time it is presented as a thumbnail during review. You'd basically start off with an image known (or very likely) to be cataloged in a CP hash database. Note its NeuralHash. Find a non-CP image that would, after…

my aim was to point out that the above reverenced "image scaling attack" is easily protected against, because it is fragile to alternate scaling methods -- it breaks if you don't use the scaling algorithm the attacker planned for, and there exist secure scaling algorithms that are immune. [0] Since defeating the image scaling attack is trivial, it means that, if it is addressed, the thumbnail will always resemble the full image.

With that out of the way, that, obviously, just forecloses this one particular attack, specifically, where you want the thumbnail to appear dramatically different than the full image in order to fool the user that it's an innocent image and the reviewer that it's an illegal image. It's still, never-the-less, possible to have a confusing thumbnail -- perhaps an adult porn image engineered to have a CSAM hash collision will be enough to convince a beleaguered or overeager reviewer to pull the trigger. The "Image Scaling Attack" is neither sufficient or necessary.

(However, that confusing image would almost certainly not also fool Apple's unspecified secondary server-side hashing algorithm, as referenced on page 13 of Apple's Security Threat Model Review, so would never be shown to a human reviewer: "as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash" [1])

[0] Understanding and Preventing Image-Scaling Attacks in Machine Learning https://www.sec.cs.tu-bs.de/pubs/2020-sec.pdf

[1] https://www.apple.com/child-safety/pdf/Security_Threat_Model...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#453

Earlier quoted context omitted.

> "The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it." This can be abused to spam Apple's manual review process, grinding it down to a halt. You've cost Apple time and money by making them review each such fake report.

> You've cost Apple time and money by making them review each such fake report. Ok, but… how do I profit? If I wanted to waste Apple employee time, I could surely find a way to do it, but why would I? The functioning of society relies on the fact that people generally have better things to do than waste each others time.

Perhaps somebody could make unflaggable CSAM images by giving them the same hashes as very widely available images.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#454

Earlier quoted context omitted.

My understanding is that these "safety vouchers" are uploaded regardless of a match. Only when there is about 30 matches are those safety vouches able to be decrypted to determine there there was a match. So Apple claims your threat model is not technically possible. Besides, Govt. can just order Apple to hand over the photos themselves from iCloud Photos because those are not end-to-end encrypted.

Which is an individual and legal process, i.e., it requires a search warrant. There are certainly problems with this process, but at a minimum, an account needs to be already and individually identified through some process (suspicion) and a legal process vetted by a judge happens (probable cause) to allow the access. That's not at all the same as proactively casting a net and starting an investigation based on the r…

Apple doesn’t need any of this to look at your photos. They can just access them on their servers because it’s not end to end encrypted.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#455

Earlier quoted context omitted.

It's entirely possible to alter an image such that its raw form looks different from its scaled form [0]. A government or just well resourced group can take a legitimate CSAM image and modify it such that when scaled for use in the perceptual algorithm(s) it changes to be some politically sensitive image. Upon review it'll look like CSAM so off it goes to reporting agencies. Because the perceptual hash algorithms are…

This attack does seem easily defeated, even naively, by downscaling by three different means (bicubic, nearest neighbor, Lanczos, etc.) and rejecting the downscale that most differs from the other two, since the attack is tailored to a specific downscaling algorithm -- the attack seems to only be effective against systems that make no effort at all to safeguard against it. Granted, Apple makes no mention of any safeg…

The attack relies on the fact that when downscaling by a large factor, the tested downscalers (except Pillow in non-nearest neighmode mode, and all of them in area averaging mode) ignore most of the pixels of the original image and compute the result based on the select few which are the same in all modes, making the result look nearly the same regardless of the mode.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#456

There's been a lot of focus on the likelihood of collisions and whether someone could upload eg; an image with a matching hash to your device to "set you up", etc. But what's still extremely concerning is that there is still no guarantee that the hash list used can't be coopted for another purpose (eg; politically insensitive content).

On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches. They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant? Someone in the court thinks it's true and can't prosecute?, oh, it got leaked . -- Not every country has the same protections about innocent until…

There would be a lot less misinformation floating around about what this technology is and isn’t if people read the documents published about how it works—-of which there are now several—-before airing “what if” scenarios that are already covered.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#457

Earlier quoted context omitted.

You can't buy a car in EU that doesn't have a sim card. All Tractors have a computer that locks out the machine if it doesn't like something and phones home. Almost every TV on sale is 'smart' and spies on what you are saying. Coffee machines, lights and toasters are now internet connected, and all of them send data to a server that will be scanning for the 'wrong' material. in 10 years there will be nowhere to hide.

> You can't buy a car in EU that doesn't have a sim card. Wait, what?

Its used for an emergency rescue system, and i believe it's mandatory in all new cars

https://ec.europa.eu/transport/themes/its/road/action_plan/e...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#458
post #393
post #308

Earlier quoted context omitted.

>We don't care that any two images have the same hash, Why not?

Based on the pigeonhole principle alone, it will always be the case that collisions exist. The size of the digest is very likely smaller than the size of any given image.

How is that relevant to whether “per image” or “Per image pair” is the right metric?

Heck, how is it relevant to anything given that hash functions avoid collisions in practice, even given the pigeonhole principle?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#459

Earlier quoted context omitted.

You are one underpaid random guy in India looking at CSAM all day clicking the wrong button away from a raid of your home and the end of your life as you know it.

This, these charges are damning once they are made. Plus the countless legal dollars you are going to have to front and hours spent proving innocence and that's assuming the justice system actually works.. Try explain this to your employer while you start missing deadlines due to court dates.. The police also could easily leverage this to warrant hop. As they have been found doing in the past. I think the bike rider…

Try explaining to your employer that your employer's laptop was seized because the police took all of your computers and devices when arresting you.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#460
post #455

Earlier quoted context omitted.

This attack does seem easily defeated, even naively, by downscaling by three different means (bicubic, nearest neighbor, Lanczos, etc.) and rejecting the downscale that most differs from the other two, since the attack is tailored to a specific downscaling algorithm -- the attack seems to only be effective against systems that make no effort at all to safeguard against it. Granted, Apple makes no mention of any safeg…

The attack relies on the fact that when downscaling by a large factor, the tested downscalers (except Pillow in non-nearest neighmode mode, and all of them in area averaging mode) ignore most of the pixels of the original image and compute the result based on the select few which are the same in all modes, making the result look nearly the same regardless of the mode.

Thanks for that reference to Pillow. I presume it's from "Understanding and Preventing Image-Scaling Attacks in Machine Learning" [0] which mentions secure scaling algorithms immune to the attack. I wish I could mention this in the grand parent, but the editing window closed.

[0] https://www.sec.cs.tu-bs.de/pubs/2020-sec.pdf

Post reply on HN