Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

391–400 of 725 posts

Re: Hash collision in Apple NeuralHash model

#391

Post by zepto »

avianlyric wrote: post #119
FabHK wrote: post #111

Earlier quoted context omitted.

Depends very much on the process Apple uses to make the "visual derivative", though. Also, defence by producing the original innocuous image (and showing that it triggers both parts of Apple's process, NeuralHash and human review of the visual derivative) should be possible, though a lot of damage might've been done by then.

> Also, defence by producing the original innocuous image At this point you’re already inside the guts of the justice system, and have been accused of distributing CSAM. Indeed depending on how diligent the prosecutor is, you might need to wait till trial before you can defend yourself. At that point you’re life as you know is already fucked. The only thing proving your innocence (and the need to do so is itself a co…

This isn’t true at all.

If the creation of fakes is as easy as claimed, Neuralhash evidence alone will become inadmissible.

There are plenty of lawyers and money waiting to establish this.

Re: Hash collision in Apple NeuralHash model

#392

Post by HelloNurse »

snowwrestler wrote: post #351
ethbr0 wrote: post #248

Earlier quoted context omitted.

That's an incomplete statement. Currently, they must comply with warranty requests by scanning if they have the ability to scan . If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability. And that's what pisses people off about this.

If Apple launches a system for comparing iCloud uploads to a third-party hash list, then adding the ability to do targeted scans for arbitrary additional law-enforcement-provided hashes would also be a form of creating capability. The people getting pissed off about this have not, so far as I’ve seen, demonstrated why the law would require Apple to add the capability for targeted scans of arbitrary hashes. Police can…

> why the law would require Apple to add the capability for ...

This isn't the sort of thing that is mandated by law, but rather requested behind the scenes by espionage, er, "law enforcement" agencies. They might be more or less friendly deals; nice monopoly you have here, it would be a shame if something happened to it...

Re: Hash collision in Apple NeuralHash model

#393

Post by mrits »

FabHK wrote: post #41
spicybright wrote: post #31

Earlier quoted context omitted.

If you're in close physical contact with a person (like at a job) you just wait for them to put their phone down while unlocked, and do all this.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

"Then, with all due respect, the attacker could just download actual CSAM."

If you didn't have Apple scanning your drive trying to find a new way for you to go to prison then it wouldn't be a problem.

Re: Hash collision in Apple NeuralHash model

#394

Post by zepto »

GeckoEidechse wrote: post #273
FabHK wrote: post #78

Earlier quoted context omitted.

Right. So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images). But a conceivable novel avenue of attack would be to find an image that: 1. Does not look like CSAM to the innocent victim in the original 2. Does match known CSAM by NeuralHash 3. Does look like CSAM in the "visual derivative" reviewed by Apple, as you highlight.

> So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images). What if they are placed on the iDevice covertly? Say you want to remove politician X from office. If you got the money or influence you could use a tool like Pegasus (or whatever else there is out there that we don't know of) to place actual CSAM images on their iDevice. Preferably with…

If you are placing images covertly, you can just use real CSAM or other compromat.

Re: Hash collision in Apple NeuralHash model

#395

Post by newArray »

Hash collisions happen by design in the perceptual hash, its supposed to give equal hashes for small changes after all.

Something I find interesting is the necessary consequences of the property of small edits resulting in the same hash. We can show that this is impossible to absolutely achieve, or in other words there must exist an image such that changing a single pixel will change the hash.

Proof: Start with 2 images, A and B, of equal dimension, and with different perceptual hashes h(A) and h(B). Transform one pixel of A into the corresponding pixel of B and recompute h(A). At some point, after a single pixel change, h(A) = h(B), this is guaranteed to happen before or at A = B. Now A and the previous version of A have are 1 pixel apart, but have different hashes. QED

We can also ATTEMPT to create an image A with a specified hash matching h(A_initial) but which is visually similar to a target image B. Again start with A and B, different images with same dimensions. Transform a random pixel of A towards a pixel of B, but discard the change if h(A) changes from h(A_initial). Since we have so many degrees of freedom for our edit at any point (each channel of each pixel) and the perceptual hash invariant is in our favor, it may be possible to maneuver A close enough to B to fool a person, and keep h(A) = h(A_initial).

If this is possible one could transform a given CSAM image into a harmless meme while not changing the hash, spread the crafted image, and get tons of iCloud accounts flagged.

Re: Hash collision in Apple NeuralHash model

#396

Post by varispeed »

mukesh610 wrote: post #217
varispeed wrote: post #175

Earlier quoted context omitted.

> that is clearly not For one, you can't know if that's true as the image could have been manipulated to appear as such. For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image. > but I don't understand how they can get it on someone's phone There is many vectors. For example you can leave phone unattended and someone can snap a pic…

> For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image. How would NeuralHash pick a "hidden image"? It only uses the pixels of the image to get the hash. Any hidden image in the metadata would not even be picked up and no amount of steganography can fool NeuralHash. > There is many vectors. For example you can leave phone unatten…

The steganography is not to fool the NeuralHash, but to fool the viewer.

The NeuralHash would "see" the planted image, but for the viewer it would appear innocent.

I am trying to say that a person reviewing image manually, without special tools will not be able to tell if the image is a false positive and would have to report everything.

Re: Hash collision in Apple NeuralHash model

#397

Post by mannerheim »

falcolas wrote: post #380
mannerheim wrote: post #378

Earlier quoted context omitted.

I believe the hash comparisons are made on Apple's end. Then the only way to get hashes will be a data breach on Apple's end (unlikely but not impossible) or generating it from known CSAM material.

That's not what Apple's plans state. The comparisons are done on phone, and are only escalated to Apple if there are more than N hash matches, at which point they are supposedly reviewed by Apple employees/contractors. Otherwise, they'd just keep doing it on the material that's actually uploaded.

Ah, never mind, you're right:

> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Hash collision in Apple NeuralHash model

#398

Post by a-dub »

it gets even more fun when you can generate images that are imperceivably different to the human eye, but produce different (and controlled) hash values. (see shazam decoys and adversarial images). not sure if collision is really the right term here though as similarity hashing intentionally hashes multiple images to the same hash.

in other news, my google phone just helpfully scanned all client side images, rounded up everything it thought was a meme and suggested i delete them to save space. i wonder if that feature has telemetry built in...

Re: Hash collision in Apple NeuralHash model

#399

Post by robertoandred »

falcolas wrote: post #369
robertoandred wrote: post #360

Earlier quoted context omitted.

You cannot extract or reverse the CSAM hashes. They've been encrypted and blinded using server-side-only keys. If TFA said that, it's lying.

One does not need to reverse the CSAM hashes to find a collision with a hash. If the evaluation is being done on the phone, including identifying a hash match, the hashes must also be on the phone.

No, matches are not verified on the phone. On the phone, your image hash is used to look up an encrypted/blinded (via the server's secret key) CSAM hash. Then your image data (the hash and visual derivative) is encrypted with that encrypted/blinded hash. This encrypted payload, along with a part of your image's hash, is sent to Apple. Then on the server, Apple uses that part of your image's hash and their secret key to create a decryption key for the payload. If your image hash matches the CSAM hash, the decryption key would unlock the payload.

In addition, they payload is protected at another layer by your user key. Only with enough mash matches can Apple put together the user decryption key and open the very innards of your image's payload containing the full hash and visual derivative.

Re: Hash collision in Apple NeuralHash model

#400

Post by mannerheim »

robertoandred wrote: post #335
mannerheim wrote: post #184

Earlier quoted context omitted.

You don't need to do that, just use images that collide with the hashes.

How will you know something collides?

> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Post reply on HN