Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

651–660 of 725 posts

Re: Hash collision in Apple NeuralHash model

#651

Post by hannasanarion »

nullc wrote: post #631
hannasanarion wrote: post #543

Earlier quoted context omitted.

The other thread demonstrates a similar attack with pictures of dogs. The same questions still apply. Why would you save tons of pictures of dogs to your photo library? Why would pictures of dogs look like child porn? Why would Apple reviewers think pictures of dogs are child porn? Why would the NCMEC think pictures of dogs are child porn? Why would law enforcement spend time arresting someone for pictures of dogs? W…

I'm really surprised to see a fellow HN poster making such a stark failure to generalize. You see that they can do this with pictures of dogs. What makes you think they can't do exactly the same with pictures of crotches? Presumably you don't believe there is some kind inherent dog-nature that makes dog images more likely to undermine the hash. :) People are using pictures of dogs because they are a tasteful safe-for…

What kind of non-child-porn imagery is so similar to child porn that it fools the NCMEC investigator who is looking at it side-by-side with its maliciously-collisioned hash match, but is at the same time so so dissimilar from real child porn that, as you say, the target won't report its sudden appearance on their device and the the sender won't be at legal risk? Your scenario requires the image (not the hash, but the regular human-perceptible image) to be both unremarkably innocent and incontrovertibly damning at the same time.

Re: Hash collision in Apple NeuralHash model

#652

Post by floatingatoll »

neolog wrote: post #649
floatingatoll wrote: post #646

Earlier quoted context omitted.

Apple already has technology deployed to perform binary file scans of every file on macOS and iOS, and the ability to at any time release signatures for those scans, that are very difficult for normal users to prevent updates for. They've had that for years, maybe even a decade by now, and so far to date we have seen no abuse of that list. How is Apple's new CSAM list somehow increasing the chances of Apple going rog…

What technology are you referring to as already deployed?

For macOS, I'm talking about XProtect and MRT. I don't know the exact subsystem names on iOS, apologies.

https://support.apple.com/guide/security/protecting-against-...

Each system is closed source, provides a mechanism for checking content signatures against files on disk, and is thought to report telemetry to Apple when signatures are found.

How is CSAM scanning new and different from those existing closed-source systems?

Re: Hash collision in Apple NeuralHash model

#653

Post by brandon272 »

treis wrote: post #547
brandon272 wrote: post #501

Earlier quoted context omitted.

This is not at all conceptually the same. FedEx was not held liable simply because their service was used to mail illegal drugs. They were held liable because not only did they know about the specific instances in which it was mailed, they allegedly conspired with the shippers to facilitate the mailings. They were knowingly mailing packages to parking lots where drug dealers would wait to pick them up: > According to…

>They were held liable because not only did they know about the specific instances in which it was mailed, they allegedly conspired with the shippers to facilitate the mailings. Apple knows that CSAM is being sent and conspires to do so (i.e. transmits the image). Conceptually they are the same. The rest of your post details the practical differences between sending physical packages and digital images.

I'm not even sure where to start here.

FedEx knows that CSAM is sent using its services in the same way that Apple does. That is to say that both companies know that CSAM has been sent historically and both know that is possible to send that type of material using its services, and one could argue that they should assume that their services are used to transfer that data.

Why does one of these companies have to go to such great depths to search out and report these materials and the other does not? If we suggest that Apple "knows" about CSAM on their network, we must also accept that Apple "knows" about many other crimes in which the devices they are sell are used in the planning and execution of those crimes. Why are they only focused on CSAM if they'd also have legal exposure in these other crimes simply for being a hardware and service provider?

Lastly, to suggest that Apple "conspires" to transmit this content is wholly inaccurate. Conspiracy implies two or more parties are in explicit agreement to commit a crime.

Re: Hash collision in Apple NeuralHash model

#654

Post by neolog »

floatingatoll wrote: post #652
neolog wrote: post #649

Earlier quoted context omitted.

What technology are you referring to as already deployed?

For macOS, I'm talking about XProtect and MRT. I don't know the exact subsystem names on iOS, apologies. https://support.apple.com/guide/security/protecting-against-... Each system is closed source, provides a mechanism for checking content signatures against files on disk, and is thought to report telemetry to Apple when signatures are found. How is CSAM scanning new and different from those existing closed-source s…

I'd say the primary differences are that the CSAM scan is a perceptual hash rather than a regular file hash, and that the technical infrastructure of the CSAM system is designed from the ground up to be used against (rather than for) the user and report them individually to authorities for violation.

Re: Hash collision in Apple NeuralHash model

#655

Post by floatingatoll »

neolog wrote: post #654
floatingatoll wrote: post #652

Earlier quoted context omitted.

For macOS, I'm talking about XProtect and MRT. I don't know the exact subsystem names on iOS, apologies. https://support.apple.com/guide/security/protecting-against-... Each system is closed source, provides a mechanism for checking content signatures against files on disk, and is thought to report telemetry to Apple when signatures are found. How is CSAM scanning new and different from those existing closed-source s…

I'd say the primary differences are that the CSAM scan is a perceptual hash rather than a regular file hash, and that the technical infrastructure of the CSAM system is designed from the ground up to be used against (rather than for) the user and report them individually to authorities for violation.

Do you have an alternate design in mind that is both "used for the user", and is also effective at reporting CSAM content being uploaded from the device, without allowing CSAM abusers to opt-out of that reporting? I haven't been able to come up with anything myself, but maybe you've had better luck.

Re: Hash collision in Apple NeuralHash model

#656

Post by treis »

brandon272 wrote: post #653
treis wrote: post #547

Earlier quoted context omitted.

>They were held liable because not only did they know about the specific instances in which it was mailed, they allegedly conspired with the shippers to facilitate the mailings. Apple knows that CSAM is being sent and conspires to do so (i.e. transmits the image). Conceptually they are the same. The rest of your post details the practical differences between sending physical packages and digital images.

I'm not even sure where to start here. FedEx knows that CSAM is sent using its services in the same way that Apple does. That is to say that both companies know that CSAM has been sent historically and both know that is possible to send that type of material using its services, and one could argue that they should assume that their services are used to transfer that data. Why does one of these companies have to go to…

>If we suggest that Apple "knows" about CSAM on their network, we must also accept that Apple "knows" about many other crimes in which the devices they are sell are used in the planning and execution of those crimes. Why are they only focused on CSAM if they'd also have legal exposure in these other crimes simply for being a hardware and service provider?

It's not knowledge of a crime. Apple is committing a crime by possessing and distributing CSAM.

>Lastly, to suggest that Apple "conspires" to transmit this content is wholly inaccurate. Conspiracy implies two or more parties are in explicit agreement to commit a crime.

But that is exactly what they're doing. They are agreeing to possess and deliver images of which they know a portion are CSAM.

Re: Hash collision in Apple NeuralHash model

#657

Post by eightysixfour »

sierpinsky wrote: post #529

"According to media reports, the cloud computing industry does not take full advantage of the existing CSAM screening toolsto detect images or videos in cloud computing storage. For instance, big industry players, such as Apple, do not scan their cloud storage. In 2019, Amazon provided only eight reports to the NCMEC, despite handling cloud storage services with millions of uploads and downloads every second. Others,…

It is a shift in trust. If my things are scanned on my local device, I now must trust that Apple:

* Will not be compelled to change the list of targeted material by government coercion.

* Will not upload the "vouchers" unless the material is uploaded to iCloud.

* Will implement these things in such a way that hackers cannot maliciously cause someone to be wrongly implicated in a crime.

* Will implement these things in such a way that hackers cannot use the tools Apple has created to seek other information (such as state sponsored hacking groups looking for political dissidents).

And for many of us, we do not believe that these things are a given. Here's a fictional scenario to help bring it home:

Let's say a device is created that can, with decent accuracy, detect drugs in the air. Storage unit rental companies start to install them in all of their storage units to reduce the risk that they are storing illegal substances, and they notify the police if the sensors go off.

One storage unit rental company feels like this is an invasion of privacy, so they install the device in your house but promise to only check the results if you move your things into the storage unit.

This sounds crazy, right?

Re: Hash collision in Apple NeuralHash model

#658

Post by shapefrog »

icelancer wrote: post #650
shapefrog wrote: post #122

Earlier quoted context omitted.

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

> I can guarantee nobody will see the inside of a courtroom This wasn't the question I asked.

> grey noise will never pass for CSAM > You sure that one or more human operators will never make this mistake.

Yes I can say 100% that no human operator will ever classify a grey image for a child being raped. Happy to put money on it.

Re: Hash collision in Apple NeuralHash model

#659

Post by Grustaf »

fsflover wrote: post #471
Grustaf wrote: post #233

Earlier quoted context omitted.

Well of course if you believe that FBI can ask Apple to share anything and they will agree, and not tell us, then what is even the point of this discussion? Then they already have access to every photo, every email, every text. Because Apple controls those apps on your phone.

The difference is that now they confirmed that they do search your private images.

Why would that make it any likelier that they are also secretly searching your files, with another method? Seems completely orthogonal to me.

Re: Hash collision in Apple NeuralHash model

#660

Post by cirrus3 »

Kalium wrote: post #194
topynate wrote: post #185

Second preimage attacks are trivial because of how the algorithm works. The image goes through a neural network (one to which everyone has access), the output vector is put through a linear transformation, and that vector is binarized, then cryptographically hashed. It's trivial to perturb any image you might wish so as to be close to the original output vector. This will result in it having the same binarization, he…

A police raid on a person's home, or even a gentler thorough search, can be enough to quite seriously disrupt a person's life. Certainly having the police walk away with all your electronics in evidence bags will complicate trying to work remotely. Of course, this is assuming everything works as intended and they don't find anything else they can use to charge you with something as they search your home. If you smoke…

The police do not show up until a human has compared the matched image to the actual.

Just stop.

Post reply on HN