Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

321–330 of 725 posts

Re: Hash collision in Apple NeuralHash model

#321

Post by rootusrootus »

notRobot wrote: post #162
shapefrog wrote: post #122

Earlier quoted context omitted.

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

Many people never see the inside of a courtroom when false or unproven rape accusations are made against them, but their lives still get ruined because of the negative publicity.

We give way, way too much weight in the legal system to eye witness and victim statements, in absence of any corroborating evidence. That's a problem.

But not really comparable, IMO. You won't even know you got investigated until after the original images have been shipped off to NCMEC for verification.

Re: Hash collision in Apple NeuralHash model

#322

Post by UncleMeat »

grishka wrote: post #292
UncleMeat wrote: post #109

Why is this meaningfully different than, say, what Google Photos has been doing for years? If you can get rooting malware on the target device then you could 1. Produce actual CSAM rather than a hash collision 2. Produce lots of it 3. Sync it with Google Photos This attack has been available for many years and does not need convoluted steps like hash collisions if you have the means to control somebody's phone with a…

The big difference is that cloud providers do the scanning on their own infrastructure. If you don't want something scanned, you don't upload it to the could, that simple. But here, your own device is snitching on you.

But the discussion at hand is about malicious actors causing innocent people to trip the alarms. Given the clear capabilities of tools like Pegasus, "just don't enable cloud syncing" is obviously not sufficient to protect against a malicious actor who wants to plant illegal content on your device and trip the alarms.

Re: Hash collision in Apple NeuralHash model

#323

Post by floatingatoll »

Edd314159 wrote: post #181
bo1024 wrote: post #152

Earlier quoted context omitted.

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation. A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash,…

> A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash, allowing them to trigger false positives any time. This is my understanding too. But is this not also true for other (cloud-based) CSAM scanning systems? Why is Apple's special in this regard?

There’s a lot of overlap between Apple product buyers and “fight the thoughtcrime slippery slope” hackers. The CSAM abusers are presumably (if they’re not stupid) also fanning the flames on that slippery slope perception, because it’s to their benefit if the hackers defeat Apple.

Re: Hash collision in Apple NeuralHash model

#324

Post by short_sells_poo »

UncleMeat wrote: post #157
short_sells_poo wrote: post #127

Earlier quoted context omitted.

Google was not standing on a pedestal preaching privacy. In contrast Apple was trying to appear as the privacy conscious hardware/software vendor. To now implement such a blatantly obvious stepping stone to dragnet surveillance of actual devices is such a hypocritical move that it beggars belief. Ignoring the whataboutism in your question, we know that privacy once lost is practically impossible to get back. Once the…

It just seems to me that there are two very different conversations happening at the same time, with people swapping back and forth between them 1. There can be false positives or other mechanisms for innocent people to get flagged. 2. It is bad to do this sort of check on the local disk. The discussion at hand started as entirely #1. But now you've swapped to #2, talking about government spying on local files. It ma…

I'd argue they are both important points that are tightly interconnected.

#2 is bad on it's own and in my mind it shouldn't even get to the merits of discussing #1 because at that point the debate is already lost in favor of surveillance.

But beyond that, #1 is highly problematic, doubly so given the fact that government surveillance is basically a non-decreasing function.

Re: Hash collision in Apple NeuralHash model

#325

Post by JKCalhoun »

sandworm101 wrote: post #215
Grustaf wrote: post #200

Earlier quoted context omitted.

How likely is it that you will have enough colliding images in your photo library to even trigger a review? I'm guessing you need at least 5 images, perhaps much more, to trigger it. In any case, 1 image is definitely not enough.

For a normal random person, a grey man. In the real world, a single collision could be enough for the police to acquire further access to people they are already looking at. If the police want access to your phone for other reasons (drugs, taxes, illegal speech) they can use that one collision to get a warrant which will give them greater access. It is akin to cops wanting to search a car. They don't need a warrant.…

Since, as I have read, Microsoft and Google already do this, where are all the illicit cop take-downs? I have not heard of any.

Re: Hash collision in Apple NeuralHash model

#326

Post by SXX »

halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

> The real challenge is generating a real image that could be mistaken for CSAM at low res + is actually benign (or else just send CSAM directly) + matches the hash of real CSAM.

Why do you have an idea that image have to be benign? Almost everyone watch porn and it's will be so much easier to find collisions by manipulating actual porn images which are not CSAM.

Also this way you'll more likely to trigged false-positive from Apple staff since they aren't suppose to see how actual CSAM looks like.

Re: Hash collision in Apple NeuralHash model

#327

Post by brokensegue »

bo1024 wrote: post #147
halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

> The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. Strongly disagree. (1) The primary feature of any decent hash function is that this should not happen. (2) Any preimage attack opens the way for further manipulations like you describe.

cryptographic hashes are different from image fingerprints

Re: Hash collision in Apple NeuralHash model

#328

Post by snowwrestler »

sandworm101 wrote: post #201
nbzso wrote: post #106

Earlier quoted context omitted.

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

>> there is no law who requires them to "scan" on device. There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a…

I don’t think this is correct.

1) Even if Apple does roll out this system, it’s not a system for scanning files on customer devices. It’s a system for comparing photos being uploaded to iCloud to a single standardized (everyone has the same one) list of hashes.

2) Apple likely has the legal power to refuse to alter that list of hashes, by the same argument they used against the FBI’s request to bypass the unlock code on specific iPhones.

3) Any argument about what Apple will and will not need to do with this system needs to explain how Microsoft Defender (or other AV products) interact with law enforcement, since those are software systems that scan client devices (ALL files, typically) for signature matches.

Re: Hash collision in Apple NeuralHash model

#329

Post by rootusrootus »

vesinisa wrote: post #13

Now this offers Apple a very delicate opportunity to back out of the whole scanning controversy due to technological vulnerabilities.

The only way they will back out is if this misunderstanding of perceptual hashes results in significant blowback from average customers. Apple is certainly not going to be surprised that a perceptual hash collision is technically easy.

Re: Hash collision in Apple NeuralHash model

#330

Post by floatingatoll »

notquitehuman wrote: post #282
nojito wrote: post #179

Any matches are matched again server side to thwart this type of attack. >Once Apple's iCloud Photos servers decrypt a set of positive match vouchers for an account that exceeded the match threshold, the visual derivatives of the positively matching images are referred for review by Apple. First, as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, indep…

It doesn’t matter what they do after “looking for something to report to law enforcement.” Nothing after that makes it less invasive.

Incorrect. The objections here are threefold:

1) This is invasive.

2) Slippery slope.

3) Gets innocents arrested.

Their solution mitigates #3 with review and mitigates #1 with blurring, and those mitigations occur and the point in the process where you claim it doesn’t matter what they do.

Please don’t reductively dismiss facts that don’t support your narrative. Saying that it doesn’t matter is wrong and serves only to boost your particular viewpoint on #2.

Post reply on HN