Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

261–270 of 725 posts

Re: Hash collision in Apple NeuralHash model

#261

Post by UseStrict »

UncleMeat wrote: post #109

Why is this meaningfully different than, say, what Google Photos has been doing for years? If you can get rooting malware on the target device then you could 1. Produce actual CSAM rather than a hash collision 2. Produce lots of it 3. Sync it with Google Photos This attack has been available for many years and does not need convoluted steps like hash collisions if you have the means to control somebody's phone with a…

The _initial_ implementation of client-side scanning has the same initial result, but vastly different futures. Up until now it could be (foolishly or otherwise) assumed that Apple had your personal privacy in mind. Now they have demonstrated a willingness to compromise your local device privacy. Right now it's iCloud only, but I'm sure it's a boolean configuration away somewhere to make it scan everything, regardless of whether it's marked for iCloud upload or not. And I don't think Apple is strong enough to withstand a gagged demand to flip that boolean.

On their cloud services (Apple, Google, MS, etc) they can only get to content that a user has implicitly agreed to share (whether or not they understand the ramifications is another conversation). On your device, there's no more separation.

Re: Hash collision in Apple NeuralHash model

#262

Post by notquitehuman »

vesinisa wrote: post #13

Now this offers Apple a very delicate opportunity to back out of the whole scanning controversy due to technological vulnerabilities.

There’s no going back now. They revealed themselves to be enthusiastic participants in a creeping global surveillance system. They’d have to fire every single executive who passed on an opportunity to tank this thing before they get the opportunity to make the case that they’ve changed.

Re: Hash collision in Apple NeuralHash model

#263

Post by kemayo »

endisneigh wrote: post #218
Koffiepoeder wrote: post #163

Earlier quoted context omitted.

E.g. send them a whatsapp message that looks innocent

They can see the image - why would they import a random image into their library from someone they don’t know?

WhatsApp has a really weird default behavior: it imports all images you're sent into your photo library.

This is a smart thing to disable, even outside this recent discussion of CSAM.

https://faq.whatsapp.com/android/how-to-stop-saving-whatsapp...

Re: Hash collision in Apple NeuralHash model

#264

Post by dylan604 »

Grustaf wrote: post #236
dylan604 wrote: post #232

Earlier quoted context omitted.

I've only seen Apple admit defeat once, and that was regarding the trashcan MacPro. Otherwise, it's "you're holding it wrong" type of victim blaming as they quietly revise the issue on the next version. Can anyone else think of times where Apple has admitted to something bad on their end and then reversed/walked away from whatever it was?

Do you really care what they "admit"? I thought you were worried about innocent people being framed. Obviously if a way to frame people gets widespread, Apple will stop it. They don't want that publicity.

You clearly have me confused with someone else, as I never mentioned anything about innocent people being framed.

With Apple, nothing is "obvious".

Re: Hash collision in Apple NeuralHash model

#265

Post by cwizou »

tambourine_man wrote: post #209

Neuralhashes are far from my area of expertise, but I've been following Apple closely ever since its foundation and have probably watched every public video of Craig since the NeXT take over and here is my take: I've never seen him so off balance before as in his latest interview with Joanna Stern. Not even in the infamous “shaking mouse hand close up” of the early days. Whatever you say about Apple, they are an extr…

I think they clearly didn't anticipate that people would perceive it as anything but a breach of trust, that their device was working against them (even for a good cause, against the worst people).

And because of this they calibrated their communication completely wrong, focusing on the on device part as being more private. Using the same line of thinking they use for putting Siri on device.

And the follow up was an uncoordinated mess that didn't help either (as you rightly pointed out with Craig's interview). In the Neuenschwander interview [1], he stated this :

> The hash list is built into the operating system, we have one global operating system and don’t have the ability to target updates to individual users and so hash lists will be shared by all users when the system is enabled.

This still has me confused, here's my understanding so far (please feel free to correct me)

- Apple is shipping a neural network trained on the dataset that generates NeuralHashes

- Apple also ships (where ?) a "blinded" (by an eliptic curve algo) table lookup that match (all possible?!) NeuralHashes to a key

- This key is used to encrypt the NeuralHash and the derivative image (that would be used by the manual review) and this bundle is called the voucher

- A final check is done on server using the secret used to generate the elliptic curve to reverse the NeuralHash and check it server side against the known database

- If 30 or more are detected, decrypt all vouchers and send the derivative images to manual review.

I think I'm missing something regarding the blinded table as I don't see what it brings to the table in that scenario, apart from adding a complex key generation for the vouchers. If that table only contained the NeuralHashes of known CSAM images as keys, that would be as good as giving the list to people knowing the model is easily extracted. And if it's not a table lookup but just a cryptographic function, I don't see where the blinded table is coming from in Apple's documentation [2].

Assuming above assumptions are correct, I'm paradoxically feeling a tiny bit better about that system on a technical level (I still think doing anything client side is a very bad precedent), but what a mess did they put themselves into.

Had they done this purely server side (and to be frank there's not much difference, the significant part seems to be done server side) this would have been a complete non-event.

[1] : https://daringfireball.net/linked/2021/08/11/panzarino-neuen...

[2] This is my understanding based on the repository and what's written page 6-7 : https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Hash collision in Apple NeuralHash model

#266

Post by Grustaf »

sandworm101 wrote: post #246
Grustaf wrote: post #223

Earlier quoted context omitted.

Why would they extend the CSAM scanner? It would be much simpler to just use all the OCR and image classification functions they have already deployed. CSAM scanning is only useful for areas where Apple really doesn't want to even look at the actual material until they are extremely certain that it's a match. If they want to detect anti-government propaganda or something, there would be no such concerns, they would j…

>> useful for areas where Apple really doesn't want to even look at the actual material Correct. It has plausible deniability built in. Apple is unable to verify that the images the government are looking for are actually CSAM. They could be political. They could be protest images. They could be Winnie the Pooh. Apple can plead ignorance as it blindly scans for whatever the requesting government asks it to scan for.…

Except that Apple will review the photos once you've matched 30 of them, so it's still not possible for the government to misuse it.

Re: Hash collision in Apple NeuralHash model

#267

Post by notriddle »

bostonsre wrote: post #259
y7 wrote: post #65

Earlier quoted context omitted.

Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…

Why would someone do that? Why not just send the original if both are flagged as the original?

The victim needs to store the image in their iCloud, so it needs to not look like CSAM to them.

Re: Hash collision in Apple NeuralHash model

#268

Post by khuey »

sandworm101 wrote: post #201
nbzso wrote: post #106

Earlier quoted context omitted.

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

>> there is no law who requires them to "scan" on device. There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a…

At least in the United States, this is absolutely false. A warrant cannot force them to do something they have no capability to do.

Re: Hash collision in Apple NeuralHash model

#269

Post by zepto »

hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

Vindicated?

This can’t be used for a targeted attack. It’s no different from the previous false posting making this claim.

Post reply on HN