Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

611–620 of 725 posts

Re: Hash collision in Apple NeuralHash model

#611

Post by cyanite »

scandinavian wrote: post #12
kuu wrote: post #2

I don't understand the comment in the issue by an iPhone user. Can you see the hashes that the mobile generates for each image?? Why that is not "obfuscated" / hidden from the user? I mean, I would expect something complicated to validate that you have a collision.

It is hidden for the user. Obfuscation doesn't work. They probably just called the private API to generate a hash on a jailbroken phone. There's even a link to another piece of software that can do just that, only on macOS. https://github.com/KhaosT/nhcalc

The hash table is blinded on the device, and the device never knows if a given image is a hit or not. This is well documented.

Re: Hash collision in Apple NeuralHash model

#612

Post by robertoandred »

cyanite wrote: post #582
robertoandred wrote: post #463

Earlier quoted context omitted.

No, encrypted and blinded hashes are stored. You can’t extract them.

This also means that the device doesn’t know if a given image is a match.

Correct, only the server does.

Re: Hash collision in Apple NeuralHash model

#613

Post by cyanite »

jdlshore wrote: post #549
kuu wrote: post #2

I don't understand the comment in the issue by an iPhone user. Can you see the hashes that the mobile generates for each image?? Why that is not "obfuscated" / hidden from the user? I mean, I would expect something complicated to validate that you have a collision.

You're correct. The amount of misinformation in this thread (and in the other responses to you) is out of control. The database of CSAM hashes is blinded and no one has the hashes. Without the hashes, this attack is useless. It's also mitigated by a LOT of checks and balances. First they have to know 30 hashes to target (they're secret). They have to get 30 colliding images on your phone. The images have to be unnoti…

> The amount of misinformation in this thread (and in the other responses to you) is out of control.

True it’s not perfect here, but you should see Reddit, then :p. People there hardly even know what they are mad about.

Re: Hash collision in Apple NeuralHash model

#614

Post by FabHK »

mrits wrote: post #393
FabHK wrote: post #41

Earlier quoted context omitted.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

"Then, with all due respect, the attacker could just download actual CSAM." If you didn't have Apple scanning your drive trying to find a new way for you to go to prison then it wouldn't be a problem.

Yes. But this whole discussion is about potential problems/exploits with hash collisions (see title).

Re: Hash collision in Apple NeuralHash model

#615

Post by cyanite »

steeleduncan wrote: post #317
rollinggoron wrote: post #297

How is this scenario unique to Apple but not everyone else who does scanning? e.g. Google, Facebook, Microsoft etc...

If the entity doing the scanning has a copy of the original image they can verify it is illegal before calling the police. With Apple's system they have to call the police on the basis of the image hash without verifying that anything illegal is on the phone. You can whatsapp someone an innocent image doctored to have a hash collision with known CSAM. If they have default settings it will be saved to their photo reel…

Apple doesn’t “call the police”, though, they contact the center for child abuse or whatever it’s called, who will then presumably verify the picture.

Re: Hash collision in Apple NeuralHash model

#616

Post by heavyset_go »

gowld wrote: post #414
topynate wrote: post #376

Earlier quoted context omitted.

On reflection, yes, there must be warrants involved. I'm raising my estimate of how likely it is that innocent people get raided due to this. The warrant would properly only be to search iCloud, not some guy's house, but I can easily see overly-broad warrants being issued.

> The warrant would properly only be to search iCloud, iCloud is encrypted, so that warrant is useless. They need to unlock and search the device.

It's "encrypted", but Apple holds the keys and they regularly give up customers' data in response to government requests for it[1].

[1] https://www.apple.com/legal/transparency/us.html

Re: Hash collision in Apple NeuralHash model

#617

Post by ggreer »

gowld wrote: post #414
topynate wrote: post #376

Earlier quoted context omitted.

On reflection, yes, there must be warrants involved. I'm raising my estimate of how likely it is that innocent people get raided due to this. The warrant would properly only be to search iCloud, not some guy's house, but I can easily see overly-broad warrants being issued.

> The warrant would properly only be to search iCloud, iCloud is encrypted, so that warrant is useless. They need to unlock and search the device.

The data is encrypted, but Apple has the keys. If they get a warrant, they'll decrypt your data and hand it over. See page 11 of Apple's law enforcement process guidelines[1]:

> iCloud content may include email, stored photos, documents, contacts, calendars, bookmarks, Safari Browsing History, Maps Search History, Messages and iOS device backups. iOS device backups may include photos and videos in the Camera Roll, device settings, app data, iMessage, Business Chat, SMS, and MMS messages and voicemail. All iCloud content data stored by Apple is encrypted at the location of the server. When third-party vendors are used to store data, Apple never gives them the encryption keys. Apple retains the encryption keys in its U.S. data centers. iCloud content, as it exists in the customer’s account, may be provided in response to a search warrant issued upon a showing of probable cause, or customer consent.

1. https://www.apple.com/legal/privacy/law-enforcement-guidelin...

Re: Hash collision in Apple NeuralHash model

#618

Post by cyanite »

marcob95 wrote: post #411

Can someone help me figure out why this is such a big deal? Thanks

I’d say that it’s not, and that many people think it is is because they haven’t fully read or understood the technical description of the system.

But what you should do is read the documents yourself.

Re: Hash collision in Apple NeuralHash model

#619

Post by cyanite »

floatingatoll wrote: post #561
hughrr wrote: post #536

Earlier quoted context omitted.

Exactly this. I see lots of people saying that Apple are forced to implement something via legislation as if it’s an excuse for it. If they want to do business in China they will be forced by their legislation. My entire argument is don’t build the mechanism.

China forced Apple by legislation to implement new iCloud algorithms for assigning China-region user data into China-hosted datacenters. Most countries, unlike the US, are not constrained by a requirement to only exercise previously-built mechanisms and not create new ones, in response to government demands. If China decides to require Apple to censor non-CSAM content on-device, they will do so whether or not CSAM co…

> China-hosted China-decryptable datacenters.

China hosted, yes, but Apple denies China-decryptable, so that’s speculation unless you have a good source.

Re: Hash collision in Apple NeuralHash model

#620

Post by heavyset_go »

Kalium wrote: post #460
topynate wrote: post #425

Earlier quoted context omitted.

Yes, it's encrypted, but part of this anti-CSAM strategy is a threshold encryption scheme that allows Apple to decrypt photos if a certain number of them have suspicious hashes.

Apple having any kind of ability to decrypt user contents is disconcerting. It means they can be subpoenaed for that information.

> It means they can be subpoenaed for that information.

They regularly are, and they regularly give up customer data in order to comply with subpeonas[1]. They give up customer data in response to government requests for 150,000 users/accounts a year[1].

[1] https://www.apple.com/legal/transparency/us.html

Post reply on HN