Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

11–20 of 725 posts

Re: Hash collision in Apple NeuralHash model

#11

Post by rdli »

halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

Are you pinning your hopes that a false positive like this will be appropriately caught because of an army of faceless, low wage workers who stare at CSAM cases all day will immediately flag?

Re: Hash collision in Apple NeuralHash model

#12

Post by scandinavian »

kuu wrote: post #2

I don't understand the comment in the issue by an iPhone user. Can you see the hashes that the mobile generates for each image?? Why that is not "obfuscated" / hidden from the user? I mean, I would expect something complicated to validate that you have a collision.

It is hidden for the user. Obfuscation doesn't work. They probably just called the private API to generate a hash on a jailbroken phone. There's even a link to another piece of software that can do just that, only on macOS.

https://github.com/KhaosT/nhcalc

Re: Hash collision in Apple NeuralHash model

#14

Post by atrus »

user-the-name wrote: post #9
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

I doubt someone who is able/wanting to attack such a system would be unwilling to own some of the material themselves.

Re: Hash collision in Apple NeuralHash model

#15

Post by reuben_scratton »

user-the-name wrote: post #9
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

"Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.

Re: Hash collision in Apple NeuralHash model

#16

Post by hughrr »

user-the-name wrote: post #9
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc.

In the mean time they lock your account which means your entire digital life stops dead until their review process is done.

No way do I accept any of this.

Also the hashes are on the device I understand and it’s not going to be that difficult to extract them.

Re: Hash collision in Apple NeuralHash model

#17

Post by csande17 »

user-the-name wrote: post #9
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

> It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography.

I've never personally seen or looked for any images like this, but if they weren't already proliferating online and widely available to criminals, why would we need to build an elaborate client-side scanning system to detect and report people who have copies of them?

Re: Hash collision in Apple NeuralHash model

#18

Post by FabHK »

How can you use it for targeted attacks?

This is what would need to happen:

1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available)

2. Attacker sends that to innocent person

3. Innocent person accepts and stores the picture

4. Actually, need to run step 1-3 at least 30 times

5. Innocent person has iCloud syncing enabled

6. Apple's CSAM detection then flags these, and they're manually reviewed

7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times

Note that other cloud providers have been scanning uploaded photos for years. What has changed wrt targeted attacks against innocent people?

Re: Hash collision in Apple NeuralHash model

#19

Post by [deleted] »

kuu wrote: post #2

I don't understand the comment in the issue by an iPhone user. Can you see the hashes that the mobile generates for each image?? Why that is not "obfuscated" / hidden from the user? I mean, I would expect something complicated to validate that you have a collision.

[deleted]

Re: Hash collision in Apple NeuralHash model

#20

Post by bko »

deadalus wrote: post #3

Expectation : Political rivals and enemies of powerful people will be taken out because c-ild pornography will be found in their phone. Pegasus can already monitor and exfiltrate every ounce of data right now, it won't be that hard to insert compromising images on the infected device. Any news about "c-ild porn" being found on someone's phone is suspect now. This has been done before : 1) https://www.deccanchronicle.…

Isn't it weird how it is weaponized against political enemies but the one person everyone knows did engage in exploitation was protected for decades?
Post reply on HN