Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

531–540 of 725 posts

Re: Hash collision in Apple NeuralHash model

#531

Post by user-the-name »

shmatt wrote: post #250
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

the blacklisted hashes are openly given to thousands of companies by a foundation called the National Center for Missing & Exploited Children, in both PhotoDNA and MD5 versions of the hashes. Yes, I can't give you an open link, but I imagine at least hundreds of thousands of people have access to download them. Some companies with better security than others It only takes 1 guy to sell it to an NSO-type company, who…

Where would that $$$ come from? Like I said, the only attack possible is to overload the human reviewers, and even that is difficult to pull off except as a coordinated effort by lots of people.

Re: Hash collision in Apple NeuralHash model

#532

Post by zepto »

zekrioca wrote: post #530
zepto wrote: post #269

Earlier quoted context omitted.

Vindicated? This can’t be used for a targeted attack. It’s no different from the previous false posting making this claim.

Imagine if WhatsApp and other apps added received photos to iPhone's iCloud Photos gallery by default.. wait, they do.

Imagine if Apps had to ask permission before they could save photos…

Imagine if the local photo storage is not the same as iCloud Photo Library…

Imagine if anyone who cared could simply switch off iCloud Photo Library…

Re: Hash collision in Apple NeuralHash model

#533

Post by fortenforge »

bo1024 wrote: post #152
Edd314159 wrote: post #105

I think I am in dire need of some education here and so I have questions: * Is this a problem with Apple's CSAM discriminator engine or with the fact that it's happening on-device? * Would this attack not be possible if scanning was instead happening in the cloud, using the same model? * Are other services (Google Photos, Facebook, etc.) that store photos in the cloud not doing something similar to uploaded photos, w…

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation. A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash,…

No, I don't think that's what we're witnessing. No one has yet demonstrated that they can take a hash alone and produce an image that matches the hash. It's true that that would be bad, but right now you still need the original colliding image.

Re: Hash collision in Apple NeuralHash model

#534

Post by sweetheart »

short_sells_poo wrote: post #385
sweetheart wrote: post #370

Earlier quoted context omitted.

Their point is that the attack vector being described isn’t new, as CSAM could already be weaponized against folks, and we never really ever hear if that happening. So the OP is simply saying that perhaps it’s not an issue we need to worry about. I happen to agree with them.

So in your mind, because so far we've seen no evidence that this has been abused, it's nothing to worry about going forward? And that making an existing situation even more widespread is also completely OK?

> So in your mind, because so far we've seen no evidence that this has been abused, it's nothing to worry about going forward?

Yeah, basically. It doesn't seem like people actually use CSAM to screw over innocent folks, so I don't think we need to worry about it. What Apple is doing doesn't really make that any easier, so it's either already a problem, or not a problem.

> And that making an existing situation even more widespread is also completely OK?

I don't know if I'd say any of this is "completely OK", as I don't think I've fully formed my opinion on this whole Apple CSAM debate, but I at least agree with OP that I don't think we need to suddenly worry about people weaponizing CSAM all of a sudden when it's been an option for years now with no real stories of anyone actually being victimized.

Re: Hash collision in Apple NeuralHash model

#535

Post by [deleted] »

Grustaf wrote: post #310
gorbachev wrote: post #288

Earlier quoted context omitted.

For individual users the risk is very small, but when you have a billion users the chances of innocent people being caught up is pretty much 100%. Platform owners like Google, Apple, Twitter and Facebook should really keep stuff like that in mind when they deploy algorithmic solutions like this. Like dhosek said, the manual review step should reduce the risk quite a bit, though.

They did keep it in mind, that's why they require 30 matches, which gets the false positive rate down to 1 in a trillion per year, so on average it will happen about 1 per millennium, given a billion users. So that's per photo library, not per photo. The rate per photo in their testing was 3 in 100 million, then they added a 30x safety margin and assumed it's actually 1 in a million. https://www.zdnet.com/article/app…

[deleted]

Re: Hash collision in Apple NeuralHash model

#536

Post by hughrr »

neolog wrote: post #523
floatingatoll wrote: post #512

Earlier quoted context omitted.

It's serious to you , but CSAM scanning is irrelevant to 99.99999% of Apple's customers, and Jobs would never have allowed an announcement about migrating CSAM scanning from uploads to the cloud to the device uploading to the cloud. That's an implementation detail that wouldn't be relevant to discuss with outsiders. Instead, I expect he would have presented it in a closed session to the FBI and/or Congress. Never to…

> CSAM scanning is irrelevant to 99.99999% of Apple's customers How long until an group of governments tells Apple to add Tank Man to the list?

Exactly this. I see lots of people saying that Apple are forced to implement something via legislation as if it’s an excuse for it.

If they want to do business in China they will be forced by their legislation.

My entire argument is don’t build the mechanism.

Re: Hash collision in Apple NeuralHash model

#537

Post by romwell »

JKCalhoun wrote: post #332
fsloth wrote: post #70

Earlier quoted context omitted.

"How can you use it for targeted attacks?" Just insert a known CSAM image on target's device. Done. I presume this could be used against a rival political party to ruin their reputation - insert bunch of CSAM images on their devices. "Party X is revealed as an abuse ring". This goes oh-so-very-nicely with Qanon conspiracy theories which even don't require any evidence to propagate widely. Wait for Apple to find the i…

> Just insert a known CSAM image on target's device. Or maybe thirty. You have to surpass the threshold. Also, if Twitter, Google, Microsoft are already deploying CSAM scanning in their services .... why are we not hearing about all the "swatting"?

>Also, if Twitter, Google, Microsoft are already deploying CSAM scanning in their services .... why are we not hearing about all the "swatting"?

>their services

>T H E I R S E R V I C E S

Because it's on their SERVICES, not on their user's DEVICES, for one.

Also, regardless of swatting, that's why we have an issue with Apple.

Re: Hash collision in Apple NeuralHash model

#538

Post by sneak »

Sebb767 wrote: post #139
cma wrote: post #69

Earlier quoted context omitted.

That's the burden for incarceration, not for making a personal best guess about guilt. Even far below best guess, would you send your kid with a camp councilor that you were 20% sure was guilty of something like that?

I probably would be doubtful even at 1%, sure. But there's no denying that this ruins the life of 99 innocent people for every actual criminal.

https://en.wikipedia.org/wiki/Blackstone%27s_ratio

Re: Hash collision in Apple NeuralHash model

#539

Post by fortenforge »

pkulak wrote: post #474

If your first thought, like mine, was "Who cares? The whole point is that there will be plenty of false positives.", this is pre-image: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...

I don't think that's true. This was a second pre-image attack.

Re: Hash collision in Apple NeuralHash model

#540

Post by hannasanarion »

fsloth wrote: post #70
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

"How can you use it for targeted attacks?" Just insert a known CSAM image on target's device. Done. I presume this could be used against a rival political party to ruin their reputation - insert bunch of CSAM images on their devices. "Party X is revealed as an abuse ring". This goes oh-so-very-nicely with Qanon conspiracy theories which even don't require any evidence to propagate widely. Wait for Apple to find the i…

> I presume this could be used against a rival political party to ruin their reputation - insert bunch of CSAM images on their devices.

Okay, and then what? You think people will just look at this cute picture of a dog and be like "welp, the computer says it's a photo of child abuse, so we're taking you to jail anyway"?

Post reply on HN