Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

41–50 of 725 posts

Re: Hash collision in Apple NeuralHash model

#41

Post by FabHK »

spicybright wrote: post #31
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

If you're in close physical contact with a person (like at a job) you just wait for them to put their phone down while unlocked, and do all this.

Then, with all due respect, the attacker could just download actual CSAM.

> If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https://. If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled with tumors, they’re going to hold a press conference and say “It wasn’t us” as they wear t-shirts that say “IT WAS DEFINITELY US,” and then they’re going to buy all of your stuff at your estate sale so that they can directly look at the photos of your vacation instead of reading your insipid emails about them.

https://www.usenix.org/system/files/1401_08-12_mickens.pdf

Re: Hash collision in Apple NeuralHash model

#42

Post by cm2187 »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Don’t imessage and whatsapp automatically store all images received in the iphone’s photo library?

Re: Hash collision in Apple NeuralHash model

#43

Post by jsdalton »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

For #4, I know for a fact that my wife’s WhatsApp automatically stores pictures you send her to her iCloud. So the grey blob would definitely be there unless she actively deleted it.

Re: Hash collision in Apple NeuralHash model

#44

Post by thinkingemote »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

the issue is step 6 - review and action

Every single tech company is getting rid of manual human review towards an AI based approach. Human-ops they call it - they dont want their employees to be doing this harmful work, plus computers are cheaper and better at

We hear about failures of inhuman ops all the time on HN. people being banned, falsely accused, cancelled, accounts locked, credit denied. All because the decisions which were once by humans are now made by machine. This will happen eventually here too.

It's the very reason why they have the neuralhash model. To remove the human reviewer.

Re: Hash collision in Apple NeuralHash model

#45

Post by reayn »

ptidhomme wrote: post #5

Yes, just like rape accusations. It doesn't matter that you prove it was false afterwards. Edit : well that was a hint to Assange of course. Probably not true in general. So yes, I mean false accusations.

Why is this getting downvoted? It’s very true, just the accusation of committing such a crime (regardless of whether the person was acquitted or not) can easily ruin many facets of a persons’ life.

Re: Hash collision in Apple NeuralHash model

#46

Post by FabHK »

soziawa wrote: post #36
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

> 6. Apple's CSAM detection then flags these, and they're manually reviewed Is the process actually documented anywhere? Afaik they are just saying that they are verifying a match. This could of course just be a person looking at the hash itself.

They look at the contents of the "safety voucher", which contains the neural hash and a "visual derivative" of the original image (but not the original image itself).

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Hash collision in Apple NeuralHash model

#47

Post by Nextgrid »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

> the NeuralHashes of which, unless I'm mistaken, are not available

Given the scanning is client-side wouldn't the client need a list of those hashes to check against? If so it's just a matter of time before those are extracted and used in these attacks.

Re: Hash collision in Apple NeuralHash model

#48

Post by nabakin »

FabHK wrote: post #35
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

Remains to be shown whether that is possible, though.

Unfortunately, it is very likely to be possible. Adversarial ML is extremely effective. I won't be surprised if this is achieved within the day, if not sooner tbh.

Re: Hash collision in Apple NeuralHash model

#49

Post by brigandish »

ptidhomme wrote: post #5

Yes, just like rape accusations. It doesn't matter that you prove it was false afterwards. Edit : well that was a hint to Assange of course. Probably not true in general. So yes, I mean false accusations.

I think you may have attracted less downvotes if the phrasing was changed to "Yes, just like false accusations of rape, it doesn't matter that you prove it was false afterwards."

I also think that those downvoting you might've applied the principle of charity and taken the best interpretation of what you've written or at least ask.

Re: Hash collision in Apple NeuralHash model

#50

Post by michaelt »

FabHK wrote: post #35
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

Remains to be shown whether that is possible, though.

Just yesterday, here on HN there was an article [1] about adversarial attacks that could make road signs get misread by ML recognition systems

I'd be astonished if it wasn't possible to do the same thing here.

[1] https://news.ycombinator.com/item?id=28204077

Post reply on HN