Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

101–110 of 725 posts

Re: Hash collision in Apple NeuralHash model

#101

Post by hanklazard »

deadalus wrote: post #3

Expectation : Political rivals and enemies of powerful people will be taken out because c-ild pornography will be found in their phone. Pegasus can already monitor and exfiltrate every ounce of data right now, it won't be that hard to insert compromising images on the infected device. Any news about "c-ild porn" being found on someone's phone is suspect now. This has been done before : 1) https://www.deccanchronicle.…

And as an extra horrible side-effect, the use of these kinds of attacks in the political arena will “vindicate” believers of the QAnon conspiracy.

Re: Hash collision in Apple NeuralHash model

#102

Post by enedil »

wisienkas wrote: post #90

Not knowing too much of the NeuralHash model, but why are they using MD5 hash, they are known to have many collisions. We don't use MD5 for private/public keys for the same reason

We don't use md5 for private/public keys because md5 is a hashing algorithm, unrelated completely to encryption. Also, what are your reasons to believe that md5 has been used there?

Re: Hash collision in Apple NeuralHash model

#103

Post by BrianOnHN »

zimpenfish wrote: post #88
hdjjhhvvhga wrote: post #74

Earlier quoted context omitted.

My WhatsApp automatically saves all images to my photo roll. It has to be explicitly turned off. When the default is on, it's enough that the image is received and the victim has CP on their phone. After the initial shock they delete it, but the image has already been sent to Apple, where a reviewer marked it as CP. Since the user already gave them their full address data in order to be able to use the app store, App…

> but the image has already been sent to Apple, where a reviewer marked it as CP No, the images are only decryptable after a threshold (which appears to be about 30) is breached. If you've received 30 pieces of CSAM from WhatsApp contacts without blocking them and/or stopping WhatsApp from automatically saving to iCloud, I gotta say, it's on you at that point.

Victim blaming because of failure to meet some seemingly arbitrary limit, ok.

Re: Hash collision in Apple NeuralHash model

#104

Post by avianlyric »

FabHK wrote: post #78
y7 wrote: post #65

Earlier quoted context omitted.

Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…

Right. So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images). But a conceivable novel avenue of attack would be to find an image that: 1. Does not look like CSAM to the innocent victim in the original 2. Does match known CSAM by NeuralHash 3. Does look like CSAM in the "visual derivative" reviewed by Apple, as you highlight.

Reading the imagine scaling attack article, it’s looks like it’s pretty easy to manufacture an image that:

1. Looks like an innocuous image, indeed even an image the victim is expecting to receive.

2. Downscales in such a way to produce a CSAM match.

3. Downscales for the derivative image to create actual CSAM for the review process.

Which is a pretty scary attack vector.

Re: Hash collision in Apple NeuralHash model

#105

Post by Edd314159 »

I think I am in dire need of some education here and so I have questions:

* Is this a problem with Apple's CSAM discriminator engine or with the fact that it's happening on-device?

* Would this attack not be possible if scanning was instead happening in the cloud, using the same model?

* Are other services (Google Photos, Facebook, etc.) that store photos in the cloud not doing something similar to uploaded photos, with models that may be similarly vulnerable to this attack?

I know that an argument against on-device scanning is that people don't like to feel like the device that they own is acting against them - like it's snitching on them. I can understand and actually sympathise with that argument, it feels wrong.

But we have known for a long time that computer vision can be fooled with adversarial images. What is special about this particular example? Is it only because it's specifically tricking the Apple CSAM system, which is currently a hotly-debated topic, or is there something particularly bad here, something that is not true with other CSAM "detectors"?

I genuinely don't know enough about this subject to comment with anything other than questions.

Re: Hash collision in Apple NeuralHash model

#106

Post by nbzso »

hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs.

Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO and there is no law who requires them to "scan" on device. So the logical explanation is "growth". Way to go Apple.

Re: Hash collision in Apple NeuralHash model

#107

Post by d33 »

hosteur wrote: post #98
reayn wrote: post #45

Earlier quoted context omitted.

Why is this getting downvoted? It’s very true, just the accusation of committing such a crime (regardless of whether the person was acquitted or not) can easily ruin many facets of a persons’ life.

Julian Assange and Jake Appelbaum being prime examples of this.

In case of Appelbaum, are there any solid reasons to believe that the accusations are untrue? For Assange, I think that the victim admitted that the accusation was fabricated, isn't that the case?

Re: Hash collision in Apple NeuralHash model

#108

Post by newsclues »

First CP.

Then leaked or unauthorized nudes will get filtered.

Filters for terrorists, and terrorist imagery or symbols.

Start scanning for guns and drugs.

Drug dealers and criminals get added to the list.

How long before before it’s dissidents, political opponents, and minorities in dictatorships?

How long before Tim Cooks CP filters are used against LGBT groups abroad?

Re: Hash collision in Apple NeuralHash model

#109

Post by UncleMeat »

Why is this meaningfully different than, say, what Google Photos has been doing for years?

If you can get rooting malware on the target device then you could

1. Produce actual CSAM rather than a hash collision

2. Produce lots of it

3. Sync it with Google Photos

This attack has been available for many years and does not need convoluted steps like hash collisions if you have the means to control somebody's phone with a RAT.

Re: Hash collision in Apple NeuralHash model

#110

Post by thulle »

ptidhomme wrote: post #5

Yes, just like rape accusations. It doesn't matter that you prove it was false afterwards. Edit : well that was a hint to Assange of course. Probably not true in general. So yes, I mean false accusations.

Huh? In what way has the accusations against Assange been disproven? Is this one of the "if he didn't jump someone random it isn't rape"-arguments?
Post reply on HN