Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

151–160 of 725 posts

Re: Hash collision in Apple NeuralHash model

#151

Post by Sebb767 »

saynay wrote: post #85
formerly_proven wrote: post #68

Earlier quoted context omitted.

Yes, the diligent review performed by the lowest-bidding subcontractor is an excellent defense against career-ending criminal accusations. Nothing can go wrong, this is fine.

I would think there is way easier ways to frame someone with CSAM then this. Like dump a thumbdrive of the stuff on them and report them to the police.

The police will not investigate every hint and a thumbdrive still has some plausible deniability. Evidence on your phone looks far worse and, thanks to this new process, law enforcement will receive actual evidence instead of just a hint.

Re: Hash collision in Apple NeuralHash model

#152

Post by bo1024 »

Edd314159 wrote: post #105

I think I am in dire need of some education here and so I have questions: * Is this a problem with Apple's CSAM discriminator engine or with the fact that it's happening on-device? * Would this attack not be possible if scanning was instead happening in the cloud, using the same model? * Are other services (Google Photos, Facebook, etc.) that store photos in the cloud not doing something similar to uploaded photos, w…

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation.

A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash, allowing them to trigger false positives any time. That appears to be what we are witnessing.

Re: Hash collision in Apple NeuralHash model

#153

Post by _jal »

coldtea wrote: post #135
d33 wrote: post #107

Earlier quoted context omitted.

In case of Appelbaum, are there any solid reasons to believe that the accusations are untrue? For Assange, I think that the victim admitted that the accusation was fabricated, isn't that the case?

Accusations must be proven true, not untrue by the accused. And besides the "victim" in the latter case there was a whole lot of diplomatic pressure and political commotion to set him up, with carrots and sticks and the aid of friendly satellite states.

They must be proven true beyond a reasonable doubt to get a person in a funny robe to put them in jail.

Normal humans are not required to prove anything in order to think them.

Re: Hash collision in Apple NeuralHash model

#154

Post by craftinator »

account42 wrote: post #100
OJFord wrote: post #73

Earlier quoted context omitted.

Vouched, because as I understand the comment, people must not be reading past 'rape' and just gut-flagging with completely the wrong impression.

> people must not be reading past 'rape' and just gut-flagging with completely the wrong impression. Which is pretty ironic, considering that kind of reaction is exactly what the comment is about.

Build it and they will come.

Re: Hash collision in Apple NeuralHash model

#155

Post by magpi3 »

bo1024 wrote: post #152
Edd314159 wrote: post #105

I think I am in dire need of some education here and so I have questions: * Is this a problem with Apple's CSAM discriminator engine or with the fact that it's happening on-device? * Would this attack not be possible if scanning was instead happening in the cloud, using the same model? * Are other services (Google Photos, Facebook, etc.) that store photos in the cloud not doing something similar to uploaded photos, w…

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation. A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash,…

But how would the attacker get the generated image on a person's phone?

Re: Hash collision in Apple NeuralHash model

#156

Post by aardvarkr »

incrudible wrote: post #81
FabHK wrote: post #41

Earlier quoted context omitted.

Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…

Also, this XKCD: https://xkcd.com/538/ People are getting nerd-sniped about hash collisions. It's completely irrelevant. The real-world vector is that an attacker sends CSAM through one of the channels that will trigger a scan. Through iMessage, this should be possible in an unsolicited fashion (correct me if I'm wrong). Otherwise, it's possible through a hacked device. Of course there's plausible deniability here, b…

Love the relevant xkcd! And to reply to your point, simply sending unsolicited CSAM via iMessage doesn’t trigger anything. That message has to be saved to your phone then uploaded to iCloud. Someone else above said repeat this process 20-30 times so I presume it can’t be a single incident of CSAM. Seems really really hard to trigger this thing by accident or maliciously

Re: Hash collision in Apple NeuralHash model

#157

Post by UncleMeat »

short_sells_poo wrote: post #127
UncleMeat wrote: post #109

Why is this meaningfully different than, say, what Google Photos has been doing for years? If you can get rooting malware on the target device then you could 1. Produce actual CSAM rather than a hash collision 2. Produce lots of it 3. Sync it with Google Photos This attack has been available for many years and does not need convoluted steps like hash collisions if you have the means to control somebody's phone with a…

Google was not standing on a pedestal preaching privacy. In contrast Apple was trying to appear as the privacy conscious hardware/software vendor. To now implement such a blatantly obvious stepping stone to dragnet surveillance of actual devices is such a hypocritical move that it beggars belief. Ignoring the whataboutism in your question, we know that privacy once lost is practically impossible to get back. Once the…

It just seems to me that there are two very different conversations happening at the same time, with people swapping back and forth between them

1. There can be false positives or other mechanisms for innocent people to get flagged.

2. It is bad to do this sort of check on the local disk.

The discussion at hand started as entirely #1. But now you've swapped to #2, talking about government spying on local files. It makes it very difficult to have a conversation because any pushback against arguments made for one point is assumed to be pushback against arguments made for the other point.

Re: Hash collision in Apple NeuralHash model

#158

Post by ziml77 »

Ensorceled wrote: post #140
Sebb767 wrote: post #137

Earlier quoted context omitted.

Testimony is seen as wonky in all kinds of cases. The problem with testimony about rape, however, is that, in those cases, supporting evidence is rarer than usual and even when it exists, proving it was non-consensual at the time is even harder (especially when relationships or affairs are involved).

Yes, rape is more difficult to prove than a number of other crimes. That is no reason to jump to a default "assume the accusation is false".

The reason to default to that is because a principle of our legal system is that people are innocent until proven guilty. On top of that proving someone guilty requires going beyond reasonable doubt.

Re: Hash collision in Apple NeuralHash model

#159

Post by magpi3 »

Can someone ELI5? I understand that a person can now generate an image with the same hash as an illegal image (such as child porn), but I don't understand how they can get it on someone's phone and I don't understand why someone would get in trouble for an image, when finally examined, that is clearly not child pornography.

Re: Hash collision in Apple NeuralHash model

#160

Post by Ensorceled »

technothrasher wrote: post #145
Ensorceled wrote: post #130

Earlier quoted context omitted.

> All criminal accusations, including true ones, should be treated as false until the accused is proven guilty. No, they need to be treated as unproven, a very critical difference. Just to be clear, witness testimony, including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle.

> No, they need to be treated as unproven, a very critical difference. Right. I have a jar of gumballs and tell you I think there is an even number of gumballs in it. Do you believe me? If you don't, does that mean you believe there are in fact an odd number of gumballs in it? No, you do not believe either that there are an odd or an even number, because you just don't know. For a criminal accusation , your initial b…

Yeah, "it is unverified or untested or unproven" seems to be a very hard concept for "black and white" brains to process. People keep following up "it is unproven" with "well then it must be false".

EDIT: judging by the downvotes, they are also making the leap to the original gumball counter must be lying ...

Post reply on HN