Hash collision in Apple NeuralHash model • Hacker News Board

Re: Hash collision in Apple NeuralHash model

#171

Post by ziml77 » Wed, Aug 18, 2021, 12:54 PM UTC

aardvarkr wrote: post #156

incrudible wrote: post #81

Earlier quoted context omitted.

Also, this XKCD: https://xkcd.com/538/ People are getting nerd-sniped about hash collisions. It's completely irrelevant. The real-world vector is that an attacker sends CSAM through one of the channels that will trigger a scan. Through iMessage, this should be possible in an unsolicited fashion (correct me if I'm wrong). Otherwise, it's possible through a hacked device. Of course there's plausible deniability here, b…

Love the relevant xkcd! And to reply to your point, simply sending unsolicited CSAM via iMessage doesn’t trigger anything. That message has to be saved to your phone then uploaded to iCloud. Someone else above said repeat this process 20-30 times so I presume it can’t be a single incident of CSAM. Seems really really hard to trigger this thing by accident or maliciously

People are saying that, by default, WhatsApp will save images directly to your camera roll without any interaction. That would be an easy way to trigger the CSAM detection remotely. There are many people who use WhatsApp so it's a reasonable concern.

Re: Hash collision in Apple NeuralHash model

#172

Post by Ensorceled » Wed, Aug 18, 2021, 12:54 PM UTC

deelowe wrote: post #169

Ensorceled wrote: post #140

Earlier quoted context omitted.

Yes, rape is more difficult to prove than a number of other crimes. That is no reason to jump to a default "assume the accusation is false".

Depends. Are certain groups disproportionally more likely to benefit from said accusations? Sorry, it needs to be said. It's easier to take one side of an argument when it's less likely to affect you personally.

What are you actually claiming here? You stance is unclear.

Re: Hash collision in Apple NeuralHash model

#173

Post by snarf21 » Wed, Aug 18, 2021, 12:54 PM UTC

deadalus wrote: post #3

Expectation : Political rivals and enemies of powerful people will be taken out because c-ild pornography will be found in their phone. Pegasus can already monitor and exfiltrate every ounce of data right now, it won't be that hard to insert compromising images on the infected device. Any news about "c-ild porn" being found on someone's phone is suspect now. This has been done before : 1) https://www.deccanchronicle.…

I'm against this change from Apple but weren't they already doing this scanning in iCloud? Couldn't someone use it the same way against a "political rival and enemies of powerful people"?

Re: Hash collision in Apple NeuralHash model

#174

Post by Sebb767 » Wed, Aug 18, 2021, 12:54 PM UTC

Ensorceled wrote: post #140

Sebb767 wrote: post #137

Earlier quoted context omitted.

Testimony is seen as wonky in all kinds of cases. The problem with testimony about rape, however, is that, in those cases, supporting evidence is rarer than usual and even when it exists, proving it was non-consensual at the time is even harder (especially when relationships or affairs are involved).

Yes, rape is more difficult to prove than a number of other crimes. That is no reason to jump to a default "assume the accusation is false".

> That is no reason to jump to a default "assume the accusation is false".

That's also not what I was trying to say. I only explained why testimony is far more doubted in rape cases than in most other cases. In general, I agree with you that accusations should be treated as unproven, so neither false nor true; we have a justice system to give a final verdict.

Re: Hash collision in Apple NeuralHash model

#175

Post by varispeed » Wed, Aug 18, 2021, 12:55 PM UTC

magpi3 wrote: post #159

Can someone ELI5? I understand that a person can now generate an image with the same hash as an illegal image (such as child porn), but I don't understand how they can get it on someone's phone and I don't understand why someone would get in trouble for an image, when finally examined, that is clearly not child pornography.

> that is clearly not

For one, you can't know if that's true as the image could have been manipulated to appear as such. For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image.

> but I don't understand how they can get it on someone's phone

There is many vectors. For example you can leave phone unattended and someone can snap a picture of an image or since a collision may look innocent to you, you would overlook it in an email etc...

Re: Hash collision in Apple NeuralHash model

#176

Post by elzbardico » Wed, Aug 18, 2021, 12:55 PM UTC

Ensorceled wrote: post #166

skhr0680 wrote: post #146

Earlier quoted context omitted.

> including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle. In a criminal trial: Victim: This person did it Defendant: No I didn't Not guilty

That is not how it works, a large number of people are convicted on eye witness testimony from a single accuser.

Yes, they shouldn't.

Re: Hash collision in Apple NeuralHash model

#177

Post by vnchr » Wed, Aug 18, 2021, 12:55 PM UTC

magpi3 wrote: post #159

Can someone ELI5? I understand that a person can now generate an image with the same hash as an illegal image (such as child porn), but I don't understand how they can get it on someone's phone and I don't understand why someone would get in trouble for an image, when finally examined, that is clearly not child pornography.

I would think a Message with the attached photo from a burner phone/account would be enough.

Re: Hash collision in Apple NeuralHash model

#178

Post by mschuster91 » Wed, Aug 18, 2021, 12:55 PM UTC

nbzso wrote: post #106

hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

> and there is no law who requires them to "scan" on device

Yet. The demands by "concerned parents" (aka fronts for secret services, puritans/other religious fundamentalists and law-and-order hardliners) to "do something against child porn" have grown ever more strong and insane over the last years. (And you can bet that what is used on CSAM will immediately be used to go after legal pornography, sex work, drug enforcement, ...)

Many current and powerful politicians don't understand a single bit about computers, to the point of bragging of never having used one (e.g. Japan's cybersecurity minister) or having assistants print out emails daily and transcribing handwritten responses (can't say more than that this is the situation for at least two German MPs) - but there are young politicians who do, and will replace them over the next decade hopefully. That means it's last chance for lobbying groups to get devastating laws passed through, and we must all be vigorous in spotting and preventing at least the worst of them.

And what is suspiciously lacking in Apple's response: what are they going to do when they are compelled to extend the CSAM scanner by law in the US, India, China and/or EU? It's feasible for Apple to say they'll just be sticking the middle finger towards markets such as Russia, Saudi-Arabia or similar tiny dictatorships, but the big markets? Apple can't ignore these, and especially India and China are heavyweights.

Re: Hash collision in Apple NeuralHash model

#179

Post by nojito » Wed, Aug 18, 2021, 12:56 PM UTC

Any matches are matched again server side to thwart this type of attack.

>Once Apple's iCloud Photos servers decrypt a set of positive match vouchers for an account that exceeded the match threshold, the visual derivatives of the positively matching images are referred for review by Apple. First, as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.

They also fuzz this process by sending false positives I think?

Re: Hash collision in Apple NeuralHash model

#180

Post by Sebb767 » Wed, Aug 18, 2021, 12:56 PM UTC

Ensorceled wrote: post #167

ziml77 wrote: post #158

Earlier quoted context omitted.

The reason to default to that is because a principle of our legal system is that people are innocent until proven guilty. On top of that proving someone guilty requires going beyond reasonable doubt.

The principle is the defendant is innocent until proven guilty, NOT that the accuser is making a false accusation. There is a reason the verdict is "not guilty" instead of "innocent". After a not guilty verdict the legal system still does not assume the accuser was making a false accusation.

> There is a reason the verdict is "not guilty" instead of "innocent".

Both verdicts exists, actually; the latter one is just far rarer (since it is both harder to prove and usually not what is argued about).