Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

561–570 of 725 posts

Re: Hash collision in Apple NeuralHash model

#561

Post by floatingatoll »

hughrr wrote: post #536
neolog wrote: post #523

Earlier quoted context omitted.

> CSAM scanning is irrelevant to 99.99999% of Apple's customers How long until an group of governments tells Apple to add Tank Man to the list?

Exactly this. I see lots of people saying that Apple are forced to implement something via legislation as if it’s an excuse for it. If they want to do business in China they will be forced by their legislation. My entire argument is don’t build the mechanism.

China forced Apple by legislation to implement new iCloud algorithms for assigning China-region user data into China-hosted datacenters. Most countries, unlike the US, are not constrained by a requirement to only exercise previously-built mechanisms and not create new ones, in response to government demands. If China decides to require Apple to censor non-CSAM content on-device, they will do so whether or not CSAM content fingerprinting exists. That China has not done so is because they benefit greatly from Apple's manufacturing and sales and do not wish to create a diplomatic incident with Apple.

Re: Hash collision in Apple NeuralHash model

#562

Post by hannasanarion »

argvargc wrote: post #345
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It didn't even last a week. Introducing the people asking us to trust them with the responsibility of mass, automated crime accusations.

Why do you think somebody will accuse you of a crime because you have a photo of a grey blob?

The real world isn't as stupid as the computer one. The justice system is not deterministic and automatic. Nobody is going to look at this grey blob and go "welp, we have no choice but to throw you in prison forever"

Re: Hash collision in Apple NeuralHash model

#563

Post by alfiedotwtf »

gowld wrote: post #414
topynate wrote: post #376

Earlier quoted context omitted.

On reflection, yes, there must be warrants involved. I'm raising my estimate of how likely it is that innocent people get raided due to this. The warrant would properly only be to search iCloud, not some guy's house, but I can easily see overly-broad warrants being issued.

> The warrant would properly only be to search iCloud, iCloud is encrypted, so that warrant is useless. They need to unlock and search the device.

Didn't the FBI stop Apple from encrypting iCloud, and only Messenger is e2e?

Re: Hash collision in Apple NeuralHash model

#564

Post by fogof »

pkulak wrote: post #474

If your first thought, like mine, was "Who cares? The whole point is that there will be plenty of false positives.", this is pre-image: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...

That was, indeed, my first thought. Can you explain why reposting the link that the OP posted is supposed to change my mind, or how it contributes to the conversation?

Re: Hash collision in Apple NeuralHash model

#565

Post by hannasanarion »

hnfong wrote: post #560
hannasanarion wrote: post #550

Earlier quoted context omitted.

Where did you get this idea, scooby doo? It is not illegal to be an unwilling recipient of illegal material. If a package shows up at your door with a bomb, you're not gonna be thrown in jail for having a bomb.

In theory, sure. At the very least, you'd be one of the primary suspects, and if you somehow got a bad lawyer, all bets are off. https://hongkongfp.com/2021/08/12/judges-criticise-hong-kong...

Okay, and when a cursory look at the bomb actually reveals it to be a fisher-price toy, what then?

What is the scenario where a grey blob gets on your phone that sets off CSAM alerts, an investigator looks at it and sees only a grey blob, and then still decides to alert the authorities even though it's just a grey blob, and the authorities still decide to arrest you even though it's just a grey blob, and the DA still decides to prosecute you even though it's just a grey blob, and a jury still decides to convict you, even though it's still just a grey blob?

You're the one who's off in theory-land imagining that every person in the entire justice system is just as stupid as this algorithm is.

Re: Hash collision in Apple NeuralHash model

#566

Post by beiller »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

Maybe they could install malware that makes all camera images taken using a technique like stenography to cause false positive matches for all the photos taken by the device. Maybe they could share one photo album where all the images are hash collisions.

Re: Hash collision in Apple NeuralHash model

#568

Post by sudosysgen »

bbatsell wrote: post #555
topynate wrote: post #425

Earlier quoted context omitted.

Yes, it's encrypted, but part of this anti-CSAM strategy is a threshold encryption scheme that allows Apple to decrypt photos if a certain number of them have suspicious hashes.

No, the threshold encryption only allows for the 30+ cryptographic “vouchers” to be unlocked, which contain details about the hash matching as well as a “visual derivative” of the image. We don’t know any details about the visual derivative.

Im guessing the visual derivative is a difference of sorts between the image and the CSAM? Of course not sure.

Re: Hash collision in Apple NeuralHash model

#569

Post by nonbirithm »

vesinisa wrote: post #13

Now this offers Apple a very delicate opportunity to back out of the whole scanning controversy due to technological vulnerabilities.

The idea that because the state of the art right now is flawed that Apple should give up misses the point.

Even if it is flawed, in a few years Apple will just release a new version or a different technique that works more effectively for its stated purpose. Other companies will silently improve the server-side scanning techniques they already use after the public outcry started by Apple blows over.

So long as companies feel the need to protect themselves from liability for hosting certain kinds of data, be it criminal or political or moral or anything else, there are no societal checks in place to stop them from doing so.

This is not a technological issue like so many people are trying to frame it as; it is a policy issue.

Re: Hash collision in Apple NeuralHash model

#570

Post by HALtheWise »

sudosysgen wrote: post #568
bbatsell wrote: post #555

Earlier quoted context omitted.

No, the threshold encryption only allows for the 30+ cryptographic “vouchers” to be unlocked, which contain details about the hash matching as well as a “visual derivative” of the image. We don’t know any details about the visual derivative.

Im guessing the visual derivative is a difference of sorts between the image and the CSAM? Of course not sure.

No, since apple _definitely_ isn't sending CSAM photos to your phone so they can be differ. Most likely, the visual derivative is a thumbnail or blurred version of the image.
Post reply on HN