Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

621–630 of 725 posts

Re: Hash collision in Apple NeuralHash model

#621

Post by dragonwriter »

cyanite wrote: post #615
steeleduncan wrote: post #317

Earlier quoted context omitted.

If the entity doing the scanning has a copy of the original image they can verify it is illegal before calling the police. With Apple's system they have to call the police on the basis of the image hash without verifying that anything illegal is on the phone. You can whatsapp someone an innocent image doctored to have a hash collision with known CSAM. If they have default settings it will be saved to their photo reel…

Apple doesn’t “call the police”, though, they contact the center for child abuse or whatever it’s called, who will then presumably verify the picture.

The National Center for Missing and Exploited Children, a “private” nonprofit created by the US government and primarily funded by the Department of Justice.

Re: Hash collision in Apple NeuralHash model

#622

Post by cyanite »

tandav wrote: post #514
dmix wrote: post #473

Earlier quoted context omitted.

There’s a significant leap from implementing something server side to on the consumer handheld devices themselves. Even if it’s just similar software it is regardless much more serious. I personally thought the unencrypted backups was enough of a death-knell as it provides everything on your phone but anything with real-time on device access is always a gold mine for surveillance hawks.

until ios source code is closed all privacy claims is only backed by trust. They easily can do whatever they want if you're not compiling from source. There's no way to ensure your data is not leaving your iphone/mac with some "system" network requests.

Security researchers and hackers routinely look at the code on the device, though.

Re: Hash collision in Apple NeuralHash model

#623

Post by nullc »

robertoandred wrote: post #526
nullc wrote: post #489

Earlier quoted context omitted.

The grey blob is a proof of concept. The existence of the original image is proof that not all images which produce the target hash are grey blobs. Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. So, you should perhaps instead imagine attackers that modify lawful nude images to have matching hashes with child porn images. With that in mind, most…

And where would they get the CSAM hash they need to match?

The bulk of the CSAM hash database comes widely circulated images, available on sleeze dark web websites (and historically on sketchy porn sites just on the web).

So someone need only find a collection of likely included images and compute their hashes and publish it.

The attack works just as well even if the attacker isn't sure that every hash is in the database, they just need to know enough likely-matches such that they get enough hits.

Consider it this way, some attacker spends a couple hours searching for child porn and finds some stuff. ... what kind of failure would the NCMEC database be if it didn't include that material?

Re: Hash collision in Apple NeuralHash model

#624

Post by cyanite »

JKCalhoun wrote: post #318
Grustaf wrote: post #200

Earlier quoted context omitted.

How likely is it that you will have enough colliding images in your photo library to even trigger a review? I'm guessing you need at least 5 images, perhaps much more, to trigger it. In any case, 1 image is definitely not enough.

Not sure why the downvotes. You are absolutely correct that a threshold in the number of positives (false or otherwise) must be met. To be fair though we do not know what the threshold is. But I would guess even higher than 5 — I would presume 12 or more. I'm no criminologist (IANAC) but when you read about someone getting busted with child pornography they have hundreds or thousand of images — not one, not five. The…

Apple says they expect to use 30 as the threshold.

Re: Hash collision in Apple NeuralHash model

#625

Post by hannasanarion »

argvargc wrote: post #599
hannasanarion wrote: post #562

Earlier quoted context omitted.

Why do you think somebody will accuse you of a crime because you have a photo of a grey blob? The real world isn't as stupid as the computer one. The justice system is not deterministic and automatic. Nobody is going to look at this grey blob and go "welp, we have no choice but to throw you in prison forever"

The real world and the computer world intersect. This is precisely typified by what is being discussed, surely? Apple is trying to automate and computerise a process that was not automated previously, apply it to a huge number of people, and with disastrous potential consequences should their wonderful design be found lacking. And within days, they have already utterly failed to provide one of their own self-stated a…

The process of making accusations and arrests is not automatic.

The Apple system sends flagged photos to reviewers, and if the reviewers find them suspicious, they send them to the poor souls at NCMEC, who will compare the flagged photo with the original illegal photo that's supposedly a match, and inform law enforcement if they are in fact a match.

Nobody will get cops at their door because somebody sent them a grey blob image.

The process that's being "automated" is merely an automatic flag that initiates a several-step process of human review. Apple isn't rolling out robocops.

Seriously, what are the "disasterous consequences" that you envision? What is the sequence of events where a hash collision leads to any inconvenience whatsoever for a user?

Re: Hash collision in Apple NeuralHash model

#626

Post by ksec »

floatingatoll wrote: post #405
ksec wrote: post #362

Earlier quoted context omitted.

I actually want Apple to stand ground and implement this feature. Like you said the double down on PR and marketing was enough for me. I may not be dumping all iOS and Mac for now. But it was " the " definite signal and evidence this is no longer the old Steve Jobs's Apple. It is like watching Mark Zuckerberg talking about privacy when he doesn't understand anything about it. ( Or more like he has a different underst…

Steve Jobs would have implemented this in secret and never told us at all, same as they already did for iCloud photos so many years ago. That would have been a far better approach than today’s Apple is taking. Oh well.

Well Steve Jobs was the one that resisted PRISM and gave the middle finger to NSA.

Re: Hash collision in Apple NeuralHash model

#627

Post by FabHK »

fsloth wrote: post #70
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

"How can you use it for targeted attacks?" Just insert a known CSAM image on target's device. Done. I presume this could be used against a rival political party to ruin their reputation - insert bunch of CSAM images on their devices. "Party X is revealed as an abuse ring". This goes oh-so-very-nicely with Qanon conspiracy theories which even don't require any evidence to propagate widely. Wait for Apple to find the i…

> "How can you use it for targeted attacks?" > Just insert a known CSAM image on target's device. Done.

Yes, but then the hash collision (topic of this article) is irrelevant.

Re: Hash collision in Apple NeuralHash model

#628

Post by Jcowell »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

one vector you can use to skip step 3 is to send on WhatsApp. I believe images sent via WhatsApp are auto saved by default last I recalled.

Re: Hash collision in Apple NeuralHash model

#629

Post by cyanite »

gjsman-1000 wrote: post #290
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It's not really end-game, because the original hashes are, themselves, hashed and not available (so you don't have a hash to work towards). And second, even if you somehow managed to get over that huge leap, raw noise won't pass Apple's review, so you have to reverse-engineer a new image that looks like CSAM, to match a hash you don't have. Big leaps required.

Technical nitpick: the hashes are not hashed (again), but rather cryptographically blinded. This is not reversible, so seems like a hash, but computations can be done on it which result can in some cases be seen by the server. This is actually all very clever, and described in a paper by Apple, linked from the technical summary article.

Re: Hash collision in Apple NeuralHash model

#630

Post by cyanite »

short_sells_poo wrote: post #315
FabHK wrote: post #136

Earlier quoted context omitted.

It only shares "visual derivatives" of images whose NeuralHash match the NeuralHash of known CSAM (either by being the same image ("perceptually") or a collision).

That to me just sounds like weasel words to avoid having to say that it shares images. Let's not beat about the bush, the "visual derivative" has to be good enough to identify what's going on in it for the manual confirmation. Are you actually arguing in good faith here at all? Because I can't see how a "visual derivative" that's nevertheless good enough for manual confirmation is any better than the source image?

> Are you actually arguing in good faith here at all?

Are you? Because you just seemed to claim that it could match against innocent pictures of your naked children, but this tells me that you don’t understand that this system looks for known pictures, not for something that looks like naked children.

Edit: if you do, apologies, but then I’d say that Apple has suggested that it’s a low resolution version of the picture. This should be contrasted with server side scanning, where the server accesses all pictures fully.

Post reply on HN