Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

491–500 of 725 posts

Re: Hash collision in Apple NeuralHash model

#491

Post by falcolas »

lifthrasiir wrote: post #444
robertoandred wrote: post #360

Earlier quoted context omitted.

You cannot extract or reverse the CSAM hashes. They've been encrypted and blinded using server-side-only keys. If TFA said that, it's lying.

After more reading of the whitepaper I think you are right. As I understand it, given the image hash H0 and CSAM hashes H[1]...H[n] (some might be duplicates in disguise) the algorithm proceeds like this: - The device generates a secret X and divides it into X[1]...X[m] with the secret sharing algorithm. m is some large number and any k (but no less) copies out of X[i] are enough to reconstruct X. - The device stores…

That synopsis disagrees with Apple's own descriptions - or rather it goes into the secondary checks, which confuses the issue that the initial hash checks are indeed performed on-device:

> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: Hash collision in Apple NeuralHash model

#492

Post by dannyw »

shapefrog wrote: post #468
dannyw wrote: post #452

Earlier quoted context omitted.

What if it is legal pornography of 21 year olds but disturbed to collide with CSAM? You are aware even defence lawyers are not allowed to look at alleged CSAM material in court right?

The process would not trigger any action. The NCMEC, who can look at the material, and are the people to whom the matter is reported, would compare the flagged image with the source material and reject it as not matching known CSAM. What if the legal porn of a 21 year old that triggered the collision match looked really really really close? So close that a human can not distinguish between the image of a 12 year old…

You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar?

I'm not talking about images of rape here. I'm taking about images that you'd see on a regular porn site, of adults and their body parts.

You are also aware that CSAM covers anywhere from 0 to 17.99 years of age, and the legal obligation to report exists equally for the whole spectrum?

So let's say I download a close up pussy collection of 31 images of what I believe to be consenting 20 year olds, and what are consenting 20 year olds.

But they are actually planted by an attacker (let's say an oppressive regime who doesn't like me) and disturbed to match CSAM, that is, pussy close ups of 17 year olds. They are all just pussy pics. They will look the same.

Should I go to jail?

Do I have a non zero chance of going to jail? Yes.

Re: Hash collision in Apple NeuralHash model

#493

Post by [deleted] »

atoav wrote: post #337
devwastaken wrote: post #306

Earlier quoted context omitted.

The collisions are supposedly reviewed, my concern is that the process for photodna isn't going to always be the same, and we don't actually have any knowledge as to whether their claims are true. Eventually they will phase out human intervention and replace with gameable AI. 3 letter agencies don't need to review the actual images, they can easily get federal warrants based on some numbers. Apple is content with put…

One thing about this is that we are farther and farther away from "if there is evidence for a crime a jury can understand it and rationally decide what do with it" and we are getting closer to "if this lightbulb is glowing the machine says they are guilty, so better trust us". This kind of stuff should not be evidence, if anything it should be a indicator where to look.

[deleted]

Re: Hash collision in Apple NeuralHash model

#494

Post by MichaelGroves »

Grustaf wrote: post #462
short_sells_poo wrote: post #344

Earlier quoted context omitted.

So some underpaid contractor reviewing "visual derivatives" that may or may not be CSAM completely prevents governments from misusing this in your mind?

Considering how high profile and incredibly sensitive this is, I doubt they will hire an underpaid contractor. It's the opposite of app review, where the volume is very high but the stakes are low.

Facebook hires underpaid and poorly treated contractors to moderate exactly this same kind of content. I see no reason to believe Apple will be any different, particularly a few months from now when the general public's attention has shifted to other matters (assuming they're even paying attention right now. I don't think the interests of HN are necessarily representative of the general public..)

Re: Hash collision in Apple NeuralHash model

#495

Post by nomel »

reuben_scratton wrote: post #15
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

"Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.

The "verified by a human" is only the the last step of the process for Apple. After that, it's given to NCMEC for review, then it's reported to authorities. The Apple subcontractor isn't the only one verifying things here. What you're saying is not realistic.

Re: Hash collision in Apple NeuralHash model

#496

Post by sandworm101 »

jasamer wrote: post #481
sandworm101 wrote: post #246

Earlier quoted context omitted.

>> useful for areas where Apple really doesn't want to even look at the actual material Correct. It has plausible deniability built in. Apple is unable to verify that the images the government are looking for are actually CSAM. They could be political. They could be protest images. They could be Winnie the Pooh. Apple can plead ignorance as it blindly scans for whatever the requesting government asks it to scan for.…

What about the human reviewers at Apple? Those need to confirm matches, and they obviously look at the photos to do so. Apple can’t claim ignorance. Also, it has to be two requesting governments.

The current Apple policy to use human reviewers. That might change at a moment's notice and there is no technical reason why Apple couldn't bypass humans for certain requests. One must always judge a new system three ways: how it was meant to be implemented, how it is actually implemented, and how it might be abused by bad actors in the future.

Re: Hash collision in Apple NeuralHash model

#497

Post by dheera »

khuey wrote: post #268
sandworm101 wrote: post #201

Earlier quoted context omitted.

>> there is no law who requires them to "scan" on device. There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a…

At least in the United States, this is absolutely false. A warrant cannot force them to do something they have no capability to do.

(a) They created the capability, that's the problem

(b) The world is not the United States

Re: Hash collision in Apple NeuralHash model

#499

Post by bengale »

teitoklien wrote: post #485
bengale wrote: post #479

Earlier quoted context omitted.

Ok so your fear is someone will target your grandparents with child porn? If this was such a major problem why has no-one been targeted like this in the last decade when everyone else was scanning for it?

Here’s an example of exactly that https://news.ycombinator.com/item?id=28221907

It's not an example though. Putting aside a lack of news report we'll assume it's true, it's got nothing to do with cloud scanning for images. A picture of someones children wouldn't trigger these systems at all.

Re: Hash collision in Apple NeuralHash model

#500

Post by johnla »

FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

I don't think this can be used to harm an innocent person. It can raise a red flag but it would be quickly unraised and perhaps an investigation into the source of the fakeout images because THAT person had to have had the real images in possession.

If anything, this gives weapons to people against the scanner as we can now bomb the system with false positives rendering it impossible to use. I don't know enough about cryptography but I wonder if there is any ramifications of the hash being broken.

Post reply on HN