Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash • Hacker News Board

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#51

Post by [deleted] » Thu, Aug 19, 2021, 2:44 AM UTC

onepunchedman wrote: post #45

systemvoltage wrote: post #42

Earlier quoted context omitted.

I posted another comment that was misunderstood as well. Folks, no one is proposing to download actual CSAM images to your photo lib. You could be duped thinking you downloaded an image of a beautiful sunset which was carefully manipulated to match the hash of an actual CSAM image.

This is the really scary part. Of course getting someone to download blobs that corrolate to CSAM would be one thing, but downloading regular photos that have nefarious hashes is a trend /pol/ could start in an afternoon.

[deleted]

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#52

Post by onepunchedman » Thu, Aug 19, 2021, 2:44 AM UTC

robertoandred wrote: post #46

onepunchedman wrote: post #41

Earlier quoted context omitted.

Most people wouldn't of course. In this scenario you'd get someone to download the CSAM unknowingly. If they have iCloud sync it automatically uploads to iCloud, thereby triggering the system. At that point the authorities will be alerted by Apple, and you can inform media outlets. They in turn will ask law enforcement who will confirm the investigation, and the reputation of the person investigated will be tarnished…

You specifically said someone would be sent known CSAM. How would that get added to their photo library?

I meant * could *. My point is that social engineering is a clear weak link in this system. They can also be sent regular photos whose hash matches the database, or use this repo to transform a regular pornographic photo's hash, making it hard for manual confirmation on Apple's part.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#53

Post by dannyw » Thu, Aug 19, 2021, 2:46 AM UTC

endisneigh wrote: post #40

Why does matter? The photo looks nothing like the target? If someone looks at the two images wouldn’t they see they’re not the same and therefore the original image was mistakenly linked with the target

Apple's reviewers, by law, cannot look at the target. No one except NCMEC is allowed to possess the target (CSAM material).

So Apple will be looking at a low-res grayscale image of whatever the collided image is, which could be legal adult pornography (let's say: a screengrab of legal "teen" 18+ porn), but the CSAM filter tells it that it's abuse material!

What would you do as the Apple reviewer?

(Hint: You only have one option, as you are legally mandated to report).

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#54

Post by XorNot » Thu, Aug 19, 2021, 2:46 AM UTC

d0100 wrote: post #31

dannyw wrote: post #19

Earlier quoted context omitted.

You can extract the hashes with a few hours spent on the darknet. Doing that is certainly illegal and not to mention VERY morally wrong, but criminals exist and criminals won't hesitate to abuse this as a mechanism for framing, extortion, or ransom. It's also possible for someone (Attacker A) to go on the darknet and get a list of 96-bit neural hashes, and then publish or sell this list somewhere to another party, At…

> If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn Even worse, just get a "teen" porn screengrab, pass it through the collider and you have pretty much a smoking gun

The "visual derivative" is not something any of us have been shown an example of either. Whatever it is, I suspect you only need to be vaguely in the same ballpark (I would wager humanoid shaped skin tones maybe).

So I suspect it would be easier then that (particularly since this whole hashing scheme has been surrounded with a lot of clear garbage - "1 in a trillion" -> on demand collisions in a couple of weeks?

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#55

Post by robertoandred » Thu, Aug 19, 2021, 2:46 AM UTC

systemvoltage wrote: post #42

robertoandred wrote: post #36

Earlier quoted context omitted.

Why would anyone save CSAM to their photo library?

I posted another comment that was misunderstood as well. Folks, no one is proposing to download actual CSAM images to your photo lib. You could be duped thinking you downloaded an image of a beautiful sunset which was carefully manipulated to match the hash of an actual CSAM image.

The parent was proposing to “just send known CSAM”.

But OK, say someone sends you a sunset that fools the hasher. Then what? Of course one match won’t do anything, so you’d need to download however many matching sunsets. Then what? The Apple reviewer would see they’re sunsets and you’d challenge the flag saying they’re sunsets. And if somehow NCMEC got involved, they’d see they’re just sunsets. And if law enforcement got involved, they’d see they’re just sunsets.

These proofs of concept might seem interesting from a ML pov, but all they do is just highlight why Apple put so many layers of checking into this.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#56

Post by Alupis » Thu, Aug 19, 2021, 2:46 AM UTC

endisneigh wrote: post #40

Why does matter? The photo looks nothing like the target? If someone looks at the two images wouldn’t they see they’re not the same and therefore the original image was mistakenly linked with the target

You could pollute the pool and overwhelm their human review process, making it untenable to operate. And that's if you just wanted to pollute it with obvious non-CSAM content.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#57

Post by dannyw » Thu, Aug 19, 2021, 2:48 AM UTC

robertoandred wrote: post #20

ryanmarsh wrote: post #7

Ok so now all we have to do is get a phone, load it with adversarial images that have hashes from the CSAM database and we wait and see what happens. Basically a honeypot. Get some top civil rights attorneys involved. Take the case to the Supreme Court. Get precedence set right. Lawfare

Where would you get the CSAM hashes?

Give it a few days, and you'll probably find someone selling a list of CSAM neural hashes on darknet marketplaces.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#58

Post by robertoandred » Thu, Aug 19, 2021, 2:48 AM UTC

onepunchedman wrote: post #52

robertoandred wrote: post #46

Earlier quoted context omitted.

You specifically said someone would be sent known CSAM. How would that get added to their photo library?

I meant * could *. My point is that social engineering is a clear weak link in this system. They can also be sent regular photos whose hash matches the database, or use this repo to transform a regular pornographic photo's hash, making it hard for manual confirmation on Apple's part.

What kind of social engineering would lead an innocent person to save known CSAM to their photo library?

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#59

Post by systemvoltage » Thu, Aug 19, 2021, 2:48 AM UTC

robertoandred wrote: post #55

systemvoltage wrote: post #42

Earlier quoted context omitted.

I posted another comment that was misunderstood as well. Folks, no one is proposing to download actual CSAM images to your photo lib. You could be duped thinking you downloaded an image of a beautiful sunset which was carefully manipulated to match the hash of an actual CSAM image.

The parent was proposing to “just send known CSAM”. But OK, say someone sends you a sunset that fools the hasher. Then what? Of course one match won’t do anything, so you’d need to download however many matching sunsets. Then what? The Apple reviewer would see they’re sunsets and you’d challenge the flag saying they’re sunsets. And if somehow NCMEC got involved, they’d see they’re just sunsets. And if law enforcement…

It would still be mentally draining to be accussed of CP. Can you imaging how terrified one would be if they see a warning message with a blurred sunset? I don't know exactly how the system works but from Apple's press release, it hides the image and gives a warning to the user. This would not go well on social media.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#60

Post by ncw96 » Thu, Aug 19, 2021, 2:50 AM UTC

only_as_i_fall wrote: post #24

ncw96 wrote: post #12

Apple has said this is not the final version of the hashing algorithm they will be using: https://www.vice.com/en/article/wx5yzq/apple-defends-its-ant...

Does it matter? Unless they're going to totally change the technology I don't see how they can do anything but buy time until it's reverse engineered. After all, the code runs locally. If Apple wants to defend this they should try to explain how the system will work even if generating adversarial images is trivial.

Apple has outlined[1] multiple levels of protection in place for this:

1. You have to reach a threshold of matches before your account is flagged.

2. Once the threshold is reached, the matched images are checked against a different perceptual hash algorithm on Apple servers. This means an adversarial image would have to trigger a collision on two distinct hashing algorithms.

3. If both hash algorithms show a match, then “visual derivative” (low-res versions) of the images are inspected by Apple to confirm they are CSAM.

Only after these three criteria are met is your account disabled and referred to NCMEC. NCMEC will then do their own review of the flagged images and refer to law enforcement if necessary.

[1]: https://www.apple.com/child-safety/pdf/Security_Threat_Model...