Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

github.com

Post reply on HN
Print view View original

141–150 of 363 posts

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#141

Post by cirrus3 »

fnord77 wrote: post #111

how long until they start scanning a device's framebuffer in realtime? why stop at CSAM? Pirated material like movies next?

They are not doing this for the fun of it. If they didn't have to, they would not do it at all.

You have made a huge leap from scanning for pre-existing CSAM while in transit to a cloud service to scanning frame buffers on device in real-time. You should get some type of Olympic medal for such a leap.

This tech is to catch the lowest possible hanging fruit of the dumbest of all CSAM-sharing/saving folks as required by law.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#142

Post by ec109685 »

godelski wrote: post #98
bArray wrote: post #48

Earlier quoted context omitted.

To defeat this, all you need to be is a state actor with a database of child porn at your disposal (which is stored for exactly the purpose of training detection systems). Then you run the hashing algorithm against images you know are in the database (Apple suggested that they would accept suggestions by some kind of multi-Country vote). Then you can pull out the hashes and figure out how to trigger false positives o…

There's already a problem that Apple can't verify the hashes. Say a government wants to investigate a certain set of people. Those people probably share specific memes and photos. Add those hashes to the list and now you have reasonable cause to investigate these people. Honestly this even adds to the danger of hash collisions because now you can get someone on a terrorist watch list as well as the kiddy porn list.

Apple is the one doing the first line of investigation.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#143

Post by onepunchedman »

cookiengineer wrote: post #135
onepunchedman wrote: post #25

This is just getting wilder and wilder by the day, how spectacularly this move has backfired. As others have commented, at this point all you need is someone willing to sell you the CSAM hashes on the darknet, and this system is transparently broken. Until that day, just send known CSAM to any person you'd like to get in trouble (make sure they have icloud sync enabled), be it your neighbour or a political figure, an…

Imagine being a parent that made pictures of their own children that bathed naked in their own backyard. I don't know about you, but my parents certainly have lots of embarassing pictures of me in their photo album. There will be so many false positives in that system, it's ridiculous. It doesn't necessarily have to be a false colliding hash, but legitimate use cases that - by definition - are impossible to train neu…

What does Apple even do in this situation? That media won't match known CSAM, but if you modify childhood images so that its hash matches CSAM, what does Apple do. There are just SO MANY things that can and will go wrong as people try to exploit this system.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#144

Post by ec109685 »

jchw wrote: post #32
only_as_i_fall wrote: post #26

Earlier quoted context omitted.

I don't see how. They're hashing on feature space (so trivial cropping and such doesn't defeat this) but they have two totally separate methods of matching those hashes? Doesn't sound right to me...

Apparently the images in question would get sent to the server, and all calculation happens there. > In a call with reporters regarding the new findings, Apple said its CSAM-scanning system had been built with collisions in mind, given the known limitations of perceptual hashing algorithms. In particular, the company emphasized a secondary server-side hashing algorithm, separate from NeuralHash, the specifics of whic…

Hard to exploit is better phrasing.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#145

Post by zepto »

onepunchedman wrote: post #25

This is just getting wilder and wilder by the day, how spectacularly this move has backfired. As others have commented, at this point all you need is someone willing to sell you the CSAM hashes on the darknet, and this system is transparently broken. Until that day, just send known CSAM to any person you'd like to get in trouble (make sure they have icloud sync enabled), be it your neighbour or a political figure, an…

Except that it isn’t. The hashes don’t enable an attack.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#146

Post by hypothesis »

robertoandred wrote: post #38
dannyw wrote: post #30

Earlier quoted context omitted.

The "send known CSAM" attack has existed for a while but never made sense. However, this technology enables a new class of attacks: "send legal porn, collided to match CSAM perceptual hashes". With the previous status quo: 1. The attacker faces charges of possessing and distributing child pornography 2. The victim may be investigated and charged with child pornography if LEO is somehow alerted (which requires work, a…

LEO is not alerted automatically, where’d you get that idea?

They will be if you collide a low-res image that resembles CSAM.

Why would person doing manual review risk his job in case if he’s unsure? Naturally he will just play it safe and report images.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#147

Post by ec109685 »

throwaway384950 wrote: post #67

Some people seem to be confused why a hash collision of a cat and a dog matters. Here's a potential attack: share (legal) NSFW pictures that are engineered to have a hash collision with CSAM to get someone else in trouble. The pictures are flagged as CSAM, and they also look suspicious to a human reviewer (maybe not enough context in the image to identify the subject's age). To show that this can be done with real NS…

Keep in mind that you have to also collide with another perceptual hash function that only Apple has to trigger a match.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#148

Post by zepto »

onepunchedman wrote: post #143
cookiengineer wrote: post #135

Earlier quoted context omitted.

Imagine being a parent that made pictures of their own children that bathed naked in their own backyard. I don't know about you, but my parents certainly have lots of embarassing pictures of me in their photo album. There will be so many false positives in that system, it's ridiculous. It doesn't necessarily have to be a false colliding hash, but legitimate use cases that - by definition - are impossible to train neu…

What does Apple even do in this situation? That media won't match known CSAM, but if you modify childhood images so that its hash matches CSAM, what does Apple do. There are just SO MANY things that can and will go wrong as people try to exploit this system.

You can’t modify your childhood images so their hash matches csam because the visual derivative won’t match.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#149

Post by simondotau »

Dylan16807 wrote: post #79
robertoandred wrote: post #58

Earlier quoted context omitted.

What kind of social engineering would lead an innocent person to save known CSAM to their photo library?

If some commenters can be believed about their experience with the database, there are a bunch of completely innocuous images in it because they're from the same photosets or distributed alongside CSAM. Is that enough to cause an investigation? Maybe, maybe not, but I wouldn't want it to be a risk.

Photos in the database are classified for their content. Only images classified as A1 (A: prepubescent minor, 1: sex act) are being included in the hash set on iOS. So this doesn't even include A2 (2: lascivious exhibition), B1 or B2 (B: pubescent minor) let alone images which are in the database and aren't classified as any of A1, A2, B1 or B2.

While I've no doubt that there's a lot of "before and after" images (which are still technically CSAM even if they're not strictly child porn) and possibly many innocuous images, they would not have been flagged as "A1".

I'm sure there's probably still a few images flagged as A1 which shouldn't be in the database at all, but that number is going to be small. How many of these incorrectly flagged images are going to make their way into your photo library? One? Two?

You need 30 in order for your account to be flagged.

Re: Show HN: Neural-hash-collider – Find target hash collisions for NeuralHash

#150

Post by lamontcg »

fay59 wrote: post #123
dannyw wrote: post #30

Earlier quoted context omitted.

The "send known CSAM" attack has existed for a while but never made sense. However, this technology enables a new class of attacks: "send legal porn, collided to match CSAM perceptual hashes". With the previous status quo: 1. The attacker faces charges of possessing and distributing child pornography 2. The victim may be investigated and charged with child pornography if LEO is somehow alerted (which requires work, a…

Before they make it to human review, photos in decrypted vouchers have to pass the CSAM match against a second classifier that Apple keeps to itself. Presumably, if it doesn’t match the same asset, it won’t be passed along. This is explained towards the end of the threat model document that Apple posted to its website. https://www.apple.com/child-safety/pdf/Security_Threat_Model...

You can still hack someone's phone and upload actual CSAM images. That exposes the attacker to additional charges, but they're already facing extortion and all that anyway. I don't understand the "golly gee whizz, they'd have to commit a severe felony first in order to launch that kind of attack" argument.

Don't know why this hasn't already been used on other cloud services, but maybe it will be now that its been more widely publicized.

Post reply on HN