Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

vice.com

Post reply on HN
Print view View original

321–330 of 465 posts

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#321

Post by rlpb »

shapefrog wrote: post #289
rlpb wrote: post #282

Earlier quoted context omitted.

> Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere. Define "get anywhere". Why won't you get raided by the police and have all your devices seized first?

No, probably wouldnt even get to the desk of the police. End of the road would be NCMEC, to whom apple would refer hash matching images that pass human verification that are close enough to porn. If your 30 or so hash matching images matched their corresponding known CSAM then that goes on to the police and then they knock on your door.

> to whom apple would refer hash matching images that pass human verification

I am under the impression that Apple's scheme allows them only to verify the output of the matching algorithm (the "safety vouchers"), and not the image content itself. So in the hypothetical situation described in the thread, it won't be possible for Apple to detect the false positive, and they could pass on the report to NCMEC.

I fear that this will lead to a law enforcement raid without any actual human verification of the offending image itself.

If I'm wrong, I welcome citations which demonstrate the opposite.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#322

Post by mschuster91 »

zepto wrote: post #252
mschuster91 wrote: post #139

Earlier quoted context omitted.

> Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children. They're not going to find predators, all they are going to find is people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones and dumb enough to have iCloud backup turned on. To catch actual predators they would need to run AI scanning on phones for all photos…

> people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones Who do you think has 30 or more CSAM images on their phone?

Pedophiles. But these people are consumers of extremely old content, which means the CSAM can won't even make a tiny dent against active, current abusers!

CSAM scanning is sold as beneficial, while in reality it won't do shit while it opens dangerous precedence backdoors!

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#323

Post by kbelder »

crooked-v wrote: post #14
geoah wrote: post #5

> The system relies on a database of hashes—cryptographic representations of images—of known CSAM photos provided by National Center for Missing & Exploited Children (NCMEC) and other child protection organizations. “Cryptographic representations of images”. That’s not the case though right? These are “neuralhashes” afaik which are nowhere close to cryptographic hashes but rather locality sensitive hashes which is a…

Something to note here is that in the hash collision that was discovered, the two images look nothing alike. One is a picture of a dog, the other is blobby grey static.

I'd be more interested in whether you could take an image of, say, an underage dog, and tweak a vaguely similar image of a grown dog so that it was a match. That's what will get innocent people thrown in jail.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#324

Post by ultrarunner »

floatingatoll wrote: post #205
laverya wrote: post #193

Earlier quoted context omitted.

And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied? Bonus points if you match poses, coloration, background, etc.

The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image,…

Cops no-knock raid the wrong houses all the time; what makes you think that the procedure you outlined above will be followed without error? Especially against suspected child abusers, from an anti-child-abuse agency?

What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#325

Post by ultrarunner »

floatingatoll wrote: post #307
laverya wrote: post #283

Earlier quoted context omitted.

This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one. And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that…

Apple's reviewers are comparing blurred originals to blurred matches. It needs to look like the blurred original associated with the checksum that matched. It is irrelevant whether the blurred match looks pornographic or not.

This has already been demonstrated.

https://twitter.com/SarahJamieLewis/status/14280837442802565...

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#326

Post by cirrus3 »

They are down to hash collision arguments against this now?

Sounds pretty desperate.

As if any normal user is going to upload a photo to iCloud that is a collision.

The fact that such images are possible to generate means nothing by itself.

Also, they would need to accidentally have 30 of them.

Also, a human would have to not be able to tell the difference.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#327

Post by petersellers »

endisneigh wrote: post #320
petersellers wrote: post #318

Earlier quoted context omitted.

> iCloud Photos can be used on Windows, for example. This scanning only applies to iOS devices. You could use iCloud Photos and not be subject to the scanning. That's great for the >Nothing about what they're doing is contradictory with privacy beyond what they're already doing. The only reason they're even implementing it this way is because they do care about privacy. They could just not encrypt and scan on the ser…

NCMEC is an arm of the government. Either they share voluntarily or they get subpoenaed endlessly. Why do you think all of these companies even do this? lol in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.

> NCMEC is an arm of the government. Either they share voluntarily or they get subpoenaed endlessly.

A subpoena is a lot different than scanning every single photo on every user's device automatically.

> Why do you think all of these companies even do this? lol

Because most companies don't give a shit about privacy? And they will cave at even the slightest government pressure to do so. Honestly it's probably easier for them that way (but worse for the consumer).

> in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.

Gee thanks, your "solution" is 1000x harder to use than just using iCloud Photos on your iPhone. One of the biggest selling points of it now is for convenience, and this is pretty obvious so I'm starting to think you're just trolling at this point.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#328

Post by nojito »

__blockcipher__ wrote: post #183
fraa-orolo wrote: post #171

Earlier quoted context omitted.

Because it's not a cryptographic hash where a one bit difference results in a completely different hash. It's a perceptual hash that operates on a smaller bitmap derived from the image so it's plausible that some innocuous images might result in similar derivations; and there might be intentionally crafted innocently-looking images that result in an offensive derivative. Salvador Dali could do something similar by ha…

This is a great answer but that’s not actually the GP’s contention. Their argument is essentially “so what if there’s a collision, the human review will catch it”. And to that I’d say that the same is supposed to occur for the no-fly list and we all know how that works in practice. The mere accusal itself of possessing CSAM can be life ruining if it gets to that stage. More importantly, a collision will effectively a…

That’s one check. There are other system checks to make the client side hash collision meaningless.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#329

Post by lupire »

CoolGuySteve wrote: post #293
slg wrote: post #220

Earlier quoted context omitted.

But this doesn't change any of that. It only changes whether the scanning happens on your device as part of the upload process instead of on the server after the upload.

Yeah that’s right and I don’t think either method is ethical.

Then don't ask other peole to hold your non encrypted files for you.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#330

Post by int_19h »

nine_k wrote: post #247
mox1 wrote: post #130

Earlier quoted context omitted.

But who was complaining about google and microsoft doing the cloud scanning? I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.

That's the point: catching the bad guy foolish enough to keep known CSAM images on their phone, while not technically invading the privacy of any good guys. Anyway, if apple wanted to covertly invade their users' privacy, they'd have no technical problems to do so. What it takes to accept is the "nothing to hide" mentality: your files are safe to scan (locally) because they can't be known CSAM files. You have to trus…

The "nothing to hide" mentality is exactly what's wrong about all this.
Post reply on HN