Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

drive.google.com

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#81

Post by soneil »

mcintyre1994 wrote: post #33

What hash algorithm are they using? If this is legitimate (I’ll be honest - I’m not clicking) then surely any hash this easy to pre-image attack is completely useless? Why wouldn’t they be using a cryptographic hash here?

As I understand it the NN "perceptual hash" is supposed to hash the image, not the file. eg, if I take a photo of my cat, and hash the file. Then remove my geo data from the exif - the hash no longer matches. It is still very clearly my cat, but cryptographic hashes don't match. This could be resizing the image, saving as png, mirroring/flipping it, etc.

The "perceptual hash" should be able to say "no, that's still the same image" while the file data has been entirely transformed.

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#82

Post by krrrh »

victor871129 wrote: post #2

These harmless generated images have a neuralhash equivalent to those provided in the NCMEC database submitted for testing. I repeat: Dont upload these harmless images to iCloud as Apple will assume its Child Porn (CSAM). Scripts were available on a GitHub repo but were removed because they may cause damage to others.

If Google Drive scans with the same database then how is your link working?

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#83

Post by salawat »

zepto wrote: post #56
nitrogen wrote: post #51

Earlier quoted context omitted.

They will not also match the visual derivative. How can you be certain, and what prevents a generated image from matching both?

I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM. What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like , in order to make a false positive of both. I.e. the attacker would ne…

There are only so many ways features can be permuted. The I'll defined nature of NN's requires the manuals step because of a neural hash collision.

My challenge to you is this: what stops this system from being abused for non child pornography purposes?

The answer is: nothing. That's what has people's knockers in a twist. It is a backdoor, invisibly crafted, waiting to be subverted by an abusive power that manages to get into an advantageous enough position.

Arguing that Apple's algorithms are fine misses the point. The behavior should not exist.

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#84

Post by zepto »

salawat wrote: post #83
zepto wrote: post #56

Earlier quoted context omitted.

I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM. What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like , in order to make a false positive of both. I.e. the attacker would ne…

There are only so many ways features can be permuted. The I'll defined nature of NN's requires the manuals step because of a neural hash collision. My challenge to you is this: what stops this system from being abused for non child pornography purposes? The answer is: nothing. That's what has people's knockers in a twist. It is a backdoor, invisibly crafted, waiting to be subverted by an abusive power that manages to…

> what stops this system from being abused for non child pornography purposes

You are changing the subject. That challenge has nothing at all to do with the OP’s false claims. They are still false.

Someone who can poison the database can indeed match non-child abuse images. The safeguard against that is that both Apple and NCMEC would need to conspire. This mechanism does not prevent such a conspiracy.

> Arguing that Apple's algorithms are fine misses the point.

Who is arguing that they are ‘fine’? I’m simply pointing out that they are not vulnerable in the way the poster claims them to be.

The images they have posted will not trigger the system.

If you want to debate the ethics of other aspects of what Apple is doing, there are plenty of threads elsewhere. This thread is about a false claim about a vulnerability in the system.

False claims about the vulnerabilities don’t help us to reason about what the risks actually are and detract from the moral or ethical debate.

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#85

Post by ugjka »

mindslight wrote: post #73
ugjka wrote: post #11

Not clicking on that link, don't want my G account go poof

Why are you reading Hacker News from the same nym/container/IP as a logged in Google account that you care about?

Yeah, I installed tor-browser later and checked the link

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#86

Post by zepto »

salawat wrote: post #80
zepto wrote: post #57

Earlier quoted context omitted.

That technique can’t fool Apple’s algorithm. For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point.

How do you know? Do you have access to Apple's algorithms and an account to generate enough hits, and access to the safety vouchers and decryption system to verify your assertions? I mean, if you're calling someone out, at least provide some evidence yourself. Short of a reproducible outcome, you're just as questionable in conclusion as the poster.

No. I have access to the published information on how the system works, and I have access to the poster’s claim.

The poster’s claim is false based on what they have said.

> you're just as questionable in conclusion as the poster.

Not correct. You don’t need evidence to disprove a claim that is logically false. The poster’s claim is logically false.

Here is a copy of the explanation I gave elsewhere:

—-

I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.

What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like, in order to make a false positive of both. I.e. the attacker would need a copy of the original CSAM, and the spoofed file would end up looking like it could be at least plausibly mistaken for that exact image.

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#87

Post by ac29 »

zepto wrote: post #86
salawat wrote: post #80

Earlier quoted context omitted.

How do you know? Do you have access to Apple's algorithms and an account to generate enough hits, and access to the safety vouchers and decryption system to verify your assertions? I mean, if you're calling someone out, at least provide some evidence yourself. Short of a reproducible outcome, you're just as questionable in conclusion as the poster.

No. I have access to the published information on how the system works, and I have access to the poster’s claim. The poster’s claim is false based on what they have said. > you're just as questionable in conclusion as the poster. Not correct. You don’t need evidence to disprove a claim that is logically false. The poster’s claim is logically false. Here is a copy of the explanation I gave elsewhere: —- I can be certa…

> I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.

Isnt this making the relatively huge assumption that humans and Apple's algorithms have the exact some opinion of what something "looks like"?

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#88

Post by zepto »

ac29 wrote: post #87
zepto wrote: post #86

Earlier quoted context omitted.

No. I have access to the published information on how the system works, and I have access to the poster’s claim. The poster’s claim is false based on what they have said. > you're just as questionable in conclusion as the poster. Not correct. You don’t need evidence to disprove a claim that is logically false. The poster’s claim is logically false. Here is a copy of the explanation I gave elsewhere: —- I can be certa…

> I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM. Isnt this making the relatively huge assumption that humans and Apple's algorithms have the exact some opinion of what something "looks like"?

> Isnt this making the relatively huge assumption that humans and Apple's algorithms have the exact some opinion of what something "looks like"?

No. The visual derivative is designed to be matchable by human inspection.

Even if that was not true, which it is, the poster’s claim would still be false, since the poster doesn’t have access to the source CSAM and therefore would not be able to produce the visual derivative regardless of whether I could visually inspect it.

Being able to see by inspection that they images don’t match CSAM is one of two independent ways in which the claim can be shown to be false.

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#89

Post by Applejinx »

Applejinx wrote: post #28
ReactiveJelly wrote: post #12

"Now you can block people in Drive. To prevent people from sharing unwanted files with you, ..." hahaha. What a coincidence, Google! So you got a hold of the neural hashes, and then used an error function and descent to generate images that match a 'hash'? It feels wrong to call them 'hashes' when they're so weak to pre-image attacks. They're not the same idea as cryptographic hashes at all. Also want to underline ho…

I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin. Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images. More info on how this is done?

Followup, since I can't edit: if my assumption isn't correct, well then, I stand corrected. I said in past tense, 'I assumed', and then asked for more info. That's not forthcoming, just a bunch of very upset assertions that of course I'm wrong and these things can't be reverse engineered from real porn.

I'm sure not interested in proving they can. Mind furnishing the info about how it's really done, then? Since according to you (for very obvious reasons) you can never compare these images to the source for the hashes, where did you extract the hashes from?

If you can so easily reverse engineer false positives from random data without ever seeing or using genuine porn to produce it, shouldn't you be disseminating this content as widely as you possibly can, rather than warning people about the danger of interacting with these false-positive images?

Still puzzled how and why this is being done. Are you trying to render Apple's system useless, or not?

Summary: I'm saying "there may be a way to take existing images that are illegal even to possess, and process them to obliterate the image while maintaining the hash. Is that what's being done here?" and the response is "AM NOT!!"

Re: Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn

#90

Post by Applejinx »

YokoSix wrote: post #37
Applejinx wrote: post #28

Earlier quoted context omitted.

I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin. Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images. More info on how this is done?

Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?

Also relevant question: if these images were not at all generated from illegal porn, but they connect to hashes being used to flag illegal porn, is the purpose of this exercise to generate methods to SWAT people over the internet?

As in, pursue a mechanism to get these onto somebody's computer in a way that they'll be backed up via iCloud (for instance, if a person's got their email account including trash folder backed up in iCloud, and you send them the pictures which they 'throw away' because it means nothing to them, placing the images in a trash folder in the mail preferences)

Is that (a) practical and (b) the intent of this exercise? Seeing as every question I've had here has led to karma burning I figured I'd double down and ask if the person doing this is trying to prepare a weapon for swatting people. There are times I respond to downvoting pressure to 'stop talking!' by getting more interested, which I'm sure is a common reaction among some hackers.

Post reply on HN