Apple enabling client-side CSAM scanning on iPhone tomorrow • Hacker News Board

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#61

Post by ulfw » Thu, Aug 05, 2021, 4:13 AM UTC

I'm a little bit confused here and hope maybe some of you can clear this up.

My parents took lots of photos of me as a baby/small child. Say lying naked on a blanket or a naked 2yr old me in a kiddie pool in the summer in our backyard. Those are private photos and because it was the 1970s those were just taken with a normal non-digital camera. They were OBVIOUSLY never shared with others, especially outside immediate family.

Transform that into the 2020s and today these type of pictures would be taken with your iPhone. Would they now be classified as child pornography even though they weren't meant to be shared with anyone nor were they ever shared with anyone? Just your typical proud parent photo of your toddler.

Sounds a bit like a slippery slope, but maybe I am misunderstanding the gravity here. I'm specifically highlighting private "consumption" (parent taking picture of their child who happens to be naked as 1yr olds tend to be sometimes) vs "distribution" (parent or even a nefarious actor taking picture of a child and sharing it with third parties). I 100% want to eliminate child pornography. No discussion. But how do we prevent "false positives" with this?

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#62

Post by Shank » Thu, Aug 05, 2021, 4:15 AM UTC

I was under the impression that one of the reasons why these tools aren’t available for public download is because the hashes and system can be used to design defeat mechanisms? Doesn’t this mean that someone who has an image and a jail broken device can just watch the system, identify how the photo is detected, and modify it so that it doesn’t trip the filter?

PhotoDNA and systems like it are really interesting, but it seems like clientside scanning is a dangerous decision, not just from the privacy perspective. It seems like giving a CSAM detector and hashes to people is a really risky idea, even if it’s perfect and it does what it says it does without violating privacy.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#63

Post by jsjohnst » Thu, Aug 05, 2021, 4:15 AM UTC

drvdevd wrote: post #54

3pt14159 wrote: post #47

I'm really conflicted about this. For context, I deeply hate the abuse of children and I've worked on a contract before that landed 12 human traffickers in custody that were smuggling sex slaves across boarders. I didn't need to know details about the victims in question, but it's understood that they're often teenagers or children. So my initial reaction when reading this Twitter thread was "let's get these bastards…

Since you worked on an actual contract catching these sorts of people you are perhaps in a unique position to answer the question: will this sort of blanket surveillance technique in general but also in iOS specifically - actually work to help catch them?

I have direct knowledge of examples of where individuals were arrested and convicted of sharing CP online and they were identified because a previous employer I worked for used PhotoDNA analysis on all user uploaded images. So yeah, this type of thing can catch bad people. I’m still not convinced Apple doing this is a good thing, especially on private media content without a warrant, even though the technology can help catch criminals.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#64

Post by [deleted] » Thu, Aug 05, 2021, 4:16 AM UTC

0xy wrote: post #58

3pt14159 wrote: post #47

I'm really conflicted about this. For context, I deeply hate the abuse of children and I've worked on a contract before that landed 12 human traffickers in custody that were smuggling sex slaves across boarders. I didn't need to know details about the victims in question, but it's understood that they're often teenagers or children. So my initial reaction when reading this Twitter thread was "let's get these bastards…

The NCMEC database that Apple is likely using to match hashes, contains countless non-CSAM pictures that are entirely legal not only in the U.S. but globally. This should be reason enough for you to not support the idea. From day 1, it's matching legal images and phoning home about them. Increasing the scope of scanning is barely a slippery slope, they're already beyond the stated scope of the database.

[deleted]

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#65

Post by meowface » Thu, Aug 05, 2021, 4:16 AM UTC

0xy wrote: post #58

3pt14159 wrote: post #47

I'm really conflicted about this. For context, I deeply hate the abuse of children and I've worked on a contract before that landed 12 human traffickers in custody that were smuggling sex slaves across boarders. I didn't need to know details about the victims in question, but it's understood that they're often teenagers or children. So my initial reaction when reading this Twitter thread was "let's get these bastards…

The NCMEC database that Apple is likely using to match hashes, contains countless non-CSAM pictures that are entirely legal not only in the U.S. but globally. This should be reason enough for you to not support the idea. From day 1, it's matching legal images and phoning home about them. Increasing the scope of scanning is barely a slippery slope, they're already beyond the stated scope of the database.

Can you give more information about this? What kind of legal images might it match?

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#66

Post by voldacar » Thu, Aug 05, 2021, 4:18 AM UTC

0xy wrote: post #58

3pt14159 wrote: post #47

I'm really conflicted about this. For context, I deeply hate the abuse of children and I've worked on a contract before that landed 12 human traffickers in custody that were smuggling sex slaves across boarders. I didn't need to know details about the victims in question, but it's understood that they're often teenagers or children. So my initial reaction when reading this Twitter thread was "let's get these bastards…

The NCMEC database that Apple is likely using to match hashes, contains countless non-CSAM pictures that are entirely legal not only in the U.S. but globally. This should be reason enough for you to not support the idea. From day 1, it's matching legal images and phoning home about them. Increasing the scope of scanning is barely a slippery slope, they're already beyond the stated scope of the database.

Why would there be legal images in the db? do you have a source for that?

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#67

Post by FractalHQ » Thu, Aug 05, 2021, 4:18 AM UTC

echelon wrote: post #39

bigiain wrote: post #32

Earlier quoted context omitted.

> Stop. Using. Apple. But is there a realistically better alternative? Pinephone with a personally audited Linux distro? A jailbroken Android device with a non-stock firmware that you built yourself? A homebuilt RaspberryPi based device? A paper notepad and a film camera and an out of print street map?

Not really, and I'm not going to sway anyone deeply into the ecosystem. My hope is that those of you that share my viewpoint will call your legislators and demand regulations or a break up. There are forces of good within the DOJ that are putting together an antitrust case against Apple, and the more of us that lend our voices, the louder and more compelling the argument. The DOJ is really the last lever we have, and…

Too bad Apple and the rest of big money already have our legislators in their pockets.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#68

Post by pseudalopex » Thu, Aug 05, 2021, 4:21 AM UTC

ulfw wrote: post #61

I'm a little bit confused here and hope maybe some of you can clear this up. My parents took lots of photos of me as a baby/small child. Say lying naked on a blanket or a naked 2yr old me in a kiddie pool in the summer in our backyard. Those are private photos and because it was the 1970s those were just taken with a normal non-digital camera. They were OBVIOUSLY never shared with others, especially outside immediate…

The idea is it detects specific images humans classified. Not any unclothed child.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#69

Post by akersten » Thu, Aug 05, 2021, 4:22 AM UTC

ulfw wrote: post #61

I'm a little bit confused here and hope maybe some of you can clear this up. My parents took lots of photos of me as a baby/small child. Say lying naked on a blanket or a naked 2yr old me in a kiddie pool in the summer in our backyard. Those are private photos and because it was the 1970s those were just taken with a normal non-digital camera. They were OBVIOUSLY never shared with others, especially outside immediate…

As with all horribly-ill-defined laws, it depends how the judge is feeling that day and their interpretation of the accused's intent. If the case can be made that the images arouse inappropriate gratification, they can be deemed illegal.

If that sounds absurd - most laws are like that. For better or worse, there's a human who interprets the law, not a computer. It's unfortunate Apple is choosing to elect a computer as the judge here, for exactly concerns like yours.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#70

Post by 0xy » Thu, Aug 05, 2021, 4:23 AM UTC

wbl wrote: post #60

0xy wrote: post #58

Earlier quoted context omitted.

The NCMEC database that Apple is likely using to match hashes, contains countless non-CSAM pictures that are entirely legal not only in the U.S. but globally. This should be reason enough for you to not support the idea. From day 1, it's matching legal images and phoning home about them. Increasing the scope of scanning is barely a slippery slope, they're already beyond the stated scope of the database.

Could you say more about these legal photos? That's a pretty big difference from what I thought was contained in the DB.

If there are pictures recovered alongside CSAM but are not CSAM themselves they can be included in the database.

The thing I can publicly say is that the database is not strictly for illegal or even borderline imagery.

NCMEC try to keep the contents, processes and access to the database under wraps for obvious reasons.