Apple enabling client-side CSAM scanning on iPhone tomorrow • Hacker News Board

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#541

Post by saagarjha » Thu, Aug 05, 2021, 2:14 PM UTC

toxik wrote: post #275

google234123 wrote: post #125

Earlier quoted context omitted.

How hard would it be to create a valid image that matches some 128bit hahs

If the details of the "hashing" scheme used is publicized, I imagine it will be near trivial. It's a long-standing problem in computer vision, to find a digital description of an image such that two similar images compare equal or at least similar. State-of-the-art for this field is deep learning, and a /huge/ problem with the DL approach is that you can generate adversarial examples. So for example, a picture of a t…

Since this algorithm presumably runs on-device, I am sure it won’t be long before someone has reverse engineered it…

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#542

Post by raxxorrax » Thu, Aug 05, 2021, 2:15 PM UTC

neycoda wrote: post #486

GeekyBear wrote: post #437

Earlier quoted context omitted.

Have people already forgotten that Microsoft implemented the tech to routinely scan your cloud storage a decade ago? >The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who…

I'm OK with software generating signatures from cloud drive images to eliminate child porn pictures and catch pedophiles.

Have you ever had the impression that special interests are much more interested in copyright violations than child pornography? Or that it might extend to memes that sabotages carefully crafted propaganda? I do have that impression a lot.

I don't step in the lowest parts of the internet hell, but I am also not very picky about it. I have never encountered child pornography in 10 years of almost pathological internet usage.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#543

Post by raxxorrax » Thu, Aug 05, 2021, 2:17 PM UTC

gjulianm wrote: post #385

xyzal wrote: post #326

Earlier quoted context omitted.

The fact that even the 'smart' people from HN can't wait for their new M1 laptop to arrive convinced me that humans are a lost cause.

This is a moot point unless you always verify and check all hardware and software that you use, including communications devices.

Not at all. Here we have a manufacturer that thinks he would be allowed to scan the contents of your machine. If you scan a machine, you can read everything on the machine.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#544

Post by OJFord » Thu, Aug 05, 2021, 2:17 PM UTC

disiplus wrote: post #217

jsjohnst wrote: post #63

Earlier quoted context omitted.

I have direct knowledge of examples of where individuals were arrested and convicted of sharing CP online and they were identified because a previous employer I worked for used PhotoDNA analysis on all user uploaded images. So yeah, this type of thing can catch bad people. I’m still not convinced Apple doing this is a good thing, especially on private media content without a warrant, even though the technology can he…

now im afraid, i have two young children < 5 years old. i have occasionally took pictures of them naked with some bumps on the skin or mosquito bite and sent them to my wife over whatsapp to look at and decide do we need to send them to doctor, do i have to fear now that i will be marked as distributing CP.

You would only be flagged if the photos (' hashes) were added as part of some investigation right? So you only have to fear for your criminal record in the event that an actual criminal gets ahold of your indecent (in their hands!) photographs. In which eventuality you might be (relatively to not, but still leaked) glad they'd been discovered and arrested etc. assuming your good name could be cleared.

Just playing devil's advocate, my gut (and I think even considered) reaction is in alignment with surely just about the whole tech industry: it's over-reach (if they're not public images).

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#545

Post by GeekyBear » Thu, Aug 05, 2021, 2:18 PM UTC

raxxorrax wrote: post #542

neycoda wrote: post #486

Earlier quoted context omitted.

I'm OK with software generating signatures from cloud drive images to eliminate child porn pictures and catch pedophiles.

Have you ever had the impression that special interests are much more interested in copyright violations than child pornography? Or that it might extend to memes that sabotages carefully crafted propaganda? I do have that impression a lot. I don't step in the lowest parts of the internet hell, but I am also not very picky about it. I have never encountered child pornography in 10 years of almost pathological internet…

Scanning for DRM violations was one of the use cases mentioned in the article linked above.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#546

Post by gjulianm » Thu, Aug 05, 2021, 2:19 PM UTC

fsflover wrote: post #499

gjulianm wrote: post #490

Earlier quoted context omitted.

> This is how Signal and Matrix appeared and became (relatively) famous. And what happens when another app comes and says that "it's secure" and people start using it instead of Signal or Matrix? What happens if Signal starts requiring some payments (running servers is not free) and people move to other apps? Maybe those other apps are open source and federated, but the federation protocol is found later to have a ba…

> And what happens when another app comes and says that "it's secure" and people start using it instead of Signal or Matrix? First, early adopters come and verify it. They bring their friends. If it's really secure and they find no serious bugs, more people join. Then, a bridge is created between the services. > What happens if Signal starts requiring some payments (running servers is not free) and people move to oth…

> First, early adopters come and verify it. They bring their friends. If it's really secure and they find no serious bugs, more people join. Then, a bridge is created between the services.

That's quite the optimistic path. What if the app starts being used by teenagers, for example? Or by people with less technical abilities?

> This is a problem with a non-federated protocol actively fighting against third-party apps and servers.

Federated services still need to pay for their servers.

> Such backdoor will be quick and easy to fix

Again, pretty optimistic on that.

> and to verify that it's fixed. Unlike with Apple's Pegasus. No system is ever 100% secure.

Pegasus was external malware. What makes you think a Pegasus for federated servers or open source phones can't exist?

> Users are typically very slow to move. See Whatsapp & Facebook. But what's your point?

Security research takes time, probably more time than users need to move from apps.

> There is such legislation already in Europe: GDPR.

And GDPR has accomplished way more in way less time than technical solutions. I wonder why.

> Unfortunately it cannot dramatically change the industry quickly, because of the monopolies and network effects.

Don't those monopolies and network effects affect the technical solutions you propose too?

My point is that of course you need good technical solutions, but just those by themselves are useless, because most people don't have the time and knowledge to reliably distinguish which ones are good and which ones are bad (and "good" and "bad" are relative too), and other differential features (price, capabilities, ease of use) that are easier to notice will weigh more on their decisions.

This is not a problem unique to tech and privacy. Food security, climate, building safety... almost everything you buy has had the similar issue of how to have "things done right" where deciding whether it's done right or not is hard for most people. Almost everything has been solved (or almost solved) with regulation, and just "better products" haven't been enough.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#547

Post by OJFord » Thu, Aug 05, 2021, 2:23 PM UTC

meowface wrote: post #313

jsjohnst wrote: post #129

Earlier quoted context omitted.

> Simple nudity does not count inherently. That’s not entirely true. If a police officer finds you in possession of a quantity of CP, especially of multiple different children, you’ll at least be brought in for questioning if not arrested/tried/convicted, whether the images were sexualized or not. > nor would it ever end up in a database That’s a bold blanket statement coming from someone who correctly argued that NC…

I believe both you and the other poster, but I still haven't seen anyone give an example of a false positive match they've observed. Was it an actual image of a person? Were they clothed? etc. It's very concerning if the fuzzy hash is too fuzzy, but I'm curious to know just how fuzzy it is.

There are examples (from the OP, but in reply tweets) in the submission.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#548

Post by LinuxBender » Thu, Aug 05, 2021, 2:24 PM UTC

Does this not encourage spawning a new arms race? New or modified apps that randomly change hashes of multimedia files as they are stored? If the CSAM DB is just simple hashes like sha256, md5/md4, etc.. then evading detection would be trivial. Or would Apple block applications that could rewrite randomized data into files? People don't have to be against CSAM to dislike something scanning their devices and many developers love puzzle challenges. I assume perhaps incorrectly that whatever app is doing the scanning could potentially also accept additional hash DB's, allowing Apple to enable categories to detect on per region. One of the iPhone emulators should facilitate reverse engineering the application.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#549

Post by hprotagonist » Thu, Aug 05, 2021, 2:26 PM UTC

raxxorrax wrote: post #530

snarf21 wrote: post #500

Earlier quoted context omitted.

Great, so what's the solution? What are you doing to fix it? Do you roll your own silicon? Do you grow your own food (we have no idea what someone could be putting in it)? Are you completely off-grid? Or are you as completely dependent on society writ large as everyone else? Making holier than thou comments about everyone else being sheep isn't helpful or thought provoking. Offer an alternative if it is a bad one (lo…

> Great, so what's the solution? Seriously? Perhaps heed the warnings? Whenever Apple tightened the reigns, thousand of apologists came to their defense. I wouldn't even have minded if they kept their obedience to personal decisions. But they extended their enlightenment to others.

> Perhaps heed the warnings?

And then take what actions, exactly? “Guys this is trouble” is …fine, but without “and we should therefore do”, it’s just kind of spitting into the wind.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#550

Post by jaywalk » Thu, Aug 05, 2021, 2:28 PM UTC

oauea wrote: post #414

onetimeusename wrote: post #141

Earlier quoted context omitted.

The database seems legally murky. First of all, who would want to actually manually verify that there aren't any images in it that shouldn't be? If the public can even request to see it, which I doubt, would you be added to a watch list of potentially dangerous people or destroy your own reputation? Who adds images to it and where do they get those images from? My point is that we have no way to verify the database w…

Not an American, but shouldn't you be able to FOIA this?

The NCMEC, who manages the CSAM database, is a private organization.