Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is macOS Look Up Destined for CSAM?

eclecticlight.co

Post reply on HN
Print view View original

61–70 of 143 posts

Re: Is macOS Look Up Destined for CSAM?

#61

Post by azinman2 »

perihelions wrote: post #60
azinman2 wrote: post #58

Earlier quoted context omitted.

You mean one of the countries sales of devices just stopped in? And the other already was announced to be a US org? And you need the intersection of both?

- "And the other already was announced to be a US org?" Then one rogue employee in a US org could be sufficient to get selective root to every Apple device everywhere? That's easy for a nation-state adversary. Here's demonstrated examples: MBS had US-based moles in Twitter corporate spying on Khashoggi [0], and Xi had Chinese-based Zoom employees spying on dissidents in America [1]. [0] https://www.npr.org/2019/11/06…

You need the intersection of both, which this hypothetical doesn’t account for.

In terms of planting, it’s much easier to do that already across the many, many cloud services that secretly scan on the backend. Going a whole weird route just to get images into a hash database, and then the matching images onto the device, that then get independent human verification seems totally unnecessary if you’re a state agent. Why do something so complicated when there are easier routes to go?

Re: Is macOS Look Up Destined for CSAM?

#62

Post by noasaservice »

I'm fine with child (prepubescent) rapists whom are adults to be sentenced to death. If you were an accomplice to that, as a videographer or similar, I'm also OK with death.

(There's a really weird social area from 13-18, with the weirdness and illegality going away up at 18. Stuff in this realm, especially around 2 similar ages, gets very stupid. This is where you can get 2 16yo's sexting and being charged with CSAM of their own body. I'm avoiding this in this post.)

But what does this CSAM scanner do? It only catches already-produced pictures of CSAM. In other words, it's evidence of said crime. In no other area of criminal law is there a law against said evidence itself. And yes, given the statutory nature of these images (possession is criminal, even if you didnt' put them there), I'm not at all comfortable in charging people for simple possession.

Even if they have urges of liking age-inappropriate pornography and "CSAM", as long as they're not doing any physical actions of harming humans, I'd much rather them do so in their own bedroom alone.

Nor do I buy into the gateway theory that CSAM leads to production of CSAM by raping children. This smacks to me of the DARE drug propaganda and gateway theory (which is complete bullshit).

And, we also already have harder situations that have been deemed legal: SCOTUS stated that Japanese Manga Hentai featuring schoolgirls (obviously under 18, sometimes by quite a lot), are completely and 100% legal. Again, SCOTUS foscused on 1fa and the fact that no children were harmed in its production.

And that leads to what just happened a few days ago. With the Zelenskyy (badly done) deepfake, when can we expect 18yr women with very petite bodies, being deepfaked into 10-13 year olds? In those cases, we could attest that everyone in the production is of legal age and provided ongoing consent. Will this fall under the same as Hentai?

Tl;Dr: I'm for the criminal legalization of CSAM. I'm for death penalty for child rapists/child sexual assault. But this, I can see going very very bad, in easily overscoping CSAM to the "cause of the day".

Re: Is macOS Look Up Destined for CSAM?

#63

Post by mejutoco »

omginternets wrote: post #45
passivate wrote: post #42

Earlier quoted context omitted.

>You may be correct, but it's going to take more than a superficial comparison to convince anyone. Well you just hand-waved the idea the self-driving cars can reduce harm by a "significant" margin - based on what exactly?

If that isn't true, then yes, we should seriously consider abandoning the idea of self-driving cars. I don't understand, though. Are you saying that this is a reason to accept Apple's CSAM-detection?

> If that isn't true, then yes, we should seriously consider abandoning the idea of self-driving cars.

I wanted to share a (I found) controversial thought: Self-driving cars are the US response to trains. No taxes for railways are palatable, but private vehicles on special roads and profit to be made, great for the car industry. That, IMO, is what drives (pun intended) self-driven cars.

Re: Is macOS Look Up Destined for CSAM?

#64

Post by _fat_santa »

buildbuildbuild wrote: post #59
willcipriano wrote: post #50

Earlier quoted context omitted.

Come up with something like: X51!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-CHILD-ABUSE-CONTENT-TEST-FILE!$H+H* https://en.m.wikipedia.org/wiki/EICAR_test_file

I thought about this, but you're still stuck trusting the implementation unless you test with actual illegal data, which is often criminal and immoral to obtain. Example: How does a researcher test whether algorithmically-classified illegal imagery stored on user devices is being scanned and reported home to Apple's servers, and what those bounds of AI-classified criminality are? (presumably with respect to what is i…

> Example: How does a researcher test whether algorithmically-classified illegal imagery stored on user devices is being scanned and reported home to Apple's servers, and what those bounds of AI-classified criminality are? (presumably with respect to what is illegal in the user's jurisdiction)

I'm not an expert in AI so this might be totally off base but I feel like you would be able to use an "intersection" of sorts for this type of detection. You detect children and pornography, the children portion trains it for age recognition and the porn portion trains it to see sexual acts. Slap those two together and you've got CSAM detection.

Re: Is macOS Look Up Destined for CSAM?

#65

Post by avazhi »

rootusrootus wrote: post #57
avazhi wrote: post #18

Earlier quoted context omitted.

CSAM is, by definition, photography or videography of something that has already happened. Therefore, quite literally, doing absolutely nothing about CSAM itself would result in no harm to any child, as the harm has already occurred. Now you're probably going to then cry about incentivising or normalising CSAM - but that's a different argument. And if you then try to argue that the normalisation of CSAM would somehow…

> the harm has already occurred Circulating images of a minor child engaged in sexual abuse do not constitute an ongoing harm to that child? That's a fascinating viewpoint. > Apple and everybody else can fuck right off with this Orwellian shit. Right along with people who think child abuse images should be okay to keep as long as you aren't the one who made them.

First, no, I don't think continued circulation ex post is comparable to the harm that occurs at the time the abuse physically occurs. My own view is that whatever feelings flow from knowing that the images are 'circulating' isn't harm at all. Less personally, lingering negative effects from some event in the form of flashbacks or unpleasant memories are not new instances of harm as a matter of law (for whatever that's worth), and I think it goes beyond straining common sense to use the term 'harm' in that way.

But let's assume you're right.

You think that pedophiles won't find ways to share content even if every tech company in the world implemented this? You think pedophiles don't and won't have terabytes of child porn backed up on hard disks around the world that will be distributed and circulate for the next millennium and beyond, even if it has to be carried around on USBs or burned to CDs (which aren't exactly amenable to CSAM scanning), and then saved to offline computers? Put another way, even if the internet shut down tomorrow, plenty of pedophiles around the world would continue jacking off to those images and sharing them with their buddies - you don't need the internet for that.

Further, even if you could convince me that it's harmful in the sense that I understand the word, I'm not sure I'd ever be persuaded that the amount of harm could be sufficient to outweigh the harms that would result from the scanning itself.

Happy to listen, though.

Re: Is macOS Look Up Destined for CSAM?

#66

Post by School-Cotton »

vetinari wrote: post #12
cmckn wrote: post #11

Earlier quoted context omitted.

What’s the right number of lives to destroy from CSAM? There’s a middle ground between doing nothing and totalitarianism. “Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. It’s fair to be concerned about false positives and ensure the system handles such failures appropriately. It’s also fair to directly intervene in the widespread circulation of CSAM.

There's no really a middle ground; the right number is 0. It is better that ten guilty persons escape than that one innocent suffer. -- Blackstone's ratio

There’s a reason he said ten and not a million. This argument is absurd when taken to its maximalist conclusion.

Whether a rational person would accept chance X of wrongfully being convicted of a crime to decrease the chance of being a victim of crime by Y obviously depends on the values of X and Y.

Re: Is macOS Look Up Destined for CSAM?

#68

Post by mejutoco »

KarlKemp wrote: post #39
version_five wrote: post #29

Earlier quoted context omitted.

This is ignorant. There is lots of information about the safety of human driven cars, it is not a high bar to improve upon, and can be verified. The technology in question is introducing a new form of potentially ruinous statistical surveillance that didn't exist before.

You’re ignoring the Trolley Problem of it all: is it moral to knowingly let uninvolved person X die if it saves the lives of Y and Z? Fortunately, the policy choice at issue here isn’t one where there is definitive harm on the track, just risks that can be compared.

IMO It is more: is it immoral to let person X die justifying it with an unreasonable fetish for tech and its unrealized potential?

I am only half kidding :)

Re: Is macOS Look Up Destined for CSAM?

#69

Post by Brian_K_White »

buildbuildbuild wrote: post #59
willcipriano wrote: post #50

Earlier quoted context omitted.

Come up with something like: X51!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-CHILD-ABUSE-CONTENT-TEST-FILE!$H+H* https://en.m.wikipedia.org/wiki/EICAR_test_file

I thought about this, but you're still stuck trusting the implementation unless you test with actual illegal data, which is often criminal and immoral to obtain. Example: How does a researcher test whether algorithmically-classified illegal imagery stored on user devices is being scanned and reported home to Apple's servers, and what those bounds of AI-classified criminality are? (presumably with respect to what is i…

I always imagine aliens hearing about something like this and being stunned.

"How can data be illegal?"

"There are bad things, but how can you decide what is bad and show how it's bad without examining and discussing it?"

You can only go so far merely alluding to things. Somewhere the rubber has to meet the road and you have to have concrete data and examples of anything you need to study or make any sort of tools or policy about.

It's like parents not talking to kids about sex. You can avoid it most of the time because decorum, but if you take that to it's extreme you have just made your child both helpless and dangerous through ignorance.

Somewhere along the way, you have to explicitly wallow directly in the mess of stuff you seek to avoid most of the time. That "seek to avoid" can only ever be "most of the time". It's insane and counter-productive to try to see that "most of the time" as an incomplete job and improve that to 100%.

I guess in this case there will eventually be some sort of approved certified group. A child porn researcher or investigator license. Cool. Cops with special powers never abuse them, and inhibiting study to a select few has always yielded the best results for any subject, and a dozen approved good guys can easily stay ahead of the world of bad guys.

Re: Is macOS Look Up Destined for CSAM?

#70

Post by sebzim4500 »

Spooky23 wrote: post #48
agildehaus wrote: post #32

Earlier quoted context omitted.

Why can't they just scan iCloud uploads on-server then? Why does anything need to be done on-device?

They almost certainly do. Every major provider does, and even some enterprises. The whole point of the CSAM stuff was that it would allow for end to end encryption while not turning Apple’s ecosystem into preferred tool of child pornographers. Apple poorly communicated the feature, then the EFF put out a deliberately misguided written hitpiece that conflated parental controls with CSAM, and started an online freak ou…

I think that's a pretty bad summary of the concerns that were raised. Sure they are scanning your files on icloud, but there is a 100% reliable way to prevent that: just don't upload them.

In their proposal they would scan your files on device, which is fundamentally different. Initially they would not run the scanning when icloud upload was disabled but how long would that last for?

Post reply on HN