Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is macOS Look Up Destined for CSAM?

eclecticlight.co

Post reply on HN
Print view View original

51–60 of 143 posts

Re: Is macOS Look Up Destined for CSAM?

#51

Post by passivate »

omginternets wrote: post #45
passivate wrote: post #42

Earlier quoted context omitted.

>You may be correct, but it's going to take more than a superficial comparison to convince anyone. Well you just hand-waved the idea the self-driving cars can reduce harm by a "significant" margin - based on what exactly?

If that isn't true, then yes, we should seriously consider abandoning the idea of self-driving cars. I don't understand, though. Are you saying that this is a reason to accept Apple's CSAM-detection?

Oh, I'm with you on the CSAM. I just wanted clarity about the self-driving cars, its a pet peeve of mine.

Re: Is macOS Look Up Destined for CSAM?

#52

Post by aunty_helen »

xanaxagoras wrote: post #30
dev_tty01 wrote: post #3

Earlier quoted context omitted.

No. Why would you think that? It goes against everything they have stated and the designs of the software. They are heavily focused on keeping all of that on device.

I don't think it's far fetched at all, that they'd do that without mentioning it. It certainly phones home when you open Preview [1], what's another little ping when you're looking at CSAM or whatever else they've been instructed to look for? The recent debacle made it clear enough to me that the their privacy reputation is little more than carefully curated marketing, and they're likely under tremendous pressure fro…

There is some reprieve by using a good firewall or even an off device firewall.

However with many of these services if you try to kill them, they come back. If you delete them sometimes it will literally break your OS.

Example, if you remove the ocsp daemon, you can’t start any program on your computer.

Re: Is macOS Look Up Destined for CSAM?

#53

Post by rootusrootus »

version_five wrote: post #27
buildbuildbuild wrote: post #21

How does someone truly test how this feature is being used without possessing illegal content? This is a nearly-impossible area to research. Frightening. (edit: I'm of course referring to possessing anti-Putin memes) (sarcasm)

By test it, do you mean see if the police show up at your door? If you know how it works, you just need a list of hashes and a way to find a collision which I believe exists. Otherwise, you're really just highlighting the problem with all closed source software, you don't really have a way to check what it does so you have to trust the vendor.

We already know that a hash collision doesn't get far enough to involve police showing up at your door, so a full test would take something more substantial.

Re: Is macOS Look Up Destined for CSAM?

#54

Post by passivate »

mdoms wrote: post #49
passivate wrote: post #42

Earlier quoted context omitted.

>You may be correct, but it's going to take more than a superficial comparison to convince anyone. Well you just hand-waved the idea the self-driving cars can reduce harm by a "significant" margin - based on what exactly?

1.35 million deaths per year caused by car accidents. That doesn't even account for the people who survive, but are maimed.

Self-driving cars are unproven tech. Just pointing to the harm that humans contribute towards doesn't mean much. Soldiers also accidentally kill innocent civilians - are you in favor of AI-drones and AI-soldiers??

Re: Is macOS Look Up Destined for CSAM?

#55

Post by ggreer »

amelius wrote: post #19
User23 wrote: post #10

What's the right number of lives to destroy over false positives from an algorithm? Is it some number other than zero? Why or why not?

You can ask the same question about self-driving cars.

Self-driving cars substitute for human-driven cars, which currently kill over a million people a year. If the first mass-adopted self-driving cars have half the fatality rate per mile of human-driven cars, then slowing their rollout by a day causes 1,800 deaths. Current prototype self-driving vehicles already have a lower fatality rate per mile than human-driven vehicles. Obviously this isn't an apples-to-apples comparison since current self-driving cars are constrained to certain locations and weather conditions, but if the goal is to minimize deaths, then we should be more gung-ho about this technology than we currently are.

In contrast, CSAM scanning substitutes for... I'm not sure what. In addition to the risk of false positives, there's also the risk that the scanning technology will be used for other purposes in the future. I could easily see governments forcing Apple to scan everyone's hard drives for hate speech, 3d models of prohibited objects (such as gun parts), or communications sympathetic to certain groups. Once that door is cracked open, there is no closing it.

Re: Is macOS Look Up Destined for CSAM?

#56

Post by perihelions »

buildbuildbuild wrote: post #21

How does someone truly test how this feature is being used without possessing illegal content? This is a nearly-impossible area to research. Frightening. (edit: I'm of course referring to possessing anti-Putin memes) (sarcasm)

Obviously, definitionally, it's impossible to verify that server-side logic isn't doing something evil. (Local homeomorphic protocols count, when the secret logic is imported from remote servers).

This is one reason FOSS is actually-important and actually-relevant. Isn't it valid to know exactly what your personal computer is doing, to be able to trust your own possessions? Richard Stallman was *never* crazy; his understanding of these issues is so cynical as to be shrill and off-putting, but that's well-calibrated to the severity of the issues at stake.

You joke about anti-Putin memes. Here's a thought for well-calibrated cynics: Apple solemnly swears its hashes are attested by at least two independent countries. Russia and Belarus are two independent countries.

Re: Is macOS Look Up Destined for CSAM?

#57

Post by rootusrootus »

avazhi wrote: post #18
cmckn wrote: post #11

Earlier quoted context omitted.

What’s the right number of lives to destroy from CSAM? There’s a middle ground between doing nothing and totalitarianism. “Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. It’s fair to be concerned about false positives and ensure the system handles such failures appropriately. It’s also fair to directly intervene in the widespread circulation of CSAM.

CSAM is, by definition, photography or videography of something that has already happened. Therefore, quite literally, doing absolutely nothing about CSAM itself would result in no harm to any child, as the harm has already occurred. Now you're probably going to then cry about incentivising or normalising CSAM - but that's a different argument. And if you then try to argue that the normalisation of CSAM would somehow…

> the harm has already occurred

Circulating images of a minor child engaged in sexual abuse do not constitute an ongoing harm to that child? That's a fascinating viewpoint.

> Apple and everybody else can fuck right off with this Orwellian shit.

Right along with people who think child abuse images should be okay to keep as long as you aren't the one who made them.

Re: Is macOS Look Up Destined for CSAM?

#58

Post by azinman2 »

perihelions wrote: post #56
buildbuildbuild wrote: post #21

How does someone truly test how this feature is being used without possessing illegal content? This is a nearly-impossible area to research. Frightening. (edit: I'm of course referring to possessing anti-Putin memes) (sarcasm)

Obviously, definitionally , it's impossible to verify that server-side logic isn't doing something evil. (Local homeomorphic protocols count, when the secret logic is imported from remote servers). This is one reason FOSS is actually-important and actually-relevant. Isn't it valid to know exactly what your personal computer is doing, to be able to trust your own possessions? Richard Stallman was *never* crazy; his un…

You mean one of the countries sales of devices just stopped in? And the other already was announced to be a US org? And you need the intersection of both?

Re: Is macOS Look Up Destined for CSAM?

#59

Post by buildbuildbuild »

willcipriano wrote: post #50
buildbuildbuild wrote: post #21

How does someone truly test how this feature is being used without possessing illegal content? This is a nearly-impossible area to research. Frightening. (edit: I'm of course referring to possessing anti-Putin memes) (sarcasm)

Come up with something like: X51!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-CHILD-ABUSE-CONTENT-TEST-FILE!$H+H* https://en.m.wikipedia.org/wiki/EICAR_test_file

I thought about this, but you're still stuck trusting the implementation unless you test with actual illegal data, which is often criminal and immoral to obtain.

Example: How does a researcher test whether algorithmically-classified illegal imagery stored on user devices is being scanned and reported home to Apple's servers, and what those bounds of AI-classified criminality are? (presumably with respect to what is illegal in the user's jurisdiction)

Testing by using a test phrase, like in a spam context, is inadequate here as a scanning system can trivially be architected to pass those publicly-known tests, while still overreaching into people's personal files and likely miscategorizing content and intent.

If a user connects via a VPN to Russia for whatever reasons, does their personal content start getting reported to that country's law enforcement by their notion of what is illegal?

Parents often have all sorts of sensitive photos around which are not held with exploitative intent. "Computer says arrest."

Re: Is macOS Look Up Destined for CSAM?

#60

Post by perihelions »

azinman2 wrote: post #58
perihelions wrote: post #56

Earlier quoted context omitted.

Obviously, definitionally , it's impossible to verify that server-side logic isn't doing something evil. (Local homeomorphic protocols count, when the secret logic is imported from remote servers). This is one reason FOSS is actually-important and actually-relevant. Isn't it valid to know exactly what your personal computer is doing, to be able to trust your own possessions? Richard Stallman was *never* crazy; his un…

You mean one of the countries sales of devices just stopped in? And the other already was announced to be a US org? And you need the intersection of both?

- "And the other already was announced to be a US org?"

Then one rogue employee in a US org could be sufficient to get selective root to every Apple device everywhere? That's easy for a nation-state adversary. Here's demonstrated examples: MBS had US-based moles in Twitter corporate spying on Khashoggi [0], and Xi had Chinese-based Zoom employees spying on dissidents in America [1].

[0] https://www.npr.org/2019/11/06/777098293/2-former-twitter-em...

[1] https://www.justice.gov/opa/pr/china-based-executive-us-tele...

That second example is topical: the Chinese state used their Zoom assets to attempt to frame Americans for CSAM possession.

- "As detailed in the complaint, Jin’s co-conspirators created fake email accounts and Company-1 accounts in the names of others, including PRC political dissidents, to fabricate evidence that the hosts of and participants in the meetings to commemorate the Tiananmen Square massacre were supporting terrorist organizations, inciting violence or distributing child pornography. The fabricated evidence falsely asserted that the meetings included discussions of child abuse or exploitation, terrorism, racism or incitements to violence, and sometimes included screenshots of the purported participants’ user profiles featuring, for example, a masked person holding a flag resembling that of the Islamic State terrorist group."

Post reply on HN