Is macOS Look Up Destined for CSAM? • Hacker News Board

Re: Is macOS Look Up Destined for CSAM?

#31

Post by avazhi » Sun, Mar 20, 2022, 6:04 PM UTC

cmckn wrote: post #24

avazhi wrote: post #18

Earlier quoted context omitted.

CSAM is, by definition, photography or videography of something that has already happened. Therefore, quite literally, doing absolutely nothing about CSAM itself would result in no harm to any child, as the harm has already occurred. Now you're probably going to then cry about incentivising or normalising CSAM - but that's a different argument. And if you then try to argue that the normalisation of CSAM would somehow…

I’ve worked jobs in which I was exposed to this content regularly. It’s disturbing, sometimes extremely so. Just because someone does not abuse a child after viewing this content does not mean the content causes no harm to either individuals or our society at large. I don’t want to live in a world where CSAM is tolerated in order to keep pedophiles satiated.

First you couched your position as being about the children and harm to children; now you're talking about - as best I can tell - psychological harm to society writ large which you're asserting would occur based on your own experience. The part about CSAM being tolerated in order to keep pedos satiated seems like a non sequitur but honestly I don't really understand what you're trying to say, so... it doesn't sound like a very good faith discussion to me.

I do hope you get whatever support you need for something that's apparently affected you. Take care.

Re: Is macOS Look Up Destined for CSAM?

#32

Post by agildehaus » Sun, Mar 20, 2022, 6:06 PM UTC

JimDabell wrote: post #25

tylersmith wrote: post #8

Earlier quoted context omitted.

And if the filter thinks the image is positive for CSAM it sends it to Apple, correct? Otherwise there's literally no point.

> And if the filter thinks the image is positive for CSAM it sends it to Apple, correct? No. The logic was never “if there’s a match the image is uploaded”. The device never knows if there’s a match. Under the process Apple described, an extra packet of data is attached to every iCloud upload. If there are enough matches, Apple can decode those packets to get a low-res thumbnail, which they then check against a secon…

Why can't they just scan iCloud uploads on-server then? Why does anything need to be done on-device?

Re: Is macOS Look Up Destined for CSAM?

#33

Post by xanaxagoras » Sun, Mar 20, 2022, 6:06 PM UTC

JimDabell wrote: post #25

tylersmith wrote: post #8

Earlier quoted context omitted.

And if the filter thinks the image is positive for CSAM it sends it to Apple, correct? Otherwise there's literally no point.

> And if the filter thinks the image is positive for CSAM it sends it to Apple, correct? No. The logic was never “if there’s a match the image is uploaded”. The device never knows if there’s a match. Under the process Apple described, an extra packet of data is attached to every iCloud upload. If there are enough matches, Apple can decode those packets to get a low-res thumbnail, which they then check against a secon…

So in reply to your parent, the answer is yes. It sends a low resolution copy of the image to Apple with extra steps.

Re: Is macOS Look Up Destined for CSAM?

#34

Post by omginternets » Sun, Mar 20, 2022, 6:06 PM UTC

devwastaken wrote: post #15

cmckn wrote: post #11

Earlier quoted context omitted.

What’s the right number of lives to destroy from CSAM? There’s a middle ground between doing nothing and totalitarianism. “Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. It’s fair to be concerned about false positives and ensure the system handles such failures appropriately. It’s also fair to directly intervene in the widespread circulation of CSAM.

The middle ground is 0. The idea of equal jailed innocents to jailed criminals is mind numbing leaps of logic that is against the very ethos the united states was founded on.

I understand the sentiment and even share it to some extent, but we should really be arguing against the strongest possible version of claim, which is as follows: signal-detection theory tells us that there is a direct relationship between the number of correct-detections and false-alarms, and the only way to achieve 0 false-alarms is not to label anything a hit. In the present case, this means not prosecuting anyone, ever. That's probably not a solution you're happy with.

Therefore, the argument is one of degree. I agree with you that Apple's CSAM-detection is going too far, and this is what we should be articulating. Chanting "not even one" is not particularly convincing, nor sensible.

Re: Is macOS Look Up Destined for CSAM?

#35

Post by xanaxagoras » Sun, Mar 20, 2022, 6:13 PM UTC

agildehaus wrote: post #32

JimDabell wrote: post #25

Earlier quoted context omitted.

> And if the filter thinks the image is positive for CSAM it sends it to Apple, correct? No. The logic was never “if there’s a match the image is uploaded”. The device never knows if there’s a match. Under the process Apple described, an extra packet of data is attached to every iCloud upload. If there are enough matches, Apple can decode those packets to get a low-res thumbnail, which they then check against a secon…

Why can't they just scan iCloud uploads on-server then? Why does anything need to be done on-device?

The theory is they were going to use this tech to finally enable E2EE iCloud Photos and reactionary privacy absolutist psychopaths who didn't understand how it works -- such as myself -- made a big ruckus and spoiled everything.

Also at the point that the image hash matches, Apple thinks it is CSAM (and it probably is). Doing it locally lets them avoid storing it, which they definitely do not want to do.

Re: Is macOS Look Up Destined for CSAM?

#36

Post by omginternets » Sun, Mar 20, 2022, 6:13 PM UTC

amelius wrote: post #19

User23 wrote: post #10

What's the right number of lives to destroy over false positives from an algorithm? Is it some number other than zero? Why or why not?

You can ask the same question about self-driving cars.

Self-driving cars have the potential to reduce serious harm by a significant margin. Are you saying the same is true with Apple's CSAM-detection measures?

If so, how is curbing CSAM consumption going to prevent children from being raped, exactly? And I do mean exactly. The only arguments I have heard thus far appeal to some vague link between producers and consumers, predicated on the idea that CSAM producers are doing it for the celebrity/notoriety, or for financial profit. Both of these claims are highly suspect, and seem to rest on a confusion between trading CSAM online and paying traffickers for sex with children.

You may be correct, but it's going to take more than a superficial comparison to convince anyone.

Re: Is macOS Look Up Destined for CSAM?

#37

Post by JimDabell » Sun, Mar 20, 2022, 6:14 PM UTC

agildehaus wrote: post #32

JimDabell wrote: post #25

Earlier quoted context omitted.

> And if the filter thinks the image is positive for CSAM it sends it to Apple, correct? No. The logic was never “if there’s a match the image is uploaded”. The device never knows if there’s a match. Under the process Apple described, an extra packet of data is attached to every iCloud upload. If there are enough matches, Apple can decode those packets to get a low-res thumbnail, which they then check against a secon…

Why can't they just scan iCloud uploads on-server then? Why does anything need to be done on-device?

This scheme is obviously designed to work when Apple has no direct access to the photos. Because of this, there is a lot of speculation that Apple plans on making iCloud photos encrypted. This scheme would continue to work in that situation, whereas the on-server approach would fail. However that’s just speculation, Apple haven’t announced anything.

Re: Is macOS Look Up Destined for CSAM?

#38

Post by kaba0 » Sun, Mar 20, 2022, 6:16 PM UTC

version_five wrote: post #29

amelius wrote: post #19

Earlier quoted context omitted.

You can ask the same question about self-driving cars.

This is ignorant. There is lots of information about the safety of human driven cars, it is not a high bar to improve upon, and can be verified. The technology in question is introducing a new form of potentially ruinous statistical surveillance that didn't exist before.

Do you mean the terribly done statistics that collect data from uneventful motorways’ miles, while people automatically switch back to manual driving when a problematic situation arises, effectively filtering out any interesting data?

Re: Is macOS Look Up Destined for CSAM?

#39

Post by KarlKemp » Sun, Mar 20, 2022, 6:16 PM UTC

version_five wrote: post #29

amelius wrote: post #19

Earlier quoted context omitted.

You can ask the same question about self-driving cars.

This is ignorant. There is lots of information about the safety of human driven cars, it is not a high bar to improve upon, and can be verified. The technology in question is introducing a new form of potentially ruinous statistical surveillance that didn't exist before.

You’re ignoring the Trolley Problem of it all: is it moral to knowingly let uninvolved person X die if it saves the lives of Y and Z?

Fortunately, the policy choice at issue here isn’t one where there is definitive harm on the track, just risks that can be compared.

Re: Is macOS Look Up Destined for CSAM?

#40

Post by passivate » Sun, Mar 20, 2022, 6:16 PM UTC

version_five wrote: post #29

amelius wrote: post #19

Earlier quoted context omitted.

You can ask the same question about self-driving cars.

This is ignorant. There is lots of information about the safety of human driven cars, it is not a high bar to improve upon, and can be verified. The technology in question is introducing a new form of potentially ruinous statistical surveillance that didn't exist before.

>There is lots of information about the safety of human driven cars, it is not a high bar to improve upon, and can be verified.

That doesn't make sense. The 'average' driver gets into millions of accidents. I don't want a slight above average driver driving my family around, and I don't want slightly above average drivers around me - especially those who I can't communicate with - by honking, by shouting to get their attention, and algorithms who have no fear of their own life, etc. All software has bugs and I don't want my safety contingent on developers making mistakes. The self driving car must be orders of magnitude above the BEST human driver, and there must be punitive damages in place as a deterrent, to compensate aggrieved parties, etc. At present, self-driving cars are unproven dangerous technology that is being rightfully scrutinized.