Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

wired.com

Post reply on HN
Print view View original

301–310 of 336 posts

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#301

Post by nottorp »

stale2002 wrote: post #300
nottorp wrote: post #260

Earlier quoted context omitted.

I don't see any reference to child porn there. Who decides what's obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable? Especially otherwise objectionable?

> Who decides Judges and juries. And I can assure you that every single judge in the USA, and almost every single member of a jury would decide that CSAM is obscene. Thats how the law works. We have tons of laws that use general words like this, and trying to be "clever" usually just results in a lost court case or prison time for the person who thinks they found a loophole.

What else is obscene? And why would I have to accept an US jury's opinion?

Also you didn't define 'otherwise objectionable'.

For example what I think is excessive violence may be considered normal by a jury in the US...

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#302

Post by networkchad »

tomjen3 wrote: post #125
figlett wrote: post #10

Earlier quoted context omitted.

> So despite looking a bit fishy at first, this doesn't seem to come from a christofascist group. Why would you assume this in the first place?

Because when they couldn't win the war on porn, some right Christians decided to cloak their attack in "concerns" of "abuse". See project Excedus. Of course it has nothing to do with abuse and everything to do with their attempts to keep people from seeing pixels of other people having sex. Backpage was shut down despite being good at removing underage and trafficed women - which meant that sex workers had to find ot…

[dead]

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#303

Post by robertoandred »

olliej wrote: post #255
robertoandred wrote: post #236

Earlier quoted context omitted.

How exactly would you be able to "filter" LGBT content? I don't think you understand how this system would've worked.

Hypothetically you have hashes for two people of gender X (lets be honest, based on popularity of different types of porn two men). This is not meaningfully different from opaque hash of "CSAM". But you're missing the point: Step 1. generate some opaque hash of the "semantics" of an image Step 2. compare those hashes to some list of hashes of "CSAM", which again fundamentally cannot be audited Step 3. report any hits…

Except this system wouldn't have looked at "semantics". You can't simply match a hash of a cross or star or flag, you have to match a specific photograph. Which photograph do you use?

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#304

Post by dang »

glogla wrote: post #34
gnfargbl wrote: post #9

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.” Both of th…

[flagged]

Please don't cross into personal attack.

https://news.ycombinator.com/newsguidelines.html

Edit: we've had to warn you about this kind of thing many times before:

https://news.ycombinator.com/item?id=33476972 (Nov 2022)

https://news.ycombinator.com/item?id=30612118 (March 2022)

https://news.ycombinator.com/item?id=26857565 (April 2021)

https://news.ycombinator.com/item?id=21678059 (Dec 2019)

https://news.ycombinator.com/item?id=18550058 (Nov 2018)

We eventually have to ban accounts that won't stop breaking the rules. I don't want to ban you, so if you'd please review https://news.ycombinator.com/newsguidelines.html and stick to them, we'd appreciate it.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#305

Post by manxman »

stale2002 wrote: post #299
FabHK wrote: post #294

Earlier quoted context omitted.

1. What you’re asking for (“The option … where the device encrypts files before uploading them to iCloud, iCloud may scan the encrypted bits anyway to do their legal duty, and that's the end of the story.”) is impossible. 2. The division you envisage (“The software running on the phone should be representing the owner of the phone, period.”) is wishful thinking. Do you think the JavaScript in your browser does only t…

> The division you envisage (“The software running on the phone should be representing the owner of the phone, period.”) is wishful thinking. In this specific case it is not wishful thinking. The feature got scrapped. Users and people who support privacy won.

You sure about that? Like really sure? Like you have definitive evidence that this assertion is true. Or are you placing faith in the news you read?

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#306

Post by nullc »

gruturo wrote: post #23

> "Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit" > "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types." Yes, and it was patently obviou…

Apple had a bunch of technical obfuscation that may have convinced them that they'd engineered their way out of a tough spot. "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should."

When it was made public people shot all sorts of holes both in the technical mechanisms themselves as well as their insufficiency even if they all worked as intended.

We should have some sympathy-- they did have some neat technology. But neat technology isn't enough.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#307

Post by nullc »

gnfargbl wrote: post #9

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.” Both of th…

> The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to.

You mean at rates greater than in the 1970s when it was legal in the US and there were a bunch of commercial publications creating and distributing it (prior to the "Protection of Children Against Sexual Exploitation Act" in 1977 (which targeted the production) and "Child Protection and Obscenity Enforcement Act" in 1988 (which targeted the distribution))? I find that doubtful.

> Does the industry feel that it has any responsibility at all to do so?

Why should it? Should every home builder and landlord install a camera in every bedroom to stream back to the government to be sure that no children are being molested? Why not? It's technologically possible today.

The fact that garbage people do garbage things to vulnerable people doesn't mean that the entire world must be commandeered to stop it.

E2E encryption has replaced a lot of in person meeting that also would have been secure against dragnet surveillance. We shouldn't lose our privacy just because it became technically possible to take it away, no different than the cameras in bedrooms mentioned above.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#308

Post by godelski »

extraduder_ire wrote: post #278
withinboredom wrote: post #259

Earlier quoted context omitted.

This feature is back in iOS 17, but it doesn’t do notifications and can be enabled per-device. It basically just blurs nsfw photos when they’re on the screen. It’s great when on a business trip and sexting with the SO. You can open your messages in front of your coworkers without having to worry about the messages being open from the night before.

I would be far too worried about false-negatives to do that. Especially as they get rarer, and I'd become more confident about loading up images in public. Big fan of keeping different forms of content separate.

I'm a bit confused here. This seems like failure analysis gives us a clear winning direction. When the system fails: would you rather a nude image show clear or a non-nude image be blurred? I think 99% of people would rather have the latter. There is little risk to a normal image being falsely flagged but the outcome is likely far worse when a nude image is not censored.

It really surprises me a lot of times how little failure analysis is done in software engineering considering it is basically the cornerstone of most physical engineering. It is critical that you design systems to fail in certain ways. That is error messaging... (Also see Blackstone's Ratio for an example in law)

Different forms for different content is a nice answer, but you also have to remember that we're talking a product for the masses. So while I don't think your answer is wrong, it is.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#309

Post by stale2002 »

nottorp wrote: post #301
stale2002 wrote: post #300

Earlier quoted context omitted.

> Who decides Judges and juries. And I can assure you that every single judge in the USA, and almost every single member of a jury would decide that CSAM is obscene. Thats how the law works. We have tons of laws that use general words like this, and trying to be "clever" usually just results in a lost court case or prison time for the person who thinks they found a loophole.

What else is obscene? And why would I have to accept an US jury's opinion? Also you didn't define 'otherwise objectionable'. For example what I think is excessive violence may be considered normal by a jury in the US...

> What else is obscene?

Quite a lot of things may or may not fall under that definition. Thats how the law works.

> And why would I have to accept an US jury's opinion?

Well, because they and judges are the ones empowered by the government monopoly on violence to judge the law and then have it be enforced, thats why.

Ignore the law at your own peril. But anyway, even if you did ignore the law, this doesn't have anything to do with you.

This is about companies immunity from prosecution. So, even if you disagree with the law, those companies are still immune under section 230, for good faith efforts to remove obscene content.

> Also you didn't define 'otherwise objectionable'.

It would be defined as whatever judges and juries define it as. I don't define it. Instead it is defined by those people.

> For example what I think

If you are not a judge or currently on a jury, then what you think is irrelevant.

The law is not computer code. Instead, it is interpreted by humans. And that is the case for basically all of law.

Re: Apple clarifies why it abandoned plan to detect CSAM in iCloud photos

#310

Post by spacedcowboy »

kobalsky wrote: post #232
theshrike79 wrote: post #42

Earlier quoted context omitted.

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

> but the internet got a collective hissy-fit of intentionally misunderstanding the feature how was it misunderstood? your device would scan your photos and notify apple or whoever if something evil was found. wasn't that what they were trying to do?

Your device would scan your photo at the point of you uploading it to the cloud and then it could encrypt it before sending it to the cloud. That meant that Apple's cloud servers didn't need to be able to scan it to comply with US Govt "recommendations" for cloud providers.

Whereas right now all the other cloud providers just send the photo as-is and scan it on the cloud servers.

With Apple's approach, the cloud servers don't get to look at every single one of your photos like cloud vendors do today, scanning happens within the privacy of your own phone, and only known-kiddy-porn signatures are flagged.

Apple came up with a way to make things way more private, but the concept of your own device working "against" you if you happen to be a pedophile was too much of a leap.

Post reply on HN