The deceptive PR behind Apple’s “expanded protections for children” • Hacker News Board

Re: The deceptive PR behind Apple’s “expanded protections for children”

#411

Post by farmerstan » Fri, Aug 13, 2021, 2:39 AM UTC

stetrain wrote: post #333

farmerstan wrote: post #326

Earlier quoted context omitted.

There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.

You don’t seem to have read what Apple has said on the issue so that feels a bit extreme. And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option. If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.

I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn

Re: The deceptive PR behind Apple’s “expanded protections for children”

#412

Post by simondotau » Fri, Aug 13, 2021, 2:40 AM UTC

sandworm101 wrote: post #398

simondotau wrote: post #388

Earlier quoted context omitted.

> your family photos ... could be construed as illegal in the hands of pervs, the images become child porn and can be added to the databases. It's not as simple as that. Photos in the NCMEC database are tagged based on the severity of their content. The categories are A1, A2, B1, B2. According to this[0] PowerPoint presentation, page 22: A = prepubescent minor B = pubescent minor 1 = sex act 2 = "lascivious exhibitio…

A1 is nowhere near the most extreme material with which police must deal. Not even close. This is called child abuse imagery for a reason. Sex acts are literally the entry level for A1.

"Sex act" is specifically what the "1" in A1 refers to. A1 does not cover images of child abuse unless they involve sex acts. Yes, even if these are "more extreme" in other ways.

The definitions are horrifyingly, depressingly, tragically very clear.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#413

Post by gonehome » Fri, Aug 13, 2021, 2:40 AM UTC

salawat wrote: post #371

gonehome wrote: post #118

Earlier quoted context omitted.

> "They've turned your device into a dragnet for content the powers that be don't like. It could be anything. They're not telling you" They're pretty explicitly telling us what it's for and what it's not for. > "And you're blindly trusting them to have your interests at heart, to never change their promise. You don't even know these people." You should probably get to work building your own phone, along with your own…

Yes. All the fabs and stuff we have now shoufd be devoted to implementing a surveillance state. This must happen. It cannot be any other way. This is what you sound like. The problem here isn't the tech. It's that Big Tech has deluded society into believing privacy and personal ownership of devices doesn't exist because it would inconvenice Big Tech. Law enforcement echoes it because they were spoiled by the brief pe…

I’m not making statements about the way things ought to be, I’m being pragmatic about the way things are.

We’re trusting a lot of the stack. Apple’s policy as described is reasonable. If you distrust it because of the things they could do, that same logic applies to the entire stack.

The nature of modern software distribution is that the majority of the stuff we use from centralized corporations is governed by policy, not technical capability or controls, and you don’t get to know the details.

You don’t own the OS you use, you don’t own the important parts of your phone.

This can be different. Decentralized applications via protocols are interesting (DeFi blockchain stuff like Audius or other apps on Ethereum). If the UX can get figured out.

Urbit is interesting too - if you want to actually own your stack, use Urbit: https://media.urbit.org/whitepaper.pdf

Outside of decentralized protocols you’re ultimately just trusting policy somewhere. It seems dumb to me to arbitrarily be upset at Apple’s policy here, when the specifics are reasonable (and allow for e2ee).

Re: The deceptive PR behind Apple’s “expanded protections for children”

#414

Post by devwastaken » Fri, Aug 13, 2021, 2:41 AM UTC

fortenforge wrote: post #137

querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

Today PhotoDNA, tomorrow NeuralNet smart AI. PhotoDNA is limited to hashes because performance 15 years ago, which makes it limited to known CSAM. However there is no doubt a push to pick up new CSAM content before it's ever distributed, which will absolutely introduce false positives.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#415

Post by stetrain » Fri, Aug 13, 2021, 2:45 AM UTC

danShumway wrote: post #354

grishka wrote: post #339

Earlier quoted context omitted.

Good luck if you have two screens with different DPIs.

Hello, user with two screens with two different DPIs (and two separate resolutions as well, and one of them a giant touch-screen drawing tablet). Nice to meet you, I use Arch btw. X11's handling of different DPIs is annoying but workable; there's a couple of different possible methods of handling it that have their own pros/cons. Per-monitor scaling is supported, but I personally don't like the way it's handled, so i…

So your answer about different DPIs is that you solve it by using a single DPI setting for both monitors?

And the “year of Linux on the desktop” meme isn’t about it not being a workable desktop or workstation OS. Of course it is and many people use it as such.

But it’s not a mainstream option that puts any kind of competitive pressure on the alternatives. You can’t go into Best Buy and walk out with a linux laptop/desktop that doesn’t have some proprietary layer (Chrome OS, Samsung, Google services) baked in.

So yes Linux works on the desktop, but “everyone should just use Linux” isn’t a realistic answer to issues like this one from Apple.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#416

Post by jhayward » Fri, Aug 13, 2021, 2:45 AM UTC

tylerhou wrote: post #273

FabHK wrote: post #178

Question: Would Apple report CSAM matches worldwide to one specific US NGO? That's a bit weird, but ok. Presumably they know which national government agencies to contact. Opinion: If Apple can make it so that a) the list of CSAM hashes is globally the same, independent of the region (ideally verifiably so!), and b) all the reports go only to that specific US NGO (which presumably doesn't care about pictures of Winni…

NCMEC is not exactly a normal NGO; it was opened with Reagan present and Congress frequently gives it funding. It also works with other government organizations on a frequent basis.

NCMEC isn't really an NGO; it is an agent of the US government, written in to US law.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#417

Post by [deleted] » Fri, Aug 13, 2021, 2:47 AM UTC

farmerstan wrote: post #399

simondotau wrote: post #395

Earlier quoted context omitted.

I'd be surprised if a baby in the bath—or indeed any legitmately innocent photograph taken by a parent—could ever be a perceptual match to images tagged as "A1" by NCMEC and other agencies. This is the most extreme of the extreme: prepubescent minors actively engaged in sex acts.

I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Things may work right now in 2021. Things will always be changing. New hashes will be introduced. New code will be introduce. New laws in different countries will be introduced. Now that Apple has introduced this technology that no other phone manufacturer has, it can and will be changed to decrease p…

[deleted]

Re: The deceptive PR behind Apple’s “expanded protections for children”

#418

Post by stetrain » Fri, Aug 13, 2021, 2:51 AM UTC

farmerstan wrote: post #411

stetrain wrote: post #333

Earlier quoted context omitted.

You don’t seem to have read what Apple has said on the issue so that feels a bit extreme. And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option. If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.

I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn

The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption.

I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data.

If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#419

Post by simondotau » Fri, Aug 13, 2021, 2:51 AM UTC

farmerstan wrote: post #399

simondotau wrote: post #395

Earlier quoted context omitted.

I'd be surprised if a baby in the bath—or indeed any legitmately innocent photograph taken by a parent—could ever be a perceptual match to images tagged as "A1" by NCMEC and other agencies. This is the most extreme of the extreme: prepubescent minors actively engaged in sex acts.

I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Things may work right now in 2021. Things will always be changing. New hashes will be introduced. New code will be introduce. New laws in different countries will be introduced. Now that Apple has introduced this technology that no other phone manufacturer has, it can and will be changed to decrease p…

> I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are.

Speak for yourself, because that didn't surprise me at all. When your corpus of "copyrighted" material is so utterly massive and almost entirely devoid of defined rules or boundaries, this kind of error is inevitable. If anything I'm more surprised that we haven't seen even more of these kinds of matches, like the sound of generic telephone ringtones, recordings of mechanised church bells, or machinery which performs highly uniform tasks, etc.

In the case of A1-categorised CSAM, the quantity of items is many orders of magnitude lower, the degree of technical curation will be orders of magnitude higher, and the thresholds for matches will be narrower. Is there a chance that the A1 corpus will have a few images that should have been classified as A2, B1 or B2? Yes. Is there a chance that it includes pictures of The Statue of Liberty or Westminster Tower? Almost certainly not.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#420

Post by devwastaken » Fri, Aug 13, 2021, 2:51 AM UTC

querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Eventually, yes. As we work more towards AI's and "smart" image recognition eventually there will be a system that will have these false positives on innocent images.

Current CSAM depends on humans to "verify" the imagery, this is something companies desperately want to get rid of, and so do the employees understandbly so. Nobody wants a job (99.99%) of comparing CSAM. It costs companies money in labor costs, and draws them bad PR when those employees inevitably develop permanent/semi-permanent mental health issues from it.

The only reason it hasn't happened yet is because a startup can't just start scanning CSAM. They need the blessing of the feds to do that, which requires political connections, and of course requires competing with companies that already have that blessing - something that politics prevents.

PhotoDNA and current CSAM scanning only gets known CSAM, but not new CSAM. The end goal is to detect CSAM before it's ever even distributed, to be "closer to the victim", rather than just those consuming it.

Even with current PhotoDNA you can generate hash collisions, which flag the image for review, and a real human compares the material. This is of course subject to change for the reasons stated above.

Secondarily, automatic scanning and ID'ing of imagery is how you can easily throw an FBI raid at someone. Apps like Telegram automatically download every image/video in the thread.

Ontop of that you can create images that appear different at differing resolutions. At one resolution a harmless meme, at another, CSAM. Meaning that you can again throw an FBI raid at someone using simple tricks.