Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The deceptive PR behind Apple’s “expanded protections for children”

piotr.is

Post reply on HN
Print view View original

221–230 of 595 posts

Re: The deceptive PR behind Apple’s “expanded protections for children”

#221

Post by GuB-42 »

908B64B197 wrote: post #170
querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Wait until someone manages to create an image (white noise) that's a hash collision for anything in that database. And then starts spamming random strangers via airdrop. Enjoy explaining why your mugshot and arrest record had these charges attached to it! (Actually, in this case the prosecution would probably use the other pictures on the phone that were not detected by the scanning tool as a way to get a guilty plea…

Assuming it is possible (I think it is), there is a manual verification process if you have a match. And obviously, the white noise will be rejected, like all pictures that do not look remotely like the original.

But it can be a form of denial of service: saturate the system with hash collisions so that people can't keep up.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#222

Post by nicce »

jet_32951 wrote: post #210
throwaway212135 wrote: post #182

Earlier quoted context omitted.

I am not sure the right questions are being asked. 1. Who is adding these photos to NCMEC? 2. How often are these photos added? 3. How many people have access to these photos - both adding and viewing? Everyone is focused on Apple and no one is looking at MCMEC. If I wanted to plant a Trojan horse, I would point everyone towards Apple and perform all of the dirty work on the NCMEC end of things.

Exactly. An unknown mechanism adds hashes to a NGO subject to exactly what conditions? This initiative makes me extremely leery of black boxes, to the extent that any algorithm between subject and accusation had damned well better be explainable outside the algorithm; else I as a jury member am bound to render a "not guilty" verdict.

Their system needs real images in the training phase, because they are building the system which produces hashes. There must be someone to confirm from Apple, that indeed correct photos are flagged. At least in the beginning.

We don’t know really how adding new hashes work. NCMEC has the whole new algorithm and they drag-n-drop new images? Hopefully not like that.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#223

Post by FabHK »

joe_the_user wrote: post #106
madeofpalk wrote: post #89

Earlier quoted context omitted.

But some of those technical points are important. Parent comment was concerned that photos of their own kids will get them in trouble - it appears the system was designed to explicitly to prevent that.

The Daring Fireball article actually is a little deceptive here. It goes over a bunch of that won't get parents in trouble and gives a further couched justification of the finger printing example. The question is whether an ordinary baby photo is likely to collide with the one of the CSAM hashes Apple will be scanning for. I don't think Apple can give a definite no here (Edit: how could give a guarantee that a system…

> how could give a guarantee that a system that finds any disguised/distorted CSAM won't tag a random baby picture with a similar appearance.

Cannot guarantee, but by choosing a sufficiently high threshold, you can make the probability of that happening arbitrarily small. And then you have human review.

> And given such collision, the picture might be looked at by Apple and maybe law enforcement

No, not "the picture", but a "visual derivative".

Re: The deceptive PR behind Apple’s “expanded protections for children”

#225

Post by hirundo »

blairbeckwith wrote: post #211
jliptzin wrote: post #168

There is something you can do about it: don’t use Apple products

That strategy will last ~15 minutes until Google is doing the same thing. Then what? I would argue that what Google is doing already is way more privacy-compromising than this.

That's a great argument for a Linux phone or de-googled Android build.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#226

Post by ajsnigrutin »

Barrin92 wrote: post #214

>The worst part is: how do I put my money where my mouth is? Am I going back to using Linux on the desktop (2022 will be the year of Linux on the desktop, remember) people really need to retire this meme. On the desktop in particular as a dev environment Linux is completely fine at this point. I can understand people not wanting to run a custom phone OS because that really is a ton of work but for working software de…

I hate ubuntu from the bottom of my heart, for breaking stuff and changing stuff that used to "just work" all the time, but 99.999% of the time, that means "background stuff", "normal users" never mess around with, and for normal users, a "usb key -> install -> next, next, next -> finish -> reboot" just works.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#227

Post by farmerstan »

fortenforge wrote: post #137
querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive i…

How do new hashes get added to this database? How do we know that all the hashes are of CSAM? Who is validating it and is there an audit trail? Or can bad actors inject their own hashes into the database and make innocent people get reported as pedophiles?

Re: The deceptive PR behind Apple’s “expanded protections for children”

#228

Post by ummonk »

wyager wrote: post #64
macintux wrote: post #51

Earlier quoted context omitted.

Apple almost never talks about features like that until they’re ready, so while you’re correct, it doesn’t mean much.

It's been leaked before that Apple folded under pressure from the FBI not to add iCloud encryption for images.

That would seem to back up the theory that they plan to roll out E2EE and are adding on-device scanning first to enable that.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#229

Post by fortenforge »

farmerstan wrote: post #219

Whoever controls the hash list controls your phone from now on. Period. End of sentence. Apple has not disclosed who gets to add new hashes to the list of CSAM hashes or what the process is to add new hashes. Do different countries have different hash lists? Because if the FBI or CIA or CCCP or KSA wants to arrest you, all they need to do is inject the hash of one of your photos into the “list” and you will be flagge…

There are numerous incorrect statements in your comment.

First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safety organizations. https://twitter.com/AlexMartin/status/1424703642913935374/ph...

Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists.

Second: Why can't the FBI / CCCP inject a hash into the list. Here's a tweet thread gamifying that scenario: https://twitter.com/pwnallthethings/status/14248736290037022...

The short answer is that at some point an Apple employee must visually review the flagged photo, and confirm that it does represent CSAM content. If it does not, then Apple is under no legal obligation to report it.

Third: You claim that abusers will simply opt not to use iPhones to distribute their CSAM content rendering the feature useless. This is in fact not how things have played out on other platforms like Google and Facebook that do already scan for CSAM. These organizations report on the order of millions of flagged images per year. [1] Clearly the abusers have simply not moved on to a different platform.

[1] https://www.businessinsider.com/facebook-instagram-report-20...

Re: The deceptive PR behind Apple’s “expanded protections for children”

#230

Post by matwood »

still_grokking wrote: post #103
AlexandrB wrote: post #20

Earlier quoted context omitted.

That's the crux of it. Why bother with on-device identification, unless one of: a. Apple intends to E2E encrypt iCloud data. b. This is intended to extend to all photos on the device in the future. I'm hoping it's (a), but it's probably (b). And in either case it sets a bad precedent for other companies to follow. Edit: This also turns every jailbreak into a possible CSAM detection avoidance mechanism, giving the gov…

Where is this stance coming form that Apple needs to break E2E crypto to be "able" to "E2E encrypt iCloud data"? That makes absolutely no sense. There is nowhere such a requirement. They could just E2E encrypt iCloud data. Point.

There is no requirement right now, but you only need to look at what's happening in the US, UK, and EU to see the battle setting up around E2EE. Apple may see this feature as a way to quiet critics of E2EE. Hard to know if it will be enough.

But, I think it's safe to say if Apple did turn on E2EE w/o any provision for things like CSAM, it would help drive legislation that is likely more heavy handed.

Post reply on HN