Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The deceptive PR behind Apple’s “expanded protections for children”

piotr.is

Post reply on HN
Print view View original

131–140 of 595 posts

Re: The deceptive PR behind Apple’s “expanded protections for children”

#131

Post by samename »

gambiting wrote: post #16
samename wrote: post #8

Earlier quoted context omitted.

Unless those pictures are also in the NCMEC database, there won’t be a match.* * As addressed in the comments below, this isn’t entirely true: the hash looks for visually similar picture and there may be false positives.

Absolutely not true. Apple is using a similarity based hash, so if the NCMEC database contains a picture that's similar to one that you have, it could produce a match even if it's not the same. Apple says this isn't an issue, because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) and judge whether they are pictures of child abuse or not. If this unknown pe…

Good point, thanks, updated my comment.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#132

Post by zepto »

zionic wrote: post #65
czzr wrote: post #26

Earlier quoted context omitted.

As I said, the design enables this, if Apple chose to do it. It remains to be seen if they will.

The design more plausible enables total device surveillance than questionable iCloud Backups. (I refuse to call a backdoored setup E2EE)

That’s silly. The design is so narrowly tailored to scan for CSAM that nobody can use it for anything else.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#133

Post by outworlder »

seph-reed wrote: post #9

I really don't see why the scanning would ever be done on the phone instead of on iCloud if it only affects iCloud images. But I do have guesses why.

In-device scanning is used for the feature that warns teens (and if =Biggest mistake Apple has ever done was to roll out three different features at once and announce at the same time. This is creating all sorts of confusion.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#134

Post by dev_tty01 »

querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

No. The CSAM (Child Sexual Abuse Material) scanning is comparing hashes of photos about to be uploaded to iCloud against a specific set of images at NCMEC (National Center for Missing and Exploited Children) which are specific to missing and exploited children. It is not machine learning models looking for nudes or similar. It is not a generalized screening. If enough matched images are found, the images are flagged for manual verification. If the manual verification confirms that the images match specific images in the NCMEC database, law enforcement is informed.

Be aware that almost all cloud providers screen photos. Facebook reported 20 million images in 2020, Google reported half a million. Dropbox, Box, and many, many others report images. See https://www.missingkids.org/content/dam/missingkids/gethelp/... to see a complete list of companies that screen and report images.

The other thing Apple announced which is completely separate from the CSAM photo scanning is additional parental controls for the Messages app. If a parent opts in for their under-13 children, a machine learning model will look for inappropriate material and warn the child prior to showing the image. The child is also told that their parent will be flagged if the child looks at it anyway. For 13-18 year olds whose parents opted in, the teen is warned first about the content. If the teen continues past the warning the image is shown and no further action is taken. Parents are not flagged for children 13 and over. As I said, this is a parental control for pre-adult kids. It requires opt-in from the parents and has no law enforcement implications.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#135

Post by Componica »

Imagine taking a photo or have in your gallery a photo a dear leader doesn't want to spread. Ten minutes later you heard a knocking at your door. That's what I'm most worried about, how is this not creating the infrastructure to ensnare political dissidents.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#136

Post by sandworm101 »

querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Yes, if they wind up part of a child porn investigation. Your cloud account gets hacked. Some perv gets your images. He is then arrested and his "collection" added to the hash database... including your family photos.

Context often matters more than the nature of the actual content. Police aquire thousands of images with little hope of ever knowing where they originated. If they are collected by pervs, and could be construed as illegal in the hands of pervs, the images become child porn and can be added to the databases.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#137

Post by fortenforge »

querez wrote: post #2

I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?

Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive issue with false positives?

The fact of the matter is that unless you possess a photo that exists in the NCMEC database, your photos simply will not be flagged to Apple. Photos of your own kids won't trigger it, nude photos of adults won't trigger it; only photos of already known CSAM content will trigger (and that too, Apple requires a specific threshold of matches before a report is triggered).

[1] "The threshold is selected to provide an extremely low (1 in 1 trillion) probability of incorrectly flagging a given account." Page 4 of https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Re: The deceptive PR behind Apple’s “expanded protections for children”

#138

Post by FabHK »

Dah00n wrote: post #100
samename wrote: post #8

Earlier quoted context omitted.

Unless those pictures are also in the NCMEC database, there won’t be a match.* * As addressed in the comments below, this isn’t entirely true: the hash looks for visually similar picture and there may be false positives.

That is not true. If you read what experts who actually do something to stop CP (unlike Apple) say there are proven false positives. https://www.hackerfactor.com/blog/index.php?/archives/929-On...

Yes, and you can put a number on the probability of that happening, say a fixed p arbitrarily small by making N sufficiently large. (I'm pretty sure that's how Apple came up with their "1 in a trillion chance per year".) And then you still have manual review.

[1] you could "help" independence by requiring a certain distance between images you simultaneously flag.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#139

Post by samename »

gonehome wrote: post #21
samename wrote: post #11

Earlier quoted context omitted.

John Gruber is biased because his brand is closely tied to Apple’s brand. Ben Thompson wrote a better review on the topic: https://stratechery.com/2021/apples-mistake/ There’s also the Op-Ed by Matthew Green and Alex Stamos, cyber security researchers: https://www.nytimes.com/2021/08/11/opinion/apple-iphones-pri...

They have a podcast together called Dithering which is pretty good (but not free) - they're friends. I think John's article is better than Ben's, but they're both worth reading. Ben takes the view that unencrypted cloud is the better tradeoff - I'm not sure I agree. I'd rather have my stuff e2ee in the cloud. If the legal requirements around CSAM are the blocker then Apple's approach may be a way to thread the needle…

Friends can disagree. Everyone has their own biases - good and bad. I think it’s always good to keep people’s biases in mind when reading their work.

Re: The deceptive PR behind Apple’s “expanded protections for children”

#140

Post by lawkwok »

merpnderp wrote: post #85
lawkwok wrote: post #57

Earlier quoted context omitted.

Non-zero being technically true because of the subject matter, but I don’t see how Apple’s system increases the risk of authorities killing family or pets more than server-side scanning.

Their neural hashing is new, and they claim has a one in a trillion collision rate. There are 1.5 trillion images created in the US and something like 100 million photos in the compared database. That's a heck of a lot of collisions. And that's just a single year, Apple will be comparing everyone's back catalog. A lot of innocent people are going to get caught up in this.

We’ll have to wait and see how good their neural hashing is, but just to clarify the 1 trillion number is the “probability of incorrectly flagging a given account” according to Apple’s white paper.

I think some people think that’s the probability of a picture being incorrectly flagged, which would be more concerning given the 1.5 trillion images created in the US.

Source: https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Post reply on HN