Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple plans to scan US iPhones for child abuse imagery

ft.com

Post reply on HN
Print view View original

111–120 of 390 posts

Re: Apple plans to scan US iPhones for child abuse imagery

#111

Post by __blockcipher__ »

lstamour wrote: post #53
syshum wrote: post #34

Earlier quoted context omitted.

Plain file hashing for this kind of thing has not been used for years (or even decades), Microsoft came out with Photo DNA along time ago, and I am sure now days they have something ML powered to do it. I am not sure the accuracy of these systems but even if they are pretty accurate there is something off putting about this trend. This very much feels like a Guilty until proven innocent type of program... "if you hav…

> Guilty until proven innocent type of program But this software doesn’t make arrests? A comparison would be police asking for what cellphones were in an area. There should be a lot of due process that happens before anything comes of this. I imagine police won’t look into one photo, they would want to match more than one. I agree though, the scary part might be how to clear your name after an overreaction occurs, si…

Apple will be scanning the photos on your phone and cross-referencing them against photoDNA for a government database. You don't see how that's not a huge potential for abuse right there? The government decides the images. And you best believe they're not going to release the images publicly to verify what's in the db, for [many] obvious reasons.

This is a horrible idea, a massive regression in Apple (and therefore the broader industry's) stance on security/privacy, and the fact that you are dutifully swallowing the "child porn" justification is so incredibly concerning to me.

Re: Apple plans to scan US iPhones for child abuse imagery

#112

Post by twojacobtwo »

luckylion wrote: post #41
KingMachiavelli wrote: post #29

This could backfire in a huge way. I think a lot of people, even EFF/privacy focused people are fine with a private company doing CSAM detection (or anything really) on uploaded user content since it is actually voluntary. Now that CSAM detection is being deployed to user devices including those previously sold without it, there's a motivation to circumvent/block it. Unless Apple is doing CSAM scanning 100% inside a…

> All it takes is one security researcher to publish a tool that creates millions of false-positives for the current dataset/model used by Apple & others to be useless. And even better: someone could create false-positives and just send them to people, or put them on websites. While you're watching funny kittens tumbling around, your iphone is calling the cops on you because it thinks you're watching child porn.

Pardon my ignorance on the subject, but is it feasible to create a false-positive hash match? Isn't a hash directly correlated to the file content? So if the file content is changed at all, the hash should be completely different, should it not?

Re: Apple plans to scan US iPhones for child abuse imagery

#113

Post by getpolarized »

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

> I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc.

These are still illegal in the US right now right? It's been a major criticism of these laws that they can hurt people accidentally.

Re: Apple plans to scan US iPhones for child abuse imagery

#114

Post by shuckles »

KingMachiavelli wrote: post #29

This could backfire in a huge way. I think a lot of people, even EFF/privacy focused people are fine with a private company doing CSAM detection (or anything really) on uploaded user content since it is actually voluntary. Now that CSAM detection is being deployed to user devices including those previously sold without it, there's a motivation to circumvent/block it. Unless Apple is doing CSAM scanning 100% inside a…

The article states that the scanning is implemented as some kind of mutual operation between device and server when the device tries to upload photos into Apple’s cloud services. Presumably this is better than solely scanning photos on the server, because a client has even less control over what types of scanning Apple deploys there.

Re: Apple plans to scan US iPhones for child abuse imagery

#115

Post by mlac »

shotta wrote: post #67
lstamour wrote: post #53

Earlier quoted context omitted.

> Guilty until proven innocent type of program But this software doesn’t make arrests? A comparison would be police asking for what cellphones were in an area. There should be a lot of due process that happens before anything comes of this. I imagine police won’t look into one photo, they would want to match more than one. I agree though, the scary part might be how to clear your name after an overreaction occurs, si…

I see it as an unwarranted search, which makes me feel like the state views me as inherently guilty until innocence is proven by sifting through my photos.

It’s not the state, it’s Apple. And they are making the decision that they do not want to put out a product that makes it easy for CP to be shared or viewed. As a huge player in the market this is great.

This also limits the possible avenues for future exposure - if one has less ways to distribute or get CP or the overall amount of it is reduced, they may never get into it in the first place and the market for it will shrink. Thereby reducing the amount produced over time.

Re: Apple plans to scan US iPhones for child abuse imagery

#116

Post by [deleted] »

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

[deleted]

Re: Apple plans to scan US iPhones for child abuse imagery

#117

Post by __blockcipher__ »

shuckles wrote: post #110
Someone1234 wrote: post #52

If it matches, am I guilty until found innocent? I have no illicit images, but false positives are always going to be a problem and even at sub-1% rates if you're scanning literally every image stored on an iOS devices that could still be thousands of wrong matches. If I lose the false-positive lottery, am I going to have the police calling and have my mugshot in the evening news for "CP on their device" in particula…

In the United States, the process of reporting potential images of CSAM is defined by the government, and a technology company’s active responsibility ends with a report to the National Center for Missing and Exploited Children’s CyberTipline: https://www.missingkids.org/gethelpnow/cybertipline#whathapp...

From the article:

> The automated system would proactively alert a team of human reviewers if it believes illegal imagery is detected, who would then contact law enforcement if the material can be verified. The scheme will initially roll out only in the US.

This is dystopian and needs to be opposed, not applauded.

Re: Apple plans to scan US iPhones for child abuse imagery

#118

Post by luckylion »

mataug wrote: post #91
luckylion wrote: post #41

Earlier quoted context omitted.

> All it takes is one security researcher to publish a tool that creates millions of false-positives for the current dataset/model used by Apple & others to be useless. And even better: someone could create false-positives and just send them to people, or put them on websites. While you're watching funny kittens tumbling around, your iphone is calling the cops on you because it thinks you're watching child porn.

A high rate of false positives will create distrust in the system, and ultimately make it less effective since no one will take it seriously.

As in an activist might distribute them far and wide to sabotage the system, yeah.

If it's just a lot of people who get targeted individually, a moral panic might just wipe concerns away, saying "I guess there were more pedophiles than I thought" and "if they hadn't done anything, Apple wouldn't have found them".

But even if the courts strike it down, you'll still have all the trouble that comes with such accusations: the police searching your house, your employer, family, friends and neighbors learning that you're accused of possessing child pornography. Good luck getting back to a normal life after some forensic specialist confirms that it was a hash collision and the judge throws out the case.

Re: Apple plans to scan US iPhones for child abuse imagery

#119

Post by codr7 »

cdolan wrote: post #95
HWR_14 wrote: post #11

Earlier quoted context omitted.

That would be false testimony, not hearsay. Hearsay ("John Smith told me he heard MeinBlut kidnapped the Lindbergh baby") is not allowed. Witnesses lying about what they saw and or participated in is allowed, although it can be lies. Sorry, and as for "what is abuse", these are looking for exploited children. There are a lot of questions here, the morality of mass surveillance, the risk factors due to hash collisions…

I don't think society can function without harsh penalties for child abuse, and I think thats a widely held belief everyone agrees with. I do think your comment is conflicting though. The fuzziness of 'is this abuse' is real because collisions can occur, mass surveillance never goes in reverse, etc.

Doesn't seem to be working very well from here; risk of penalty isn't going to stop most of these people, more likely make their game even more exciting.

Of course they have to be prevented from hurting people, to the best of our ability to do so, but punishment isn't going anywhere.

Re: Apple plans to scan US iPhones for child abuse imagery

#120

Post by shuckles »

__blockcipher__ wrote: post #117
shuckles wrote: post #110

Earlier quoted context omitted.

In the United States, the process of reporting potential images of CSAM is defined by the government, and a technology company’s active responsibility ends with a report to the National Center for Missing and Exploited Children’s CyberTipline: https://www.missingkids.org/gethelpnow/cybertipline#whathapp...

From the article: > The automated system would proactively alert a team of human reviewers if it believes illegal imagery is detected, who would then contact law enforcement if the material can be verified. The scheme will initially roll out only in the US. This is dystopian and needs to be opposed, not applauded.

This is how every single cloud service has worked for a decade. You’ll need to elaborate on how, exactly, you believe it’s dystopian. The article is clearly wrong, because Apple cannot know which jurisdiction to forward a report to. That is NCMEC’s responsibility.
Post reply on HN