Apple plans to scan US iPhones for child abuse imagery • Hacker News Board

Re: Apple plans to scan US iPhones for child abuse imagery

#121

Post by aviraldg » Thu, Aug 05, 2021, 4:59 PM UTC

mlac wrote: post #102

1-6 wrote: post #23

Earlier quoted context omitted.

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of th…

Presumably the verification that these false matches are not problematic is manual?

That's not good enough given how our media currently works. Imagine articles published when information about such a check leaks that say "celebrity X's phone checked by police for suspected CSAM." While that is the truth ("suspected") no one cares about that nuance and such a person would get cancelled very quickly, even if there was no evidence of wrongdoing.

Re: Apple plans to scan US iPhones for child abuse imagery

#122

Post by devwastaken » Thu, Aug 05, 2021, 4:59 PM UTC

Reminder that even accusations of abuse imagery will get you in cuffs and your children taken away (see article at the end)

Linux truly is the last bastion of sanity. I am so glad that linux desktops took off and work as well as they do. Last thing I should be worrying about is if an employee at Microsoft or Apple decides their "automated system with no errors" decides that a hash collision or a picture of a baby is close enough to actual abuse imagery. There are plenty of people that believe even a picture of a naked baby in a bathtub is abuse imagery. Anyone remember the case of Walmart? They took their kids away "pending investigation".

https://jonathanturley.org/2009/09/18/arizona-couple-sues-wa...

Re: Apple plans to scan US iPhones for child abuse imagery

#123

Post by cm2187 » Thu, Aug 05, 2021, 5:00 PM UTC

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

And you can imagine what's on a dermatologist or urologist phone!

Re: Apple plans to scan US iPhones for child abuse imagery

#124

Post by cronix » Thu, Aug 05, 2021, 5:01 PM UTC

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

Don't worry, AI is very good at these things and never makes mistakes. It is so advanced it even understands nuance and complex communication mechanisms like sarcasm in text, even if it's only a single sentence to contrast! It will be able to know exactly the context of each image and it will affect only the sickos. We're also working on watching your face through the camera to see your true expression when viewing them. Oh, but if a few innocent people do happen get caught up in it and lives ruined, it's totally justifiable because this type of activity is the absolute worst. And false positives are exceeeeedingly rare. We'll apologize and adjust our algorithms. Sue us? How dare you? We're really trying to wipe out sickos. Are you a sicko or something? You must be. Only sicko's would disagree with protecting children.

Re: Apple plans to scan US iPhones for child abuse imagery

#125

Post by lenkite » Thu, Aug 05, 2021, 5:01 PM UTC

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

They need to stop calling it an iPhone. It's a govPhone. It now belongs to the government with "In-built Citizen Scanning" as free civic service. Thank you for serving the nation!

Re: Apple plans to scan US iPhones for child abuse imagery

#126

Post by __blockcipher__ » Thu, Aug 05, 2021, 5:01 PM UTC

twojacobtwo wrote: post #112

luckylion wrote: post #41

Earlier quoted context omitted.

> All it takes is one security researcher to publish a tool that creates millions of false-positives for the current dataset/model used by Apple & others to be useless. And even better: someone could create false-positives and just send them to people, or put them on websites. While you're watching funny kittens tumbling around, your iphone is calling the cops on you because it thinks you're watching child porn.

Pardon my ignorance on the subject, but is it feasible to create a false-positive hash match? Isn't a hash directly correlated to the file content? So if the file content is changed at all, the hash should be completely different, should it not?

You have two primary misunderstandings AFAICT.

(1) You're assuming it's a "traditional" hash, as opposed to a perceptual hash. The former is purely based off file contents and thus any transformations applied on an image will lead to a new pseudorandom hash. By contrast, perceptual hashes are made to be able to still return a positive when a photo is resized, accumulated artifacts etc. This proposal is to use phashes, not hashes

(2) You have a subtle misunderstanding of how a traditional cryptographic hash function works. Such a function maps an infinite number of possible inputs to a finite number of possible outputs. To your point, flipping even a single bit will basically give you a completely new pseudorandom hash. However, given the infinite inputs and finite outputs, for every output (every hash) there is a countably infinite number of possible inputs that would hash to that value. Thus it's entirely possible to change the file contents and end up with the same hash, although (to your point) without doing any magic to intentionally cause a collision, effectively you will never get one because of just how many damn buckets there are

Re: Apple plans to scan US iPhones for child abuse imagery

#127

Post by greatgib » Thu, Aug 05, 2021, 5:02 PM UTC

To laugh, I just imagine the society of 30 years ago and what would have happened if someone would have tried to suggest to do something like that:

ie, have private company give themselves the right to go have a look through your belonging and in your home because you might be a criminal.

It is now well known that most citizens of most countries are criminals, so no one deserve privacy anymore...

But sadly, it looks like that intelligence and common sense has generally decreased in the population.

Btw, the appleTV and iphone, that are always listening to your conversations in your home to detect some bad keywords and then denounce you to the police is not yet ready? Nazi collaborators, from their tombs, are still waiting for this nice feature!

Anyway, iphone owners, let me tell you that you did a good job by stupidly giving your money and power to Apple!

Re: Apple plans to scan US iPhones for child abuse imagery

#128

Post by eunos » Thu, Aug 05, 2021, 5:02 PM UTC

exo762 wrote: post #68

bitwize wrote: post #57

This just means child molesters will seek out PinePhones, Purism Librems, etc. And those phones will acquire reputations as "pedophones", and anyone buying or using one will be immediately considered sus. People with nothing to hide won't mind an OS with surveillance features.

Until few high profile cases of people with nothing to hide getting swatted because of false positive / image crafted by a GAN to create false positive.

New malware wont be inserting CP, but running GAN to create CP inside the target.

Re: Apple plans to scan US iPhones for child abuse imagery

#129

Post by 1-6 » Thu, Aug 05, 2021, 5:02 PM UTC

1-6 wrote: post #23

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

I actually do want a feature where I will get a notification on my phone if someone nearby has snagged a picture of me or my kids. A creepy shopper at Costco once took a pic of my 2-year-old. They pretended to be working up a conversation with my little kid meanwhile they had their phone at hip-level to snag a shot. I even confronted them about it. The Costco employees couldn't do a thing. Phones with cameras are useful but have also become a nuisance.

Edit: It makes sense that such a notification system would be impossible to implement. Perhaps, registered offenders should not only have their location placed on a map but their phones have certain privileges disabled or monitored.

Re: Apple plans to scan US iPhones for child abuse imagery

#130

Post by dylan604 » Thu, Aug 05, 2021, 5:02 PM UTC

judge2020 wrote: post #66

barbazoo wrote: post #56

Earlier quoted context omitted.

> They are using a library of human verified images to compare the hashes I didn't get that from the article, could you point to a source on this?

Images are submitted by over 1400 organizations[0] like the FBI and NCMEC, although the inner-workings of the database aren’t public and I can’t find a source or confirmation of every image actually being human-verified. The closest is how the "Child Victim Identification Program" helps identify victims in these images. 0: https://www.missingkids.org/theissues/csam#:~:text=Electroni... (ctrl-f "1,400")

Of all of the discussion, my brain has only keyed in on the child porn aspect. It just now clicked that if a missing kid is in a photo (even if not porn), it could help ID the missing kid. Some runaway at the age of 13 gets photographed in a group of other kids just doing teenager things.

So even if not running hashes against known porn, it could still be doing facial recognition. Wonder what kind of implications that would have for people in witness protection or battered spouses and other legitimately "disappeared" people.