Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple plans to scan US iPhones for child abuse imagery

ft.com

Post reply on HN
Print view View original

31–40 of 390 posts

Re: Apple plans to scan US iPhones for child abuse imagery

#31

Post by itake »

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

> perfectly innocent photos on someone's phone of their own children

They are using a library of human verified images to compare the hashes. Probably similar to PhotoDNA[0]. There can be false positives, but AFAIK the algorithms are not trying to identify naked children, but comparing 2 similar image hashes.

[0] - https://en.wikipedia.org/wiki/PhotoDNA

Re: Apple plans to scan US iPhones for child abuse imagery

#34

Post by syshum »

mkoryak wrote: post #20
partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

I think they are probably just going to match the hashes to known hashes of "bad" images.

Plain file hashing for this kind of thing has not been used for years (or even decades), Microsoft came out with Photo DNA along time ago, and I am sure now days they have something ML powered to do it.

I am not sure the accuracy of these systems but even if they are pretty accurate there is something off putting about this trend.

This very much feels like a Guilty until proven innocent type of program... "if you have nothing to hide you have nothing to fear"...

Neither of those normally work out really well.

Re: Apple plans to scan US iPhones for child abuse imagery

#35

Post by Causality1 »

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

Yeah if you ran the giant box of photos in my mom's closet through an AI you'd totally get hits from photos of baby me being bathed in the sink, baby me running around the back yard naked, etc. This can't end well.

Re: Apple plans to scan US iPhones for child abuse imagery

#36

Post by Ancapistani »

zug_zug wrote: post #18
imroot wrote: post #3

https://towardsdatascience.com/black-box-attacks-on-perceptu... I mean, who's to say that my hash is actually what they think it is... ...and if it's not uploaded to iCloud for someone to manually review what the image is, is that going to be enough to get a search warrant for my phone? "AUSA SOMEONE: But, Your Honor, The Defendant's phone has an image that matched a hash of a CSAM picture!" "DEFENDANT: I'm not going…

EDIT: It appears they are using perceptual hashes, which are likely much more prone to collisions. The following math does NOT apply to the tech Apple is using. >> I mean, who's to say that my hash is actually what they think it is... Let's do the math. assume they are using a UUID for the hash. "For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at leas…

> "The odds of This number is equivalent to generating 1 billion UUIDs per second for about 85 years."

If we're talking about random files, sure.

What if the files are intentionally created to match the desired hash? The malicious actor doing this could be a private party intending to disrupt your life, but it could also be law enforcement as a means of gaining access to your device when it would otherwise require a warrant.

Re: Apple plans to scan US iPhones for child abuse imagery

#37

Post by Causality1 »

sebzim4500 wrote: post #25
zug_zug wrote: post #18

Earlier quoted context omitted.

EDIT: It appears they are using perceptual hashes, which are likely much more prone to collisions. The following math does NOT apply to the tech Apple is using. >> I mean, who's to say that my hash is actually what they think it is... Let's do the math. assume they are using a UUID for the hash. "For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at leas…

What if an attacker has intentionally produced an image with the same hash as a well known blacklisted image? Then they could send you the new image and fuck up your life.

Hell, what if you were browsing a site like Facebook or Imgur or Reddit, saw an abuse image, reported it, but it still lives in your phone's browser cache?

Re: Apple plans to scan US iPhones for child abuse imagery

#38

Post by bastawhiz »

> Apple’s neuralMatch algorithm will continuously scan photos that are stored on a US user’s iPhone and have also been uploaded to its iCloud back-up system. Users’ photos, converted into a string of numbers through a process known as “hashing”, will be compared with those on a database of known images of child sexual abuse.

It's weird and invasive that they'd search users devices, but this makes it sound like they're just searching for files with certain hashes, which (to me) is fairly uninteresting. What I don't understand is why this is called "neuralMatch". Is it some sort of perceptual hashing (versus cryptographic hashing)? What exactly makes it "neural"?

If neuralMatch is to Google's reverse image search as Apple Maps was to Google Maps when it launched, this is going to have so many false positives.

> According to people briefed on the plans, every photo uploaded to iCloud in the US will be given a “safety voucher” saying whether it is suspect or not. Once a certain number of photos are marked as suspect, Apple will enable all the suspect photos to be decrypted and, if apparently illegal, passed on to the relevant authorities

Is this separate? Or is the article just confusing iCloud storage for on-device storage? Regardless, there's some very lazy journalism going on here.

Re: Apple plans to scan US iPhones for child abuse imagery

#39

Post by sandworm101 »

As any of the lawyers here can attest, the legal definitions here open to huge debates. Some material is unambiguous, other images reside extensive grey areas. The legality of material can even change based on context. Something that is perfectly legal in one area can be absolutely illegal in another. Example: If I had to list the most iconic photographs in world history, one of them is of a totally nude child (Vietnam). That isn't an illegal image, but try explaining that to a computer. At the moment these nuances are handled by cops, prosecutors and in extreme cases the courts. Is Apple going to put a marker down on a particular universal definition? Will that definition become the de facto world standard?

Re: Apple plans to scan US iPhones for child abuse imagery

#40

Post by mrkramer »

Imagine if Microsoft does this with Windows, lots of people would get arrested rightly so.

But like some people say this can be dangerous because evidence can be planted on your device. I remember reading articles how through torrent clients files could be planted but I never actually saw it in the wild.

Post reply on HN