Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple plans to scan US iPhones for child abuse imagery

ft.com

Post reply on HN
Print view View original

291–300 of 390 posts

Re: Apple plans to scan US iPhones for child abuse imagery

#291

Post by native_samples »

babyshake wrote: post #173
mm983 wrote: post #161

Earlier quoted context omitted.

This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. https://support.cloudflare.com/hc/en-us/articles/36004610611... How about we turn the tables and have the complainers suggest a solution. Because every single time an approach to targeted child porn t…

So this means it is checking if you are sharing known CP images? That does seem to be much less invasive and problematic as there is likely no good reason to be sharing these images.

There actually are some edge cases even for matching against image blacklists. Google has experience with hitting them because it's used this type of image simhash for years (for shared cloud files at least).

The definition of child porn varies around the world. These systems use the US definition. This is not entirely what you might expect. For example, in the USA the courts have decided that cartoons can be child porn even though no actual children are in the picture. Most of the world does not agree with this, meaning an image can be CP in one place but not another. Is Apple going to enforce the US definitions or the ones where the user actually lives?

In the USA, photos an under-age person takes of themselves can also be considered CP.

What counts as a "child" for sexual purposes also varies around the world. Some countries have a lower age of consent than other places. In some parts of the world the age of consent and the age at which a child stops being a child for CP purposes are different, meaning that a teenager can have sex legally but if they take a photo of themselves doing it, they are trafficking in CP.

Finally, what is actually on these image blacklists? Hardly anyone actually knows because of the third rail nature of CP. Tech firms are often delivered image hashes, not even the images themselves, by third party 'charities' of various kinds and tech workers are - for obvious reasons - not normally given access to the actual pixels. Additionally, appeals from users are invariably ignored because people say "legal issues, it's complicated" and so everyone clams up. If FPs occur there is no way to resolve it and the people who see your appeal, if there even is one, won't be willing to actually look at the image to find out what it was.

It should be obvious how much potential for abuse this hands the people who actually manage these CP databases. Literally any image can be made verboten immediately, without any recourse, and basically nobody will ever find out including the people who shut down the affected users.

Re: Apple plans to scan US iPhones for child abuse imagery

#292

Post by sterlind »

heavyset_go wrote: post #248
fallingknife wrote: post #197

Earlier quoted context omitted.

> Child abusers evolve and are very happy if law enforcement doesn't. Yes. And now they will evolve by developing a simple system to modify pixels in images when they copy and transmit that will easily defeat this hashing system. The only effect this will have is that moral panickers like you will have got everybody's privacy invaded over your moral panic of the day.

> And now they will evolve by developing a simple system to modify pixels in images when they copy and transmit that will easily defeat this hashing system They don't use simple file hashes to match images, but perceptual hashes. That way they can find modified derivatives of a source image. The problem with this approach, though, is that this is ripe for false positives. Two completely unrelated images can have simi…

could you use multiple perceptual hash functions with different salts, so that collisions would be less likely while allowing derivatives to be detected?

Re: Apple plans to scan US iPhones for child abuse imagery

#293

Post by heavyset_go »

sterlind wrote: post #292
heavyset_go wrote: post #248

Earlier quoted context omitted.

> And now they will evolve by developing a simple system to modify pixels in images when they copy and transmit that will easily defeat this hashing system They don't use simple file hashes to match images, but perceptual hashes. That way they can find modified derivatives of a source image. The problem with this approach, though, is that this is ripe for false positives. Two completely unrelated images can have simi…

could you use multiple perceptual hash functions with different salts, so that collisions would be less likely while allowing derivatives to be detected?

They aren't just matching exact hash hits, but are using a metric like the hamming distance between hashes to determine if one image is the same as , or a derivative of, another. The data structures that allow for efficient lookups rely on that metric, or another metric, for matching.

Re: Apple plans to scan US iPhones for child abuse imagery

#294

Post by tyingq »

heavyset_go wrote: post #265
yellow_lead wrote: post #247

Earlier quoted context omitted.

Yep, Google can break SHA1, so what hash are they using? Proprietary information I assume.

They aren't using file hashes like SHA1, but perceptual hashes and then a fuzzy metric like hamming distance to make a fuzzy match. It's entirely possible for two images that have nothing to do with each other to have similar hashes.

Yes, that's what I was alluding to. It's a perceptual hash, so you could probably take a flagged image and superficially alter it (contrast/color/crop/whatever) until it wasn't anything visually objectionable on it's own. But it would still match the perceptual hash.

Re: Apple plans to scan US iPhones for child abuse imagery

#297

Post by sandworm101 »

Retric wrote: post #285
sandworm101 wrote: post #281

Earlier quoted context omitted.

If apple really cares about security then those images in iCloud should be stored in an encrypted form that makes hash comparisons impossible. If they are hashing them then they have access to them in plaintext. If I used iCloud I would be more worried about the wider implications that creates.

iCloud needs to allow people to restore backups on a new iPhone after losing their old one. You can setup secure encrypted backups, but the customer losing the key means losing the back so that’s not what consumer focused companies are going to do. In other words any backup service that doesn’t have big warnings that losing your key loses your backup means they can read your data.

https://blog.cryptographyengineering.com/2012/04/05/icloud-w...

"The mud puddle test: You don’t have to dig through Apple’s ToS to determine how they store their encryption keys. There’s a much simpler approach that I call the ‘mud puddle test’"

Re: Apple plans to scan US iPhones for child abuse imagery

#298

Post by get52 »

xdennis wrote: post #181
itake wrote: post #31

Earlier quoted context omitted.

> perfectly innocent photos on someone's phone of their own children They are using a library of human verified images to compare the hashes. Probably similar to PhotoDNA[0]. There can be false positives, but AFAIK the algorithms are not trying to identify naked children, but comparing 2 similar image hashes. [0] - https://en.wikipedia.org/wiki/PhotoDNA

It's still bad. I used to scrape pictures from many legal pornography sites. I still have hundreds of thousands of pictures. Since many are user uploaded, I imagine a small number could be real CP. Thank God, I don't use Apple. (I should probably delete them, since I've basically never used them... although they could be useful for one of those end-of-world Raspberry Pi builds: https://back7.co/home/raspberry-pi-quic…

No offence dennis but I gotta be a bit blunt here.

If you're not trolling then you're being a fucking idiot, you could land serious prison time for having a shit ton of child porn on you. If you live in a country where it's illegal delete literally all of it of the hard drive, run DBAN on it, smash it to pieces with a hammer and then set the hard drive on fucking fire.

Literally zero courts are gonna believe "It's just a dataset for if I want to train an AI on it later XD". You will get horsefucked in court

Re: Apple plans to scan US iPhones for child abuse imagery

#299

Post by lp0_on_fire »

Spooky23 wrote: post #178
1-6 wrote: post #144

Earlier quoted context omitted.

It's not the case when you're inside Costco. I'd also argue that it's not open to the public because you're required to have a membership to go in.

The point is, Costco is taking many pictures of you, including possibly correlating with events or actions.

We accept Costco taking pictures of you as a condition of entering the store and securing a membership.

The same does not go for random creeps wanting to take close up pictures of a child.

Re: Apple plans to scan US iPhones for child abuse imagery

#300

Post by Spivak »

LeifCarrotson wrote: post #271
Spivak wrote: post #165

Earlier quoted context omitted.

The conventional wisdom for people integrating PhotoDNA is that false positives are vanishingly rare. If you run it against your data and you get a hit then you should call the FBI.

It would be better if the false positive rate was known to be nonzero. And what, in this context, is "your data?" Is a photo on an iPhone Apple's data? I'd argue that the thing you should do if you get a hit is to delete your data, unless you want to spend enormous amounts of time and money defending yourself from prosecution by people who believe that false positives are vanishingly rare.

I'm talking from the perspective of a service-owner who handles user-generated content. You typically set up PhotoDNA on two parts of your infrastructure. You scan client-side and refuse uploads to prevent the images from getting on your servers in the first place and then you scan on the backend to catch anything that slipped through. You do the first part so you don't have to call the FBI.

I really don't care about the semantics of data ownership. If Apple wants to scan photos you upload to iCloud so they don't run into a scandal years down the road that "iCloud is being used to distribute CP" then that's their call.

And to the letter of the law you can't just delete the data if you found CP that one of your customers uploaded without running afoul of mandatory reporting laws.

Post reply on HN