Apple plans to scan US iPhones for child abuse imagery • Hacker News Board

Re: Apple plans to scan US iPhones for child abuse imagery

#181

Post by xdennis » Thu, Aug 05, 2021, 5:26 PM UTC

itake wrote: post #31

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

> perfectly innocent photos on someone's phone of their own children They are using a library of human verified images to compare the hashes. Probably similar to PhotoDNA[0]. There can be false positives, but AFAIK the algorithms are not trying to identify naked children, but comparing 2 similar image hashes. [0] - https://en.wikipedia.org/wiki/PhotoDNA

It's still bad. I used to scrape pictures from many legal pornography sites. I still have hundreds of thousands of pictures. Since many are user uploaded, I imagine a small number could be real CP.

Thank God, I don't use Apple.

(I should probably delete them, since I've basically never used them... although they could be useful for one of those end-of-world Raspberry Pi builds: https://back7.co/home/raspberry-pi-quick-kit-one Edit: the non-bad ones, in case this gets misconstrued)

Edit: I've deleted my archive.

Re: Apple plans to scan US iPhones for child abuse imagery

#182

Post by judge2020 » Thu, Aug 05, 2021, 5:27 PM UTC

kingsloi wrote: post #169

I had a very sick 8 month old baby who I obviously had a million photos of. Unfortunately, my lil girl had 9 surgeries and her little body had gone through more than people can think of. She passed in May, and it's really difficult to look back at old pictures because of how much stuff was going on, various levels of critical ICU care, pre/post surgery, etc. But it wasn't until a few months after she was born I had g…

This isn't scanning for naked children, it's known CSAM photos that are already circulating online.

Re: Apple plans to scan US iPhones for child abuse imagery

#183

Post by alasano » Thu, Aug 05, 2021, 5:28 PM UTC

mlac wrote: post #102

1-6 wrote: post #23

Earlier quoted context omitted.

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of th…

Furthermore, "highly unlikely" is an understatement. Using SHA256 there are 2^256 hashes.

That's 115792089237316195423570985008687907853269984665640564039457584007913129639936. Probability of collision (two different inputs generating the same hash) is infinitesimal.

Re: Apple plans to scan US iPhones for child abuse imagery

#184

Post by Spooky23 » Thu, Aug 05, 2021, 5:29 PM UTC

syshum wrote: post #34

mkoryak wrote: post #20

Earlier quoted context omitted.

I think they are probably just going to match the hashes to known hashes of "bad" images.

Plain file hashing for this kind of thing has not been used for years (or even decades), Microsoft came out with Photo DNA along time ago, and I am sure now days they have something ML powered to do it. I am not sure the accuracy of these systems but even if they are pretty accurate there is something off putting about this trend. This very much feels like a Guilty until proven innocent type of program... "if you hav…

They work out very well. Any significant storage provider has been using them for years.

Re: Apple plans to scan US iPhones for child abuse imagery

#185

Post by anigbrowl » Thu, Aug 05, 2021, 5:29 PM UTC

partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things

How would that match a hash list of known CP imagery though?

Re: Apple plans to scan US iPhones for child abuse imagery

#186

Post by buran77 » Thu, Aug 05, 2021, 5:29 PM UTC

mm983 wrote: post #161

cronix wrote: post #124

Earlier quoted context omitted.

Don't worry, AI is very good at these things and never makes mistakes. It is so advanced it even understands nuance and complex communication mechanisms like sarcasm in text, even if it's only a single sentence to contrast! It will be able to know exactly the context of each image and it will affect only the sickos. We're also working on watching your face through the camera to see your true expression when viewing t…

This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. https://support.cloudflare.com/hc/en-us/articles/36004610611... How about we turn the tables and have the complainers suggest a solution. Because every single time an approach to targeted child porn t…

Microsoft has been using PhotoDNA [0] to scan OneDrive content for quite some time. The news is that a device manufacturer is doing it on your device with local data. There's some MS research to use machine learning on metadata to identify offending material [1].

[0] https://en.wikipedia.org/wiki/PhotoDNA

[1] https://arxiv.org/pdf/2010.02387.pdf

Re: Apple plans to scan US iPhones for child abuse imagery

#187

Post by [deleted] » Thu, Aug 05, 2021, 5:29 PM UTC

mhh__ wrote: post #92

itake wrote: post #31

Earlier quoted context omitted.

> perfectly innocent photos on someone's phone of their own children They are using a library of human verified images to compare the hashes. Probably similar to PhotoDNA[0]. There can be false positives, but AFAIK the algorithms are not trying to identify naked children, but comparing 2 similar image hashes. [0] - https://en.wikipedia.org/wiki/PhotoDNA

I bet(or know...) the people who verify those images don't get paid anywhere near enough.

[deleted]

Re: Apple plans to scan US iPhones for child abuse imagery

#188

Post by cronix » Thu, Aug 05, 2021, 5:29 PM UTC

mm983 wrote: post #161

cronix wrote: post #124

Earlier quoted context omitted.

Don't worry, AI is very good at these things and never makes mistakes. It is so advanced it even understands nuance and complex communication mechanisms like sarcasm in text, even if it's only a single sentence to contrast! It will be able to know exactly the context of each image and it will affect only the sickos. We're also working on watching your face through the camera to see your true expression when viewing t…

This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. https://support.cloudflare.com/hc/en-us/articles/36004610611... How about we turn the tables and have the complainers suggest a solution. Because every single time an approach to targeted child porn t…

Not a single joke was made about child abuse.

The specific subject of child abuse is irrelevant in my commentary. It was a commentary on the general category of AI, used all over, for many things, and more and more every day, but nice try.

Edit: You edited out what I was referring to as I was replying.

Re: Apple plans to scan US iPhones for child abuse imagery

#189

Post by kingsloi » Thu, Aug 05, 2021, 5:29 PM UTC

judge2020 wrote: post #182

kingsloi wrote: post #169

I had a very sick 8 month old baby who I obviously had a million photos of. Unfortunately, my lil girl had 9 surgeries and her little body had gone through more than people can think of. She passed in May, and it's really difficult to look back at old pictures because of how much stuff was going on, various levels of critical ICU care, pre/post surgery, etc. But it wasn't until a few months after she was born I had g…

This isn't scanning for naked children, it's known CSAM photos that are already circulating online.

Ah, I should have paid more attention to the article. I appreciate the clarification.

Re: Apple plans to scan US iPhones for child abuse imagery

#190

Post by __blockcipher__ » Thu, Aug 05, 2021, 5:29 PM UTC

shuckles wrote: post #120

__blockcipher__ wrote: post #117

Earlier quoted context omitted.

From the article: > The automated system would proactively alert a team of human reviewers if it believes illegal imagery is detected, who would then contact law enforcement if the material can be verified. The scheme will initially roll out only in the US. This is dystopian and needs to be opposed, not applauded.

This is how every single cloud service has worked for a decade. You’ll need to elaborate on how, exactly, you believe it’s dystopian. The article is clearly wrong, because Apple cannot know which jurisdiction to forward a report to. That is NCMEC’s responsibility.

It's dystopian because a private company, who in the past has been committed to maintaining user privacy even when it inconveniences the government, is now regressing and going to scan the "private" contents of peoples' phones to see if they have any matches for images that have been added to a government-supplied blacklist. This blacklist is completely opaque to us as citizens; there is no way to audit or review the contents. Additionally because it's using photoDNA, if someone ever did get ahold of even a single entry in the database, they could craft malicious images that fool the perceptual hash algorithm into flagging an innocent image. (Yes, I get there will be manual review, etc).

Furthermore in principle there's no reason this has to be, or will be, just CP. First it'll expand to blatant terroristic images, say ISIS beheading type videos. Then it will progress to subversive material: COVID "misinformation", "domestic terrorism" (read: anyone the feds don't like), the list goes on. This tool and its successors will be used to completely abolish the concept of device privacy and will be used to ruin the lives of anyone who sufficiently pisses off the feds. And that's just in America, god knows what they're going to do with this system in China.