Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple plans to scan US iPhones for child abuse imagery

ft.com

Post reply on HN
Print view View original

311–320 of 390 posts

Re: Apple plans to scan US iPhones for child abuse imagery

#311

Post by nullifidian »

luckylion wrote: post #41
KingMachiavelli wrote: post #29

This could backfire in a huge way. I think a lot of people, even EFF/privacy focused people are fine with a private company doing CSAM detection (or anything really) on uploaded user content since it is actually voluntary. Now that CSAM detection is being deployed to user devices including those previously sold without it, there's a motivation to circumvent/block it. Unless Apple is doing CSAM scanning 100% inside a…

> All it takes is one security researcher to publish a tool that creates millions of false-positives for the current dataset/model used by Apple & others to be useless. And even better: someone could create false-positives and just send them to people, or put them on websites. While you're watching funny kittens tumbling around, your iphone is calling the cops on you because it thinks you're watching child porn.

Even better -- someone could remotely exploit their adversary with malware like Pegasus, implant actual CP on the phone, hide malware traces, and then the Apple's system would alert the authorities. Who would believe the protests of innocence when there is the evidence on the phone, and the infraction is so horrific? A Perfect crime.

Re: Apple plans to scan US iPhones for child abuse imagery

#312

Post by ksec »

r00fus wrote: post #43

I bet Apple understands this is a bad idea (dilutes their "privacy" brand image) I wonder what forces have pushed Apple to this point.

>I bet Apple understands this is a bad idea

That would be Steve Job's Apple. Tim Cook's Apple will think they are doing it for the greater good.

The road to hell is paved with good intentions

Re: Apple plans to scan US iPhones for child abuse imagery

#313

Post by JohnFen »

mlac wrote: post #102
1-6 wrote: post #23

Earlier quoted context omitted.

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of th…

> it’s very hard to “innocently” end up with one of those images that they are looking for from the database.

Of course, there's no telling what images will be in the database when this inevitably expands beyond CP.

Re: Apple plans to scan US iPhones for child abuse imagery

#314

Post by sterlind »

ggreer wrote: post #272
mm983 wrote: post #191

Earlier quoted context omitted.

Yes, that's exactly it. It uses a database compiled by NGOs and specialized firms comprising of file hashes matching child porn. These lists are handled by humans. Fuzzy means that it takes compression and the like into account, because even if just one pixel out of 20 thousand is different, the hash is different too. Fuzzy hash still recognizes it as the same image, so using an algorithm to alter the color etc. won'…

> These lists are handled by humans. That's also true for the no-fly list and the Terrorist Screening Database,[1] yet those are full of false positives. And unlike those lists, CSAM databases cannot be independently verified. To do so would require having the original images, which is illegal. 1. https://en.wikipedia.org/wiki/Terrorist_Screening_Database

The no-fly list and the terrorist screening database aren't used in a court of law. The Confrontation Clause of the Sixth Amendment guarantees you access to all the evidence presented against you. You also don't need the original images to defend yourself, though apparently CP can be presented to a (traumatized) jury [0].

So if you're charged on the basis of a fuzzy hash matching, you'd subpoena Apple for the photo in your backup that matched, present it to the court (since it doesn't actually matter if it's CP or not to be admissible), and you win the case.

0. https://www.johntfloyd.com/the-difficulty-with-criminal-evid...

Re: Apple plans to scan US iPhones for child abuse imagery

#315

Post by xdennis »

get52 wrote: post #298
xdennis wrote: post #181

Earlier quoted context omitted.

It's still bad. I used to scrape pictures from many legal pornography sites. I still have hundreds of thousands of pictures. Since many are user uploaded, I imagine a small number could be real CP. Thank God, I don't use Apple. (I should probably delete them, since I've basically never used them... although they could be useful for one of those end-of-world Raspberry Pi builds: https://back7.co/home/raspberry-pi-quic…

No offence dennis but I gotta be a bit blunt here. If you're not trolling then you're being a fucking idiot, you could land serious prison time for having a shit ton of child porn on you. If you live in a country where it's illegal delete literally all of it of the hard drive, run DBAN on it, smash it to pieces with a hammer and then set the hard drive on fucking fire. Literally zero courts are gonna believe "It's ju…

I didn't have "a shit ton". I said that out of ~x00000 images, some might be bad.

But, yeah, I've deleted them now.

Re: Apple plans to scan US iPhones for child abuse imagery

#316

Post by pseudalopex »

getpolarized wrote: post #113
partiallypro wrote: post #14

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal…

> I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. These are still illegal in the US right now right? It's been a major criticism of these laws that…

I don't think just nudity was illegal ever. But just suspicion can hurt people.

Re: Apple plans to scan US iPhones for child abuse imagery

#317

Post by csw-001 »

underseacables wrote: post #270

This is extremely alarming, I don’t care what the reason, I don’t want apple or anyone looking through my personal photos. It’s none of their business, and the sheer chance of false positives and ruining peoples lives is far too great to let this go forward.

Not disagreeing with you - but this reminds me of my college job spending all day looking through personal photos.

I put myself through school working at a one-hour photo shop in a drug store. It was my job to feed the machine, keep the chemicals balanced, paper loaded and perform white-balancing and cropping to make sure the end result matched customers' expectations. It was a relatively small town, and I lived in the same neighborhood that I worked in - so I basically saw 80% of my neighbor's photos during the development, cropping, and packing process. I saw hundreds of baby's in bath tubs, and bare a* toddlers on the run, in the mud, etc. Also saw a lot more bedroom photos than you (or at least 17yo me) might expect.

In the three years of work, I only had one roll of film for which I called the police. It was obvious child abuse. I stopped the machine, called the police and they collected the film directly into evidence and made the arrest when the person came to pick it up. The guy in question thought that the 1hour process was "all automated" and he was pissed that somebody was looking at all his photos.

Not sure that has anything to do with the iPhone AI scanning - but it might be an anecdote of interest to those that don't remember a time when somebody looked at all your photos as part of the creation process.

Re: Apple plans to scan US iPhones for child abuse imagery

#318

Post by get52 »

xdennis wrote: post #315
get52 wrote: post #298

Earlier quoted context omitted.

No offence dennis but I gotta be a bit blunt here. If you're not trolling then you're being a fucking idiot, you could land serious prison time for having a shit ton of child porn on you. If you live in a country where it's illegal delete literally all of it of the hard drive, run DBAN on it, smash it to pieces with a hammer and then set the hard drive on fucking fire. Literally zero courts are gonna believe "It's ju…

I didn't have "a shit ton". I said that out of ~x00000 images, some might be bad. But, yeah, I've deleted them now.

Good I'm glad, I was only harsh because I don't want to see an innocent programmer tossed in jail because of one mistake. xoxo

Re: Apple plans to scan US iPhones for child abuse imagery

#319

Post by [deleted] »

Retric wrote: post #172
cronix wrote: post #124

Earlier quoted context omitted.

Don't worry, AI is very good at these things and never makes mistakes. It is so advanced it even understands nuance and complex communication mechanisms like sarcasm in text, even if it's only a single sentence to contrast! It will be able to know exactly the context of each image and it will affect only the sickos. We're also working on watching your face through the camera to see your true expression when viewing t…

There is zero AI classification involved, it’s a lookup vs hashes of known child porn images. You would need a hash collision, misidentified photo in the dataset, or legitimate use case to end up with a false positive.

[deleted]

Re: Apple plans to scan US iPhones for child abuse imagery

#320

Post by Ajedi32 »

Ancapistani wrote: post #36
zug_zug wrote: post #18

Earlier quoted context omitted.

EDIT: It appears they are using perceptual hashes, which are likely much more prone to collisions. The following math does NOT apply to the tech Apple is using. >> I mean, who's to say that my hash is actually what they think it is... Let's do the math. assume they are using a UUID for the hash. "For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at leas…

> "The odds of This number is equivalent to generating 1 billion UUIDs per second for about 85 years." If we're talking about random files, sure. What if the files are intentionally created to match the desired hash? The malicious actor doing this could be a private party intending to disrupt your life, but it could also be law enforcement as a means of gaining access to your device when it would otherwise require a…

> What if the files are intentionally created to match the desired hash?

That's called a second-preimage attack and is similarly infeasible to achieve against a cryptographically secure hash function.

There are good arguments to be made for why this is a bad idea, but hash collisions aren't one of them.

Post reply on HN