Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

vice.com

Post reply on HN
Print view View original

281–290 of 465 posts

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#281

Post by candiodari »

shuckles wrote: post #223
fraa-orolo wrote: post #142

Earlier quoted context omitted.

The thing is that this is a better and more privacy preserving technique. Their hubris is in not seeing or thinking that they will be able to stand up to all kinds of abuses of this system that its mere existence will invite; their hubris is also in thinking that they will be able to perfectly and without mistakes manage and overview a system making accusations so heinous that even the mere act of accusing destroy pe…

What abuses apply to CSAM scanning in this hybrid pipeline which don’t apply to iCloud Backup? If they scanned on the server, why couldn’t governments “slippery slope” abuse the system by requiring that all files on iOS end up in iCloud Backup, where they can be scanned by the system? Since the announcement, I can think of a dozen ways Apple could be easily forced into scanning all the contents of your device by asse…

Exactly. Also people are strongly objecting that their own device is being used to report them to law enforcement. Surely someone at Apple noticed this beforehand ...

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#282

Post by rlpb »

shapefrog wrote: post #231
ALittleLight wrote: post #102

Would this work as an attack? 1. Get a pornographic picture involving young though legal actors and actresses. 2. Encode a nonce into the image. Hash it checking for CSAM collisions. If you've found a collision go on to the next step, if not update the nonce and try again. 3. You now have an image that, to visual inspection will appear plausibly like CSAM, and to automated detection will appear like CSAM. Though, pre…

Your hash will match to say image 105 of the dataset. Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere. So at this point we have an image that computers think is CSAM and people think is CSAM, and when held up next to the original verified horrific image everyone agrees is the same image. At this point, someone is going to ask, rightly…

> Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere.

Define "get anywhere". Why won't you get raided by the police and have all your devices seized first?

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#283

Post by laverya »

floatingatoll wrote: post #205
laverya wrote: post #193

Earlier quoted context omitted.

And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied? Bonus points if you match poses, coloration, background, etc.

The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image,…

This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one.

And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#284

Post by GuB-42 »

GeekyBear wrote: post #88
epistasis wrote: post #42

What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning. For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all. Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.

The thing that's shocking to me is that Google, Microsoft and all the big names in tech have been scanning everything in your account (email, cloud drive, photos, etc) for the past decade, without any noticeable uproar. Apple announces that it is going to start scanning iCloud Photos only, and that their system is set to ignore anything below a threshold of ~30 positives before triggering a human review, and people l…

Google is open about the fact they scan and track everything, they even make it a point of convenience "hey, we looked at your email and found a plane ticket, based on where you are, you should leave in 2 hours if you want to be at the airport on time".

Facebook, even more so, they are explicitly anti-privacy to the point of being insulting.

Microsoft will happily show you everything they may send when you install Windows, you can sometimes refuse, but not always. They are a bit less explicit than Google, but privacy is rarely on the menu.

As for Amazon, their cloud offers are mostly for businesses, different market, but still, for consumers, they don't really insist on privacy either.

So that if any of these company scan your pictures for child porn, it won't shock anyone, because we know it is what they do.

But Apple claims privacy as a core value, half of their ads are along the lines of "we are not like the others, we respect your privacy, everything on your device stays on your device, etc...", they announce every (often legitimate) privacy feature with great fanfare, etc... So much that people start to believe it. But with that, people realize that Apple is not so different from the others after all, and if they bought an overpriced device based on that promise, I understand why they are pissed off.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#285

Post by smnrchrds »

kcb wrote: post #250
smnrchrds wrote: post #238

Earlier quoted context omitted.

I will believe them if they put their money where their mouth is: as a clause to iOS user agreement saying that if they ever use they ever use this functionality for anything other than CSAM or on anything other than iCloud photos, ever person who was subjected to this scan will be paid 100 million dollars by Apple. I will believe them if they put this clause in, and I know when they have changed their plans when the…

Until one day a box pops up. It says "We've updated our terms. See this 10,000 line document here. Please accept to continue using your device." Then your clause is gone.

I'm fine with that. Apple is a big company and changing its TOS will be instantly reported on. It will act as a canary of sorts to know when they turn evil and we know to stop using their products.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#286

Post by heavyset_go »

GeekyBear wrote: post #175
mightybyte wrote: post #162

Earlier quoted context omitted.

I think the difference here is that it's ON YOUR DEVICE. I think there's a pretty clear understanding that if you upload stuff to a cloud provider they can do whatever they want with it. This is different. This is reaching into what has up until now mostly been considered a private place. Law enforcement often has to get warrants to search this kind of thing. This is the difference between putting CSAM on a sign in y…

ON YOUR DEVICE (where it's encrypted in a way that Apple can't read until the 30 image threshold is crossed) is more private than doing the same scan on server where a single false positive can be misused by anyone who can get a subpoena.

> where it's encrypted in a way that Apple can't read

This doesn't matter because Apple can read iCloud data, including iCloud Photos. They hold the encryption keys, and they hand over customers' data for about 150,000 users/accounts a year in response to requests from the government[1].

[1] https://www.apple.com/legal/transparency/us.html

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#287

Post by treesprite82 »

floatingatoll wrote: post #56
smoldesu wrote: post #35

Earlier quoted context omitted.

Furthermore, it may well be possible to combine that blobby grey static with another image, manipulating it's visual hash to create a "sleeper" positive. If this was possible in a week , then it's going to be very interesting to watch the technology change/evolve over the next few years.

Doing so would create a positive that still doesn't pass Apple's human visual check against the (blurred) CSAM content associated with that checksum, and if it somehow did, it would still then also have to occur at qty.30 or more, and they'd have to pass a human visual check against the (unblurred) CSAM content by one of the agencies in possession of it. It's not possible to spoof that final test unless you possess r…

There are concerns without the images needing to make it all the way through their CSAM process. Apple is a US company with obligations to report certain crimes they become aware of, and NCMEC is a US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority".

Consider if the honeypot images (manipulated to match CSAM hashes) are terrorist recruitment material for example.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#288

Post by mirker »

helen___keller wrote: post #97
epistasis wrote: post #42

What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning. For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all. Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.

> I just don't understand how they acted this way at all. There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of…

> Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy

What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287...

The issues are also well known and encompass the whole subfield of adversarial examples.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#289

Post by shapefrog »

rlpb wrote: post #282
shapefrog wrote: post #231

Earlier quoted context omitted.

Your hash will match to say image 105 of the dataset. Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere. So at this point we have an image that computers think is CSAM and people think is CSAM, and when held up next to the original verified horrific image everyone agrees is the same image. At this point, someone is going to ask, rightly…

> Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere. Define "get anywhere". Why won't you get raided by the police and have all your devices seized first?

No, probably wouldnt even get to the desk of the police. End of the road would be NCMEC, to whom apple would refer hash matching images that pass human verification that are close enough to porn.

If your 30 or so hash matching images matched their corresponding known CSAM then that goes on to the police and then they knock on your door.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#290

Post by zepto »

fsflover wrote: post #269
zepto wrote: post #248

Earlier quoted context omitted.

> This is a new mechanism for scanning private pictures on the device. No it isn’t. It’s a mechanism for scanning pictures as they are uploaded to iCloud Photo Library. Private pictures on the device are not scanned.

Pictures not uploaded yet are private.

Not if you have opted to have them uploaded.
Post reply on HN