Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tainting the CSAM client-side scanning database

blog.xot.nl

Post reply on HN
Print view View original

251–260 of 276 posts

Re: Tainting the CSAM client-side scanning database

#251

Post by chatmasta »

Hizonner wrote: post #68
chatmasta wrote: post #44

Earlier quoted context omitted.

No, I don't support any automated scanning system or really any sort of "going out of our way" to find new criminals. The reason I think it's a bad idea to differentiate between real or AI is similar to the arguments against "means testing" for distributing benefits. You don't want to put real victims in a situation where they're deprived of justice because they can't prove that their victimization was "real." Imagin…

It's already a strict liability offense in many places, meaning that you don't even have to know that you possessed the image at all. You could apply the same strict liability standard and say that it doesn't matter that you didn't think it was real as long as it actually was real. "I thought she was 18" doesn't work for physical sex either.

This is the only reasonable comment in the entire thread. Make CP possession a matter of strict liability and all problems are solved.

Re: Tainting the CSAM client-side scanning database

#252

Post by astrange »

Hizonner wrote: post #217
astrange wrote: post #204

Earlier quoted context omitted.

Nothing in this post (either the system implementation details or the supposed consequences of detection) is how it actually works in reality, so I don't think you should spent much time being upset that something you just made up has a security hole in it. e.g. there isn't a high false positive rate, the attack in this article doesn't work because the attacker doesn't have access to all the hashing algorithms used,…

Um, the whole thing definitely is unaudited. > the attack in this article doesn't work because the attacker doesn't have access to all the hashing algorithms used As far as I know, there are only two hashing algorithms used: ContentID and "the Facebook one", whose name I don't remember offhand at the moment. ContentID has leaked, been reverse engineered, been published, and been broken. A method to generate collision…

> As far as I know, there are only two hashing algorithms used: ContentID and "the Facebook one", whose name I don't remember offhand at the moment.

Yes, those aren't suited for client-side scanning. If the server side can do any content scanning then you're not secure against them, so the protection isn't what kind of hashing they use, it's just that someone actually looks at the results.

> You can't keep something like that secret.

I didn't say it was secret, I said you don't have access to it. Well… that's kind of the same thing I guess, but anyway the important point is they can change it/reseed it.

> None of this "texts the police", but it does alert service providers who may delete files, lock accounts, or flag people for further surveillance and heightened suspicion.

Google has done this one looking for "novel CSAM" aka anyone's nudes, which is bad, so I recommend not doing that.

> Those clearinghouses usually aren't the police, but they're close.

No, it's extremely important that they're not the police (or other government organization); in the US NCMEC exists because, as they're a private organization, you get Fourth Amendment protections if you do get reported to them. But these systems don't automatically report to them either. Someone at the service looks at it first.

Re: Tainting the CSAM client-side scanning database

#253

Post by astrange »

Hizonner wrote: post #219
astrange wrote: post #199

Earlier quoted context omitted.

> and if the platform fulfills its obligations, that report will bubble up to the relevant authorities, who will review it and add its fingerprint to the database. I don't think NCMEC adds random images they find to the A1 list without knowing their origin. > until the average meme-savvy environmental protester's device gets flagged for further scrutiny. Who is this an attack on? A moderation contractor maybe, but no…

It means your account gets a higher risk score, which may mean it gets given a "timeout", gets downranked in "the algorithm", gets outright shadowbanned, or may even get completely shut off. All in a completely automated way.

Nobody is "downranking users in the algorithm" because they think the user /is a pedophile/. That is mixing up social media controversies.

Your account could certainly be locked until someone looks at it though, yes.

Re: Tainting the CSAM client-side scanning database

#254

Post by astrange »

Retr0id wrote: post #221
astrange wrote: post #199

Earlier quoted context omitted.

> and if the platform fulfills its obligations, that report will bubble up to the relevant authorities, who will review it and add its fingerprint to the database. I don't think NCMEC adds random images they find to the A1 list without knowing their origin. > until the average meme-savvy environmental protester's device gets flagged for further scrutiny. Who is this an attack on? A moderation contractor maybe, but no…

Regarding the first part, if that really is the relevant policy, fair enough. I don't know the specific policies used to include or exclude a given image. Where can I find the details? I consider someone looking at pictures from my local device without my consent to be an attack on my privacy, regardless of the content, or whether they send me to jail afterwards. The reason apple's threshold exists in the first place…

> Regarding the first part, if that really is the relevant policy, fair enough. I don't know the specific policies used to include or exclude a given image. Where can I find the details?

I'm not sure either or I would've linked it. I have implemented such a reporting system though (compliant to US law, which is privacy-preserving relatively speaking, not any upcoming EU laws, which like all other EU law seems like a huge pain to live under.)

> I consider someone looking at pictures from my local device without my consent to be an attack on my privacy, regardless of the content, or whether they send me to jail afterwards.

To be clear, this is only if you're using a cloud storage service like iCloud Photos, Google Drive etc. It's meant to be a strict improvement over the usual setup, which is that your data isn't hidden from the cloud provider at all and they can just look at whatever. It would certainly be had to have any scanning if you're not opting into a cloud service.

Re: Tainting the CSAM client-side scanning database

#255

Post by astrange »

LorenPechtel wrote: post #215
astrange wrote: post #199

Earlier quoted context omitted.

> and if the platform fulfills its obligations, that report will bubble up to the relevant authorities, who will review it and add its fingerprint to the database. I don't think NCMEC adds random images they find to the A1 list without knowing their origin. > until the average meme-savvy environmental protester's device gets flagged for further scrutiny. Who is this an attack on? A moderation contractor maybe, but no…

Look at some of the cases in the news--the tech giants are more interested in ensuring that bad guys don't use their systems than in justice. Remember that case not too long ago with a telehealth appointment, they sent a picture of something on their 2? year old's penis to the doc, asking if it was an issue. The police cleared him, but he's forever guilty in Google's eyes.

Google was not required to implement that the way they did, which is basically a server-side scanning policy of having contractors look at your nudes if you put them on your cloud drive. NCMEC may approve of it, sure, but another aspect of their not being a government agency is that you don't have to listen to them.

Re: Tainting the CSAM client-side scanning database

#256

Post by grayhatter »

sterlind wrote: post #172
grayhatter wrote: post #122

Earlier quoted context omitted.

PhotoDNA doesn't use CS hashes, but others I've looked at do use them. But you're right, you probably don't wanna respond to my parody of you suggestion to trade in CSAM...

You're saying that there are cryptographically-secure PhotoDNA-like hash algorithms? That's remarkable, if true, though I'm not sure if the schemes you're referring to are resistant to preimage recovery or hash collision (different things!)

I dont want to oversell it, having really only seen what I might consider marketing material... I'm pretty sure they're only using a CS hash for the database, where the photo hash is similar to photodna, and they're only adding some indirection.

My assumption was something like sha_something(photodna(image))

Re: Tainting the CSAM client-side scanning database

#258

Post by logi_sko »

Retr0id wrote: post #145

The article considers "an entity that is allowed to propose new entries to the CSAM database". You don't even need this! You could target a whole "social cluster" of people without having any special privileges within this system. As an example, lets say you want to attack environmental protesters. For image A, you create a meme about climate change. For image B, you procure something that looks, to humans, like CSAM…

Even better: Distribute image A for a while before publishing B' and it can spread via all the server side scanning channels as well. Then when f(B')=f(A) is added to the database, people sse suddenly caught with apparently legal images.

Re: Tainting the CSAM client-side scanning database

#259

Post by logi_sko »

phyzome wrote: post #183
quitit wrote: post #164

I am against the idea of scanning for the reason that the author pointed out: It's trivial to repurpose the technology to use it in dystopian ways. I however have precisely zero concerns about impersonating hashes: 1. It's trivial to deal with tainting the database: both secondary hashing and more invasive hashes deal with that problem. 2. It's trivial to deal with impersonated hashes, all positives can be scanned on…

If you can make one algorithm collide, you can make two collide.

It needs to be not just a second algorithm but a parametric hash function. So after an image B has been found which matches illegal image A on f(B)=f(A) then the server picks a random number R and sends the client R,f(A,R) and the client checks if f(B,R) =f(A,R) as well.

Do we have such a hash function?

Re: Tainting the CSAM client-side scanning database

#260

Post by logi_sko »

astrange wrote: post #200
phyzome wrote: post #183

Earlier quoted context omitted.

If you can make one algorithm collide, you can make two collide.

You can't because you don't have access to one of the algorithms.

We don't actually have access to the first algorithm either, but it's been reverse engineered and a binary published. According to article.
Post reply on HN