Tainting the CSAM client-side scanning database • Hacker News Board

Re: Tainting the CSAM client-side scanning database

#261

Post by Zuiii » Sat, Oct 14, 2023, 4:58 AM UTC

I'm completely stumped. I just don't understand how politicians can still believe that client-side scanning would somehow stop CSAM.

Couldn't those who possess this material also run those exact same CLIENT scanners (if not directly, than via isolated canary systems and emissions monitoring) to detect when files have been added to the scanners?

What about tools that keep shuffling files in ways that are not too perceivable but would defeat those scanners? Just look at copyright infringing material on Youtube (it's still there, just better hidden).

Nothing short of a government whitelist database containing all files that are legal to possess, or 100% analog human surveillance will solve CASM. Not AI and certainly not this.

I'm forced to agree with others here: This isn't about CSAM at all. It's a thinly disguised frontal attack on democracy. Imagine if the government had this technology when people were organizing to repeal discriminatory laws in the US. Imagine if authoritative governments in the east (or anywhere else) had this technology right now.

Take a long hard look at every person or organization who supports this nonsense and pay close attention to them. They are the ones who want to harm us and our children.

Re: Tainting the CSAM client-side scanning database

#262

Post by gambiting » Sat, Oct 14, 2023, 6:34 AM UTC

SV_BubbleTime wrote: post #239

gambiting wrote: post #128

Earlier quoted context omitted.

>> Imagine a real CSAM criminal claiming a defense that they "thought it was AI generated." Saying "I thought this heroin was fake" is not a defense when caught with a bag of heroin, I don't see how this would be any different. It's not a magic out for anyone.

That isn’t the argument at all. It’s that you have a bag of fake heroin, you are then arrested for it because someone thinks that by you having fake heroin you are encouraging real heroin users to do more real heroin.

Well yes, which is why my argument(sorry if it wasn't clear) is that having fake heroin shouldn't be illegal.

Re: Tainting the CSAM client-side scanning database

#263

Post by rany_ » Sat, Oct 14, 2023, 8:38 AM UTC

OfSanguineFire wrote: post #72

rany_ wrote: post #59

Earlier quoted context omitted.

Their reasoning for having it Play Store only is that Google does not require app developers to provide their signing keys/sign the APK themselves. Now this is no longer the case as Google changed their policies. NOTE: F-Droid does not require app developers to share their keys, instead they build the application themselves and sign it with their keys -- something Signal is not a fan of. tl;dr their rationale no long…

I have also seen the preference for installing from Play Store explained as being that users are safer if they use default setups and do what everyone else is doing. For users who are not savvy, going the sideloading route can expose them to risk.

For me this line of thinking doesn't make a lot of sense. The Google Play option will always exist as it is a prerequisite for widespread adoption on Android. Also arguably, you're safer on F-Droid due to the level of vetting on that platform; so providing Signal on it is a good endorsement for general user safety and privacy.

Either way, this isn't about only offering the application on Google Play or only on F-Droid; but providing the option for both. Non-tech savvy users will always pick the more familiar and easy option on average, Google Play.

Re: Tainting the CSAM client-side scanning database

#264

Post by MagicMoonlight » Sat, Oct 14, 2023, 10:59 AM UTC

The database would obviously come from a central organisation like the FBI or NCA and be built from material gathered in investigations. They wouldn’t allow larry from reddit to share his child porn with them and break the database.

Anyone with sufficient power to want to target you wouldn’t use a ridiculous exploit like this. They’ll just go to google and say “change the update server for this user to this new address” and then breach your device with the next software update.

Re: Tainting the CSAM client-side scanning database

#265

Post by jopsen » Sat, Oct 14, 2023, 11:24 AM UTC

pmontra wrote: post #22

The only thing that can save us is a huge number of false positives and a backslash on the governments in charge, when they'll have to concede for the 10,000th time that it was not CSAM but parents sending their baby pictures to the grandparents.

Looking at the cookie banner stupidity, I don't think a huge number of false positives would generate backlash.

Re: Tainting the CSAM client-side scanning database

#266

Post by tuhriel » Sat, Oct 14, 2023, 11:49 AM UTC

astrange wrote: post #255

LorenPechtel wrote: post #215

Earlier quoted context omitted.

Look at some of the cases in the news--the tech giants are more interested in ensuring that bad guys don't use their systems than in justice. Remember that case not too long ago with a telehealth appointment, they sent a picture of something on their 2? year old's penis to the doc, asking if it was an issue. The police cleared him, but he's forever guilty in Google's eyes.

Google was not required to implement that the way they did, which is basically a server-side scanning policy of having contractors look at your nudes if you put them on your cloud drive. NCMEC may approve of it, sure, but another aspect of their not being a government agency is that you don't have to listen to them.

Problem is, as they are not a goverment agency and don't really see you as customer (more like a product) you have almost no avenue to get your account reinstated...

And as there are some people (and even companies) who rely on google services and their account there it can really shoot you in the knee

Re: Tainting the CSAM client-side scanning database

#267

Post by tuhriel » Sat, Oct 14, 2023, 11:50 AM UTC

astrange wrote: post #253

Hizonner wrote: post #219

Earlier quoted context omitted.

It means your account gets a higher risk score, which may mean it gets given a "timeout", gets downranked in "the algorithm", gets outright shadowbanned, or may even get completely shut off. All in a completely automated way.

Nobody is "downranking users in the algorithm" because they think the user /is a pedophile/. That is mixing up social media controversies. Your account could certainly be locked until someone looks at it though, yes.

Yes, and we all know how easy it is to get a real person from the big companies to take a look at something... Or even reach one

Re: Tainting the CSAM client-side scanning database

#268

Post by tuhriel » Sat, Oct 14, 2023, 11:56 AM UTC

kelnos wrote: post #230

Hizonner wrote: post #176

Earlier quoted context omitted.

> That would be ridiculous. Why is it ridiculous, in a world where the penalty for sharing a single music file is $250,000?

That's an argument to reduce the penalty for sharing a single music file, not for making the penalty for a false report also ridiculous. Then again, I personally do think that knowingly filing a false report of law-breaking should be treated as a very serious crime. I think the harm of filing a false report of copyright infringement is greater than the act of infringing copyright itself.

I mean it should be AT LEAST be handled like a copyright infringement itself. As you propose as owner of the copyright...

For stuff that is actually copyrighted that probably could only be triggered by the actual rights holder.

Re: Tainting the CSAM client-side scanning database

#269

Post by LorenPechtel » Sat, Oct 14, 2023, 3:28 PM UTC

gambiting wrote: post #238

LorenPechtel wrote: post #231

Earlier quoted context omitted.

Sure it is. The bag of "heroin" on the movie set turns out to be real, think the actors are going to be convicted of possession?

I'm not sure what is the point that you are making here. An actor who was given a fake bag of heroin as a prop which then turns out to be real is no more guilty than a courier moving a package that happens to contain drugs or guns or fake money or anything else - neither would be found guilty of posession. These are situational circumstances, and no prosecutor in the world would choose to prosecute those - but there…

I was simply providing an example of where "I thought it was fake" would be a reasonable defense.

Re: Tainting the CSAM client-side scanning database

#270

Post by quitit » Sat, Oct 14, 2023, 9:17 PM UTC

astrange wrote: post #200

phyzome wrote: post #183

Earlier quoted context omitted.

If you can make one algorithm collide, you can make two collide.

You can't because you don't have access to one of the algorithms.

In online people sometimes write things the way they want them to be, not the way they factually are. In small ways it comes across as wishful thinking, in other ways it's just plain old lying.

I didn't feel that the comment trivialising multiple hash collisions was worth a direct reply, because the author is clearly not writing with bona fide intentions, or any real knowledge about an image scanning system. It's the fingers in ears, head in the sand kind of denialism that adds nothing to the conversation.

My earlier comment already addressed what happens after the hashes match: a visual comparison. This isn't an original concept, it's a standard approach and part of earlier CSAM scanning proposals.

One needs to act rather barefaced to pretend that a matched hash alone will have consequences, when we already have established that false positives are the known downside to using hash-based image matching.