Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tainting the CSAM client-side scanning database

blog.xot.nl

Post reply on HN
Print view View original

181–190 of 276 posts

Re: Tainting the CSAM client-side scanning database

#181

Post by Bewelge »

viridian wrote: post #78
nopurpose wrote: post #57

Earlier quoted context omitted.

what is happening in Youtube world?

Not sure if its what the poster above is talking about, but there's definitely a hash collision type attack that's common on Youtube with regards to classical music. The attacker in question uploads very standard renditions of thousands of pieces of classical music, and claims copyright on them. Content ID then flags any video using one of these pieces as potentially violating the rights of the rightholder. The attac…

I have some, though maybe outdated, experience with that: In 2008, I was 15 and had the idea of creating youtube videos using MIDI files of classical piano songs that had entered the public domain. They're free to use and in theory, videos stay up forever and these classical songs are timeless, so that means after uploading I'd earn passive income forever!

Or so I thought... Every single video was copyright claimed. Youtube falsely thought it was a recording (which do have copyright). The second a claim was created I stopped earning money. I would then dispute the claim, explaining that it's not even a sound recording and they had a month to react to my answer.

In almost all instances the claims were retracted but it would only take a couple of days until another claim would be filed on that video. Answering all those claims took up more time than producing the videos so I gave up after a while.

But in my experience it wasn't really malicious actors filing these claims but youtube's filter simply not managing to distinguish between a recording of a classical piano piece and a well synthesized version of it. I actually remember most claims being filed by Sony.

It's interesting to hear that it's still this way because I don't really believe that nowadays, youtube still can't distinguish between different recordings of classical music. I guess they have no incentive to improve in that regard.

But I think the fact that nobody but the large labels are able to earn money with classical music on these large platforms is actually an excellent argument against upload filters and in my experience it's an argument that non-tech people can much better relate to than hash-collisions.

Re: Tainting the CSAM client-side scanning database

#182

Post by what-no-tests »

INTPenis wrote: post #180
cabirum wrote: post #36

I think it's pretty clear this is not about "CSAM", we have to stop using the term. It's just censorship, plain and simple. Client side means you'll pay from your own pocket for this wrongthing detector to work. It can even be automated so as soon as the detector gets triggered by anything, you'll get locked out of your bank accounts, until further notice I guess. If this thing gets a serious discussion in a parliame…

What makes the debate confusing is that some people probably believe it is about CSAM. Like the Swedish PM Ylva Johansson for example. She probably believes in what she's doing, and other forces are simply taking advantage of her crusade.

That's very generous of you.

Re: Tainting the CSAM client-side scanning database

#183

Post by phyzome »

quitit wrote: post #164

I am against the idea of scanning for the reason that the author pointed out: It's trivial to repurpose the technology to use it in dystopian ways. I however have precisely zero concerns about impersonating hashes: 1. It's trivial to deal with tainting the database: both secondary hashing and more invasive hashes deal with that problem. 2. It's trivial to deal with impersonated hashes, all positives can be scanned on…

If you can make one algorithm collide, you can make two collide.

Re: Tainting the CSAM client-side scanning database

#184

Post by minitech »

olliej wrote: post #129
lelag wrote: post #45

Earlier quoted context omitted.

But then bypassing the filter would just require to change a few bits here and there.

No the whole purpose of these “hashes” is that they’re robust to that. The attack model they’re designed for is image manipulation to avoid the hash match, not manipulating manipulating images to trigger the hash. There are numerous papers on doing bit manipulation to cause miss classification.

The context of the comment you’re replying to is a cryptographic hash.

Re: Tainting the CSAM client-side scanning database

#185

Post by phyzome »

grayhatter wrote: post #103
Hizonner wrote: post #43

It says: > This shows that the database can be tainted with non-CSAM material by an entity that can submit entries to it. Actually, it can easily be tainted by anybody . Take your massaged hash-colliding image, which remember is still visually child porn , and post it on some pedos-R-us forum. The people who maintain the database actively troll those forums. They'll see the image and add the hash to the database for…

[flagged]

It's not a "cryptographically secure hashing system". PhotoDNA belongs to a separate class of algorithms called perceptual hashes. They're not particularly collision-resistant.

Re: Tainting the CSAM client-side scanning database

#186

Post by trehalose »

londons_explore wrote: post #60
chatmasta wrote: post #32

Earlier quoted context omitted.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that I disagree. The point is to reduce actual child abuse. The images are in a way only tangential. If an image is made with an AI with no actual child being abused, then it shouldn't be a crime. In a way, it's better , because it will distract the crowd of people into this sort of stuff from activit…

> In a way, it's better, because it will distract the crowd of people into this sort of stuff from activities that harm real people.

I see that assertion a lot. If that's how that works, why does the very large amount of CSAM already in existence not have the same effect? Why would synthesized CSAM distract pedophiles from their activities when real CSAM from their fellow pedophiles doesn't?

Re: Tainting the CSAM client-side scanning database

#187

Post by trehalose »

flkenosad wrote: post #161
dragonwriter wrote: post #61

Earlier quoted context omitted.

> because it will distract the crowd of people into this sort of stuff from activities that harm real people. This is actually the main point in dispute, and almost everyone arguing one side or the other on this topic seems to assume one side or the other on this point and argue from there, rather than seeking to support their position on the fundamental disputed fact question. Which results in the most of the debate…

Exactly. It may distract a crowd of people into something less harmful. Or, it may perpetuate a behaviour sort of like the commonness of cigarettes leads to more people craving nicotine.

I think it's worth asking: Does synthesized CSAM have an "advertising" effect for real CSAM and CSA?

Re: Tainting the CSAM client-side scanning database

#188

Post by Genghis_Khan »

giantrobot wrote: post #159
chatmasta wrote: post #112

Earlier quoted context omitted.

What if a police officer generates some AI CSAM and then sells it to someone who thinks it's real? There's still "no victim," but the buyer thinks that there was. Are they guilty of a crime? Your logic would seem to imply that there's no crime with possession of real CSAM either, and that the only crime lies with the original abuser who took the pictures.

> Are they guilty of a crime? Unless there is a very specific "attempt to acquire CSAM" law then no they're not fucking guilty of any crime. If you live in a state where marijuana is illegal and you smoke some oregano because you thought it was marijuana you're not guilty of actually possessing marijuana. A criminal law is composed of a number of individual statutes. When a state is trying to prosecute someone for a…

> If a cop sells you oregano and you think it's marijuana [...]

It wasn't a cop, but I recall a case some years back when someone sold something as cocaine when it wasn't. Among other things, he went down for fraud.

Re: Tainting the CSAM client-side scanning database

#189

Post by Eisenstein »

chatmasta wrote: post #156
adrr wrote: post #148

Earlier quoted context omitted.

Plenty of actions are crimes without real victims. Not having insurance while driving is an example. Possession of explosives is another one.

In fact, you might even argue that possessing real CSAM has no victim. After all, the person possessing the image isn't the one who committed the abuse and took a picture of it, right? But we've collectively decided that it's worth punishing that crime, because every viewer is an enabler of the abuser. The same logic should extend to AI-generated content. To put it another way, consider a thought experiment where a p…

You have literally proposed 'thought crimes'.

Re: Tainting the CSAM client-side scanning database

#190

Post by trehalose »

giantrobot wrote: post #159
chatmasta wrote: post #112

Earlier quoted context omitted.

What if a police officer generates some AI CSAM and then sells it to someone who thinks it's real? There's still "no victim," but the buyer thinks that there was. Are they guilty of a crime? Your logic would seem to imply that there's no crime with possession of real CSAM either, and that the only crime lies with the original abuser who took the pictures.

> Are they guilty of a crime? Unless there is a very specific "attempt to acquire CSAM" law then no they're not fucking guilty of any crime. If you live in a state where marijuana is illegal and you smoke some oregano because you thought it was marijuana you're not guilty of actually possessing marijuana. A criminal law is composed of a number of individual statutes. When a state is trying to prosecute someone for a…

> If a cop sells you oregano and you think it's marijuana you might have the intent to buy marijuana but there's no actual criminal act because oregano isn't illegal. If you make a law that only requires intent then congratulations, you've created thought crimes.

You're a lawyer, I take it? I'm not a lawyer, and I admit your analysis of this scenario confuses me. Is there no legal difference between merely having intent to commit a crime at some point in the future, and actually attempting to commit a crime?

Post reply on HN