Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tainting the CSAM client-side scanning database

blog.xot.nl

Post reply on HN
Print view View original

61–70 of 276 posts

Re: Tainting the CSAM client-side scanning database

#61

Post by dragonwriter »

londons_explore wrote: post #60
chatmasta wrote: post #32

Earlier quoted context omitted.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that I disagree. The point is to reduce actual child abuse. The images are in a way only tangential. If an image is made with an AI with no actual child being abused, then it shouldn't be a crime. In a way, it's better , because it will distract the crowd of people into this sort of stuff from activit…

> because it will distract the crowd of people into this sort of stuff from activities that harm real people.

This is actually the main point in dispute, and almost everyone arguing one side or the other on this topic seems to assume one side or the other on this point and argue from there, rather than seeking to support their position on the fundamental disputed fact question.

Which results in the most of the debate being people talking past each other based on conflicting assumptions of fact.

Re: Tainting the CSAM client-side scanning database

#62

Post by chatmasta »

sneak wrote: post #53
chatmasta wrote: post #32

Earlier quoted context omitted.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> Possession of CSAM should be illegal regardless of whether it's "real" or not. From a purely ethical standpoint: why? What is the purpose of punishing someone who has harmed no one? No victim means no crime.

From a purely ethical standpoint, sure, I agree. But we live in reality, and there are plenty of activities that seem ethically victimless, but are practically necessary to criminalize, in order to uphold societal frameworks and expectations of morality.

In this case, by giving every CSAM criminal a potential excuse that they "thought it was AI generated," the real victims are further victimized by being deprived of justice or forced to prove their realness.

Re: Tainting the CSAM client-side scanning database

#63

Post by Chabsff »

londons_explore wrote: post #60
chatmasta wrote: post #32

Earlier quoted context omitted.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that I disagree. The point is to reduce actual child abuse. The images are in a way only tangential. If an image is made with an AI with no actual child being abused, then it shouldn't be a crime. In a way, it's better , because it will distract the crowd of people into this sort of stuff from activit…

You may think that intuitively, but actual studies actually indicate the opposite. Usage of CSAM material leads to increased risks of contacting and abusing children.

This needs to be balanced against rights to privacy and expression, which I personally think take precedence, but pretending that it can serve as harm reduction is just not correct.

Re: Tainting the CSAM client-side scanning database

#64

Post by [deleted] »

miohtama wrote: post #2

Because client-side scanning is not going to work, and no one wants to government issued black box binary to send their conversations and photos to unnamed police person randomly, the non-compliance is the only way. People just start to use chat programs in the EU that do not comply. This would be Signal, Telegram, others. The EU can fine and fight with Meta/WhatsApp, Apple, others, but that’s about it. The EU bureau…

[deleted]

Re: Tainting the CSAM client-side scanning database

#65

Post by regularjack »

Hizonner wrote: post #43

It says: > This shows that the database can be tainted with non-CSAM material by an entity that can submit entries to it. Actually, it can easily be tainted by anybody . Take your massaged hash-colliding image, which remember is still visually child porn , and post it on some pedos-R-us forum. The people who maintain the database actively troll those forums. They'll see the image and add the hash to the database for…

You'd be publishing child porn, which I think is not the wisest thing to be doing.

Re: Tainting the CSAM client-side scanning database

#66

Post by dragonwriter »

chatmasta wrote: post #62
sneak wrote: post #53

Earlier quoted context omitted.

> Possession of CSAM should be illegal regardless of whether it's "real" or not. From a purely ethical standpoint: why? What is the purpose of punishing someone who has harmed no one? No victim means no crime.

From a purely ethical standpoint, sure, I agree. But we live in reality, and there are plenty of activities that seem ethically victimless, but are practically necessary to criminalize, in order to uphold societal frameworks and expectations of morality. In this case, by giving every CSAM criminal a potential excuse that they "thought it was AI generated," the real victims are further victimized by being deprived of…

Broadening the definitions of crime to make it easier to punish the ethically guilty on scant evidence while incidentally sweeping up the ethically innocent is a hack around a legal tradition that is designed exactly on the principal that it is better that the guilty go unpunished than the innocent are punished, by making the genuinely innocent administratively guilty, and we ought to reject that kind of justification every time it rears its head.

(There are times when it is important to have commonality while the choice of the common practice isn't important, which justifies regulations of obviously ethically unimportant things like "which side of the road is it correct to drive on relative to the direction of travel"; but where the purpose of a crime is purely to lower the evidentiary bar to punish people presumed guilty of a narrower crime, that's just an attempt to hack around the presumption of innocence and the burden of proof of guilt.)

Re: Tainting the CSAM client-side scanning database

#68

Post by Hizonner »

chatmasta wrote: post #44
postalrat wrote: post #40

Earlier quoted context omitted.

Seems like you want to bring all the success of the war on drugs to the war on generated images.

No, I don't support any automated scanning system or really any sort of "going out of our way" to find new criminals. The reason I think it's a bad idea to differentiate between real or AI is similar to the arguments against "means testing" for distributing benefits. You don't want to put real victims in a situation where they're deprived of justice because they can't prove that their victimization was "real." Imagin…

It's already a strict liability offense in many places, meaning that you don't even have to know that you possessed the image at all. You could apply the same strict liability standard and say that it doesn't matter that you didn't think it was real as long as it actually was real.

"I thought she was 18" doesn't work for physical sex either.

Re: Tainting the CSAM client-side scanning database

#69

Post by usrnm »

chatmasta wrote: post #47
epups wrote: post #42

Earlier quoted context omitted.

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Why do you believe that?

See my comment to a sibling reply. Basically I don't want to make victims prove their victimization was real, and I don't want to give criminals with real victims an opportunity to argue they "thought it was AI generated."

And I absolutely want both parties to have to prove crime/innocence and have an opportunity to argue. The current situation, where anything involving CSAM is so toxic that lives are ruined without trial is not healthy and isn't good for anybody

Re: Tainting the CSAM client-side scanning database

#70

Post by Hizonner »

regularjack wrote: post #65
Hizonner wrote: post #43

It says: > This shows that the database can be tainted with non-CSAM material by an entity that can submit entries to it. Actually, it can easily be tainted by anybody . Take your massaged hash-colliding image, which remember is still visually child porn , and post it on some pedos-R-us forum. The people who maintain the database actively troll those forums. They'll see the image and add the hash to the database for…

You'd be publishing child porn, which I think is not the wisest thing to be doing.

Tainting the database is probably already chargeable as something. The point is that the person doing this doesn't expect to get caught. And frankly they're probably right in that.
Post reply on HN