Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tainting the CSAM client-side scanning database

blog.xot.nl

Post reply on HN
Print view View original

141–150 of 276 posts

Re: Tainting the CSAM client-side scanning database

#141

Post by anigbrowl »

londons_explore wrote: post #60
chatmasta wrote: post #32

Earlier quoted context omitted.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that I disagree. The point is to reduce actual child abuse. The images are in a way only tangential. If an image is made with an AI with no actual child being abused, then it shouldn't be a crime. In a way, it's better , because it will distract the crowd of people into this sort of stuff from activit…

You're assuming the truth of your conclusion without testing it. It's equally possible that encountering AI-CSAM is going to incentivize collectors to pay a premium for 'the real stuff', just as many CSAM collectors end up getting caught when they try to make the leap into engaging in abusive activities for real. Your mental model of how CSAM enthusiasts think isn't anchored in reality.

Re: Tainting the CSAM client-side scanning database

#142

Post by j16sdiz »

novok wrote: post #140
flkenosad wrote: post #134

Earlier quoted context omitted.

Wow what a mess. At what point do we move on from copyright laws? Or more broadly intellectual property, in general. Even the words "intellectual property" sound ridiculous together when you think about it.

If you think deeper about it, the general concept of property is similarly ridiculous too. An arbitrary piece of land being ‘owned’ is similarly arbitrary human social concept, enforced by law and a registry

Music have no registry. It is much worse than land property

Re: Tainting the CSAM client-side scanning database

#143

Post by mandelken »

slg wrote: post #130
Kevin605 wrote: post #12

Earlier quoted context omitted.

The general public submitting CSAM directly would indeed be highly unlikely, but the scenario we need to consider involves those in positions of authority who can manipulate systems behind the scenes. Imagine that an unflattering or satirical image of Viktor Orban is circulating in France, and let's say it becomes viral, inciting discussions that the Hungarian government finds detrimental to its international image.…

>The Hungarian government - who presumably has access to the EU CSAM database (or can coerce those who do), might attempt to add a fingerprint of a manipulated CSAM image that collides with the fingerprint of the satirical image. Then what? What does that achieve? There would be a huge spike in images identified as CSAM which would obviously throw up red flags. It seems like this would mostly just be headache for the…

One idea would be for the government of Hungary to create a list of its citizens that share this inciting, dangerous or whatever you you wanna call it material. If they keep on finding the same persons distributing multiple times, they may pay them a visit, get them fired from their government job, block their bank accounts, put them on the no-fly list or whatever.

And that’s just one idea, probably there’s others.

Re: Tainting the CSAM client-side scanning database

#144

Post by ramses0 »

cabirum wrote: post #36

I think it's pretty clear this is not about "CSAM", we have to stop using the term. It's just censorship, plain and simple. Client side means you'll pay from your own pocket for this wrongthing detector to work. It can even be automated so as soon as the detector gets triggered by anything, you'll get locked out of your bank accounts, until further notice I guess. If this thing gets a serious discussion in a parliame…

But parliament wants to seed your camera with mugshots of the FBI's top-ten most wanted list so the instant a false positive appears (directly on the camera, potentially even prior to writing the image to disk, potentially even prior to pressing the snapshot button)... they beacon an alert (or exfiltrate piggybacking via Bluetooth/AirTag/Covid exposure tracking mrchanisms), and bob's you're uncle.

Re: Tainting the CSAM client-side scanning database

#145

Post by Retr0id »

The article considers "an entity that is allowed to propose new entries to the CSAM database".

You don't even need this! You could target a whole "social cluster" of people without having any special privileges within this system.

As an example, lets say you want to attack environmental protesters.

For image A, you create a meme about climate change.

For image B, you procure something that looks, to humans, like CSAM (as described in the article).

Craft B′ such that f(B′) = f(A) (also as described).

Now, all you have to do is anonymously publish B′ to a platform that is actively moderated/monitored. Unfortunate users will see it and report it as CSAM, and if the platform fulfills its obligations, that report will bubble up to the relevant authorities, who will review it and add its fingerprint to the database.

Now you can start sending out image A, the meme, to your target demographic. You won't be able to post it on "mainstream" platforms with server-side fingerprint scanning, but there are plenty of other avenues for it to spread. If it's a good meme, it will propagate organically through group-chats and DMs, and eventually find its way onto devices with client-side scanning.

Apple's proposed device scanning system had a threshold before your device would be flagged, so repeat all these steps a whole bunch of times, until the average meme-savvy environmental protester's device gets flagged for further scrutiny.

Unwitting victims who do try to post the meme to a mainstream platform with fingerprint matching may risk getting their accounts flagged and taken down, and they might have no way of knowing what triggered it. This would lead to, of course, automated censorship of environmental protest groups.

Re: Tainting the CSAM client-side scanning database

#146

Post by mandelken »

pornel wrote: post #48
bastawhiz wrote: post #5

The issue described here, to my understanding, is that you find or create csam and then manipulate it so that its fingerprint collides with another image that you want to be flagged as csam. You then submit the manipulated version of the found or generated image to the authority. First, at what point does the authority go "uh... Where did you get this from?" Practically speaking, the people doing this would have to b…

Abusers who have actual CSAM could intentionally publish the fingerprints to sabotage the scanning scheme. If the illegal fingerprints become known, it will be possible to generate false positives and overwhelm verification/enforcement with bogus matches.

Damn, a bit far fetched but theoretically possible. Jikes.

Re: Tainting the CSAM client-side scanning database

#147

Post by 0xffff2 »

gnopgnip wrote: post #137
viridian wrote: post #78

Earlier quoted context omitted.

Not sure if its what the poster above is talking about, but there's definitely a hash collision type attack that's common on Youtube with regards to classical music. The attacker in question uploads very standard renditions of thousands of pieces of classical music, and claims copyright on them. Content ID then flags any video using one of these pieces as potentially violating the rights of the rightholder. The attac…

That’s not what a hash collision is. Uploading popular (public domain) music and claiming you own it is just fraud

I don't have any direct knowledge of this, but I assume the hash collision comes in because Youtube uses some kind of fuzzy hash, so a generic music performance will flag videos using the same piece of music, but not necessarily the same performance.

Re: Tainting the CSAM client-side scanning database

#148

Post by adrr »

sneak wrote: post #53
chatmasta wrote: post #32

Earlier quoted context omitted.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> Possession of CSAM should be illegal regardless of whether it's "real" or not. From a purely ethical standpoint: why? What is the purpose of punishing someone who has harmed no one? No victim means no crime.

Plenty of actions are crimes without real victims. Not having insurance while driving is an example. Possession of explosives is another one.

Re: Tainting the CSAM client-side scanning database

#149

Post by opan »

flkenosad wrote: post #134
viridian wrote: post #78

Earlier quoted context omitted.

Not sure if its what the poster above is talking about, but there's definitely a hash collision type attack that's common on Youtube with regards to classical music. The attacker in question uploads very standard renditions of thousands of pieces of classical music, and claims copyright on them. Content ID then flags any video using one of these pieces as potentially violating the rights of the rightholder. The attac…

Wow what a mess. At what point do we move on from copyright laws? Or more broadly intellectual property, in general. Even the words "intellectual property" sound ridiculous together when you think about it.

>Even the words "intellectual property" sound ridiculous together when you think about it.

For this reason, many would suggest not using it. It's a vague way of combining the separate issues of copyright, patents, and trademarks. It also illegitimately tries to equate those things to property, which changes how many feel about it.

https://www.gnu.org/philosophy/words-to-avoid.html#Intellect...

Re: Tainting the CSAM client-side scanning database

#150

Post by SV_BubbleTime »

varispeed wrote: post #73

Ok, but why not go for routine home searches then? Is it because on the phone, it is "invisible" and cheap? Does it mean that what stops the government from treating everyone as a potential criminal is cost and inconvenience? People were raising alarms about this years ago and were branded as conspiracy theorists. I guess the EU works well in their propaganda department painting themselves as do gooders, where in fac…

>Ok, but why not go for routine home searches then? Is it because on the phone, it is "invisible" and cheap?

Underrated comparison. That is apt and the whole “for the children” part is smokescreen.

Post reply on HN