Tainting the CSAM client-side scanning database

Exactly. It may distract a crowd of people into something less harmful. Or, it may perpetuate a behaviour sort of like the commonness of cigarettes leads to more people craving nicotine.

I agree that it's not hash collision, but the fingerprinting strategy makes precise language on this topic a bit difficult/tedious.

The CSAM detection isn't proper hash collision either, in so far as I understand it. There's some fuzzy matching formula that generates the fingerprint, it's not simply a byte for byte hash taken of the image, and therein lies the comparison.

The fraud in question is reliant on content ID attempting to fuzzy match audio content, in this specific instance.

>Yes, the database maintainers would notice, but it could take them a while to get around to it. And they're not going to be eager to remove a collision, because that would effectively "legalize" the child porn member of the image pair. There are lots of images that could be useful to suppress temporarily.

They may not know it immediately, but the actual CSAM image wouldn't actually be shared in any real numbers which removes much of the concern with "legalizing" the image. You can argue this makes this system ineffectual, but that also means this isn't repeatable since weakening the impact of this system would quickly result in Hungary losing the power to add hashes to the database.

>And if you actually succeed in suppressing the false-positive image, it may not get passed around very fast, and it won't get vastly more hits than real target images, so it will take longer for anybody to notice or care.

Who is sharing the real target image? Is Hungary now an active creator and distributor of actual CSAM in addition to manipulating the database?

>There's also a possible end game where the child porn traders start perturbing their images to collide with really common images like flags, corporate logos, iconic movie stills, and whatever else. So now you either have to ban the US flag, or let this or that actual child porn image go.

Once again, this behavior would get Hungary booted pretty quickly. Also it is important to remember these are hashes. Not all images of the US flag would trigger the system only that specific image that has a hash collision.

>I don't actually think that the false hits would ruin very many lives in most places. But it's worth noticing that the original article was talking about authoritarian regimes repurposing the system without the consent of the database maintainers. In the Orbán example, it's possible that the system might flag you for child porn, but you might actually get arrested for sedition. And that continues to happen to people until the database maintainers pull the hash.

But this isn't a closed system within that authoritarian regime. It leaves bread crumbs of this behavior out for everyone to see. If Hungary is going to arrest people on made up charges, they can do that anyway.

I just think this in a complicated and ineffectual bullet that can only be fired once because it has a fingerprint of the person who fired it. That adds up to make this type of abuse less of a worry.

What I'm concerned about is a system that flags me for a crime based on a database I can't audit based on mechanisms with an entirely too high false positive rate.

Because the database can't be audited by anyone but a select group we have to trust that it only contains actual bad images. I do not trust that such databases don't also contain images that are embarrassing to powerful/connected people. I also do not trust such databases don't contain false positives.

The sort of people that are super zealous about a topic aren't simultaneously super rational and objective about that topic. There's a non-zero probability that those databases contain lewd yet entirely legal images that the submitters just didn't like.

Because of the false positive rate a photo of my dog might trigger an alarm and then my phone sends an automated message to the police. I'm then told by proponents there will be some manual review. I have to then hope that the local DA doesn't have an election coming up and wants to push a "tough on crime" message so charges me with a crime despite a review.

In short these scanning systems require far too much unearned trust. They also present a slippery slope thanks to the incendiary nature of the topic. Today it's CSAM but what undesirable content will the systems be used for tomorrow? Such systems require trust in the stewards of today and tomorrow. Do you want people of the opposite ideology to you in charge of such systems? Do you trust they'll never be abused? Do you trust well meaning people never make mistakes?

I do not trust in any of those things. I'm not worried about myself doing actual bad things, I'm worried that demonstrable false positive rates will ruin my life with the mere accusation of doing something bad.

That would be ridiculous.

But there are penalties for false claims. There is a fine for claiming copyright you don't own, and if you go further and ask for takedowns, you are also liable for damage.

The problem is that these are rarely enforced. Even a $100 fine for a false claim on YouTube would be enough to weed out bots and click farms. And for the most serious cases, have the infringer pay damage and a bigger fine. No need to change the law for that, it just has to be enforced.