Edit: comment deleted
Tainting the CSAM client-side scanning database
41–50 of 276 posts
Re: Tainting the CSAM client-side scanning database
#42How do these databases differentiate between AI generated CSAM and CSAM of real victims? (Since many jurisdictions only criminalize real CP) I know that 99% of people cannot tell an AI image from a real photo since that "Last giant irish greyhound 1902" photo has been going around on social media for weeks, and it is, to me, unbelievably obvious AI.
I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…
> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument.
Why do you believe that?
Re: Tainting the CSAM client-side scanning database
#43It says:
> This shows that the database can be tainted with non-CSAM material by an entity that can submit entries to it.
Actually, it can easily be tainted by anybody. Take your massaged hash-colliding image, which remember is still visually child porn, and post it on some pedos-R-us forum. The people who maintain the database actively troll those forums. They'll see the image and add the hash to the database for you.
Re: Tainting the CSAM client-side scanning database
#44Earlier quoted context omitted.
I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…
Seems like you want to bring all the success of the war on drugs to the war on generated images.
No, I don't support any automated scanning system or really any sort of "going out of our way" to find new criminals.
The reason I think it's a bad idea to differentiate between real or AI is similar to the arguments against "means testing" for distributing benefits. You don't want to put real victims in a situation where they're deprived of justice because they can't prove that their victimization was "real." Imagine a real CSAM criminal claiming a defense that they "thought it was AI generated." Do you want to give them that out?
If protecting those victims comes at a cost of punishing criminals possessing AI-generated CSAM with sentences equally as harsh as those for "real" CSAM, then it's a worthwhile cost to pay. They are still criminals, and they are definitely not innocent (unless they're being framed, but that's a risk with both real and AI images).
Re: Tainting the CSAM client-side scanning database
#45Any computational method that relies on a function that converts m bits (an image) to n bits (a fingerprint) where m > n will always be vulnerable to such an attack. The smaller n is compared to m, the easier it is to counterfeit something with that signature. It is not new knowledge. The only way to be certain, unfortunately, is for a human to look at it. When the allegation is as serious as CSAM, I would rather be…
> Any computational method that relies on a function that converts m bits (an image) to n bits (a fingerprint) where m > n will always be vulnerable to such an attack. The "vulnerable" depends on your definition: SHA-2 and SHA-3 are both still quite safe against preimage attacks, and even second preimage attacks require significant work to pull off for SHA-2, and I am unaware of any meaningful second preimage attack…
But then bypassing the filter would just require to change a few bits here and there.
Re: Tainting the CSAM client-side scanning database
#46Earlier quoted context omitted.
Notifications generally go through Google Firebase so by not running private messages through firebase they avoid potential leaks. At least that's my guess
That sounds pretty bad, a cloud service needs to see all notifications?
A good question. I am not sure if notifications on iOS, Android, Google Chrome or Firefox are end-to-end encrypted.
Re: Tainting the CSAM client-side scanning database
#47Earlier quoted context omitted.
I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…
> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Why do you believe that?
See my comment to a sibling reply. Basically I don't want to make victims prove their victimization was real, and I don't want to give criminals with real victims an opportunity to argue they "thought it was AI generated."
Re: Tainting the CSAM client-side scanning database
#48The issue described here, to my understanding, is that you find or create csam and then manipulate it so that its fingerprint collides with another image that you want to be flagged as csam. You then submit the manipulated version of the found or generated image to the authority. First, at what point does the authority go "uh... Where did you get this from?" Practically speaking, the people doing this would have to b…
Abusers who have actual CSAM could intentionally publish the fingerprints to sabotage the scanning scheme. If the illegal fingerprints become known, it will be possible to generate false positives and overwhelm verification/enforcement with bogus matches.
Re: Tainting the CSAM client-side scanning database
#49Because client-side scanning is not going to work, and no one wants to government issued black box binary to send their conversations and photos to unnamed police person randomly, the non-compliance is the only way. People just start to use chat programs in the EU that do not comply. This would be Signal, Telegram, others. The EU can fine and fight with Meta/WhatsApp, Apple, others, but that’s about it. The EU bureau…
Governments will mandate by law that the OS do client side scanning by peeping inside the app directories. Apple, Google, Microsoft will comply and realistically, which phones and computers are we going to use if we don't buy from them? Desktop Linux might become illegal.
Yes, the EU is already working to make this illegal
https://mullvad.net/en/blog/2023/2/1/eu-chat-control-law-wil...
The best bet is Android fork based on Android open source distribution, without Google.
Re: Tainting the CSAM client-side scanning database
#50Earlier quoted context omitted.
If you enable private chat, you don't get notifications for new messages, that way the feature is self defeating, it seems to me at least. (Something could also have gone wrong?)
Notifications generally go through Google Firebase so by not running private messages through firebase they avoid potential leaks. At least that's my guess
A common way around this is to simply send a notification through Firebase/other push notification mechanism that tells the app there is a new message.
The app can then retrieve the message and decrypt it. Most of them will then update the notification shown to the user with the full message content.