Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tainting the CSAM client-side scanning database

blog.xot.nl

Post reply on HN
Print view View original

51–60 of 276 posts

Re: Tainting the CSAM client-side scanning database

#51

Post by skyyler »

OfSanguineFire wrote: post #20
miohtama wrote: post #2

Because client-side scanning is not going to work, and no one wants to government issued black box binary to send their conversations and photos to unnamed police person randomly, the non-compliance is the only way. People just start to use chat programs in the EU that do not comply. This would be Signal, Telegram, others. The EU can fine and fight with Meta/WhatsApp, Apple, others, but that’s about it. The EU bureau…

Signal advises its users to install the app from the Play Store. While you can still get the .apk from Signal’s website, the developers warn against that. The EU can definitely exert pressure against what is hosted on the Play Store. Telegram famously lacks end-to-end encryption (unless you intentionally use its private-chat feature, which few people do) and shouldn’t be mentioned in the same context as Signal.

Telegram is a social media site pretending to be a messaging app.

The secret chats were the biggest feature when it started, but now it's an after-thought.

Re: Tainting the CSAM client-side scanning database

#53

Post by sneak »

chatmasta wrote: post #32
qingcharles wrote: post #24

How do these databases differentiate between AI generated CSAM and CSAM of real victims? (Since many jurisdictions only criminalize real CP) I know that 99% of people cannot tell an AI image from a real photo since that "Last giant irish greyhound 1902" photo has been going around on social media for weeks, and it is, to me, unbelievably obvious AI.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> Possession of CSAM should be illegal regardless of whether it's "real" or not.

From a purely ethical standpoint: why? What is the purpose of punishing someone who has harmed no one?

No victim means no crime.

Re: Tainting the CSAM client-side scanning database

#54

Post by kube-system »

kzrdude wrote: post #39
CuriousCosmic wrote: post #29

Earlier quoted context omitted.

Notifications generally go through Google Firebase so by not running private messages through firebase they avoid potential leaks. At least that's my guess

That sounds pretty bad, a cloud service needs to see all notifications?

Server side push messaging uses cloud infrastructure on both Android and iOS devices.

Some applications choose to make these notifications more 'secure' by not sending any content through them, just the fact that you have a new notification, but that makes the UX a little less friendly.

https://en.wikipedia.org/wiki/Apple_Push_Notification_servic...

https://en.wikipedia.org/wiki/Firebase_Cloud_Messaging

Re: Tainting the CSAM client-side scanning database

#55

Post by thisisabore »

OfSanguineFire wrote: post #31
CuriousCosmic wrote: post #28

Earlier quoted context omitted.

Molly is a just as good (if not better) fork of Signal that is distributed on FDroid. It's the same network and same chats/contacts, just a different front-end. And Signal avoids FDroid because they don't want someone else signing packages but they can always provide an FDroid repository like many others do and sign everything themselves. If push comes to shove they'll be fine and pressure to black box signal in the…

Signal being restricted to F-Droid would be the end of the app as any kind of mass phenomenon. Sure, like my fellow nerds here on HN, I use F-Droid. But none of the ordinary friends and relatives I managed to convince to install Signal, since it was free from the Play Store with just a few taps, would continue using the app if it were relegated to that repository they have never heard of. Signal’s developers have spo…

How is providing a F-Droid repo the same as being restricted to only being available via F-Droid?

Re: Tainting the CSAM client-side scanning database

#56

Post by boerseth »

npteljes wrote: post #30
bastawhiz wrote: post #5

The issue described here, to my understanding, is that you find or create csam and then manipulate it so that its fingerprint collides with another image that you want to be flagged as csam. You then submit the manipulated version of the found or generated image to the authority. First, at what point does the authority go "uh... Where did you get this from?" Practically speaking, the people doing this would have to b…

Another version of the attack is to manipulate an innocent image in a way that's flagged by detection systems, and then spread that image somehow. If any of the responses are automatic, then it's a lot of false positives in the system, a lot of contact from police, disabling of the host, things like this.

There's also not really any need to trick the system or manipulate other files to generate collisions. Spreading one of the original law-breaking files to unwilling targets would work as a trigger just the same.

Re: Tainting the CSAM client-side scanning database

#57

Post by nopurpose »

heisenbit wrote: post #7

Anyone who thinks the injection of malicious hashes is an unrealistic scenario should take a look at the games now being played with Youtube‘s content protection system which is leveraged by scammers and blackmailers.

what is happening in Youtube world?

Re: Tainting the CSAM client-side scanning database

#58

Post by bitvoid »

thisisabore wrote: post #55
OfSanguineFire wrote: post #31

Earlier quoted context omitted.

Signal being restricted to F-Droid would be the end of the app as any kind of mass phenomenon. Sure, like my fellow nerds here on HN, I use F-Droid. But none of the ordinary friends and relatives I managed to convince to install Signal, since it was free from the Play Store with just a few taps, would continue using the app if it were relegated to that repository they have never heard of. Signal’s developers have spo…

How is providing a F-Droid repo the same as being restricted to only being available via F-Droid?

It was in reply to:

> If push comes to shove they'll be fine and pressure to black box signal in the EU is unlikely to hold up if they can just move users to another app store.

Re: Tainting the CSAM client-side scanning database

#59

Post by rany_ »

OfSanguineFire wrote: post #31
CuriousCosmic wrote: post #28

Earlier quoted context omitted.

Molly is a just as good (if not better) fork of Signal that is distributed on FDroid. It's the same network and same chats/contacts, just a different front-end. And Signal avoids FDroid because they don't want someone else signing packages but they can always provide an FDroid repository like many others do and sign everything themselves. If push comes to shove they'll be fine and pressure to black box signal in the…

Signal being restricted to F-Droid would be the end of the app as any kind of mass phenomenon. Sure, like my fellow nerds here on HN, I use F-Droid. But none of the ordinary friends and relatives I managed to convince to install Signal, since it was free from the Play Store with just a few taps, would continue using the app if it were relegated to that repository they have never heard of. Signal’s developers have spo…

Their reasoning for having it Play Store only is that Google does not require app developers to provide their signing keys/sign the APK themselves. Now this is no longer the case as Google changed their policies.

NOTE: F-Droid does not require app developers to share their keys, instead they build the application themselves and sign it with their keys -- something Signal is not a fan of.

tl;dr their rationale no longer applies.

Re: Tainting the CSAM client-side scanning database

#60

Post by londons_explore »

chatmasta wrote: post #32
qingcharles wrote: post #24

How do these databases differentiate between AI generated CSAM and CSAM of real victims? (Since many jurisdictions only criminalize real CP) I know that 99% of people cannot tell an AI image from a real photo since that "Last giant irish greyhound 1902" photo has been going around on social media for weeks, and it is, to me, unbelievably obvious AI.

I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that argument. Possession of CSAM should be illegal regardless of whether it's "real" or not. But the proposed scanning system is the wrong solution, regardless of any "real or AI" ambiguity, because it's possible to generate false positives with nonsense images that aren't even close to the expected CS…

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that

I disagree. The point is to reduce actual child abuse. The images are in a way only tangential. If an image is made with an AI with no actual child being abused, then it shouldn't be a crime.

In a way, it's better, because it will distract the crowd of people into this sort of stuff from activities that harm real people.

Post reply on HN