Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tainting the CSAM client-side scanning database

blog.xot.nl

Post reply on HN
Print view View original

71–80 of 276 posts

Re: Tainting the CSAM client-side scanning database

#71

Post by chatmasta »

Chabsff wrote: post #63
londons_explore wrote: post #60

Earlier quoted context omitted.

> I assume the answer to that will be that there is no need to differentiate between them. And honestly, I agree with that I disagree. The point is to reduce actual child abuse. The images are in a way only tangential. If an image is made with an AI with no actual child being abused, then it shouldn't be a crime. In a way, it's better , because it will distract the crowd of people into this sort of stuff from activit…

You may think that intuitively, but actual studies actually indicate the opposite. Usage of CSAM material leads to increased risks of contacting and abusing children. This needs to be balanced against rights to privacy and expression, which I personally think take precedence, but pretending that it can serve as harm reduction is just not correct.

I'm personally highly skeptical of the "offering them an outlet" argument. I'd be less suspicious of the idea if its proponents also suggested limiting it to controlled settings, e.g. during meetings with a professional psychiatrist.

But I'm sorry, I just don't believe anyone holed up in their room with a bunch of fake CSAM is "just using it as an outlet" or "protecting real kids from harm." I mean, it almost sounds like a threat: "If you don't let me look at these pictures of fake kids, I'll hurt real kids." If that's the case then they should be seeing a psychiatrist, at minimum.

Re: Tainting the CSAM client-side scanning database

#72

Post by OfSanguineFire »

rany_ wrote: post #59
OfSanguineFire wrote: post #31

Earlier quoted context omitted.

Signal being restricted to F-Droid would be the end of the app as any kind of mass phenomenon. Sure, like my fellow nerds here on HN, I use F-Droid. But none of the ordinary friends and relatives I managed to convince to install Signal, since it was free from the Play Store with just a few taps, would continue using the app if it were relegated to that repository they have never heard of. Signal’s developers have spo…

Their reasoning for having it Play Store only is that Google does not require app developers to provide their signing keys/sign the APK themselves. Now this is no longer the case as Google changed their policies. NOTE: F-Droid does not require app developers to share their keys, instead they build the application themselves and sign it with their keys -- something Signal is not a fan of. tl;dr their rationale no long…

I have also seen the preference for installing from Play Store explained as being that users are safer if they use default setups and do what everyone else is doing. For users who are not savvy, going the sideloading route can expose them to risk.

Re: Tainting the CSAM client-side scanning database

#73

Post by varispeed »

Ok, but why not go for routine home searches then?

Is it because on the phone, it is "invisible" and cheap?

Does it mean that what stops the government from treating everyone as a potential criminal is cost and inconvenience?

People were raising alarms about this years ago and were branded as conspiracy theorists. I guess the EU works well in their propaganda department painting themselves as do gooders, where in fact they are just wannabe Stalinists.

Re: Tainting the CSAM client-side scanning database

#74

Post by Macha »

nopurpose wrote: post #57
heisenbit wrote: post #7

Anyone who thinks the injection of malicious hashes is an unrealistic scenario should take a look at the games now being played with Youtube‘s content protection system which is leveraged by scammers and blackmailers.

what is happening in Youtube world?

People claiming (via manual reports or via content ID) content that is not theirs to either obtain other people's ad revenue or cause the takedown of content they disagree with

Re: Tainting the CSAM client-side scanning database

#75

Post by jabroni_salad »

nopurpose wrote: post #57
heisenbit wrote: post #7

Anyone who thinks the injection of malicious hashes is an unrealistic scenario should take a look at the games now being played with Youtube‘s content protection system which is leveraged by scammers and blackmailers.

what is happening in Youtube world?

This one happened just a month ago so it might be what the parent comment is referring to. Basically, somebody made a fake company to make bogus copyright claims against someone to hurt their channel. Youtube refuses to deliver counterclaims unless the YTer puts their government name on it (he originally tried to deliver it via an attorney).

Additionally, the other party is actively trying to compromise the YTer's other accounts and identity to damage him, so any new data point given to this person represents risk.

I don't really care for this YTer (they made a video essay about someone who doesn't really want to be in the media circus anymore, could just MYOB) but the methods documented in this video can be used to doxx any user that uploads a video. All you gotta do is convince youtube that you own some piece of common non-royalty media that a youtuber uses to gain leverage on them. A lot of this kind of media is old with unfindable owners, so even if YT does want to spend the time to validate, there's nowhere to go with it.

https://www.youtube.com/watch?v=hixwIOd_C44

Re: Tainting the CSAM client-side scanning database

#76

Post by yieldcrv »

OfSanguineFire wrote: post #20
miohtama wrote: post #2

Because client-side scanning is not going to work, and no one wants to government issued black box binary to send their conversations and photos to unnamed police person randomly, the non-compliance is the only way. People just start to use chat programs in the EU that do not comply. This would be Signal, Telegram, others. The EU can fine and fight with Meta/WhatsApp, Apple, others, but that’s about it. The EU bureau…

Signal advises its users to install the app from the Play Store. While you can still get the .apk from Signal’s website, the developers warn against that. The EU can definitely exert pressure against what is hosted on the Play Store. Telegram famously lacks end-to-end encryption (unless you intentionally use its private-chat feature, which few people do) and shouldn’t be mentioned in the same context as Signal.

Their point isn’t about Telegram end to end encryption or lack thereof, its that they don’t comply

Nothing gets taken down from telegram and they ignore court orders they just go to null

Re: Tainting the CSAM client-side scanning database

#77

Post by Hizonner »

usrnm wrote: post #69
chatmasta wrote: post #47

Earlier quoted context omitted.

See my comment to a sibling reply. Basically I don't want to make victims prove their victimization was real, and I don't want to give criminals with real victims an opportunity to argue they "thought it was AI generated."

And I absolutely want both parties to have to prove crime/innocence and have an opportunity to argue. The current situation, where anything involving CSAM is so toxic that lives are ruined without trial is not healthy and isn't good for anybody

Point of order: victims are not "parties" in criminal cases, not in remotely modern legal systems. The parties are the accused and the state.

For the same reason, victims don't get to pardon crimes committed against them.

... which is the way it should be, because criminal punishment should not be seen as a form of revenge, but as a deterrent.

Re: Tainting the CSAM client-side scanning database

#78

Post by viridian »

nopurpose wrote: post #57
heisenbit wrote: post #7

Anyone who thinks the injection of malicious hashes is an unrealistic scenario should take a look at the games now being played with Youtube‘s content protection system which is leveraged by scammers and blackmailers.

what is happening in Youtube world?

Not sure if its what the poster above is talking about, but there's definitely a hash collision type attack that's common on Youtube with regards to classical music. The attacker in question uploads very standard renditions of thousands of pieces of classical music, and claims copyright on them. Content ID then flags any video using one of these pieces as potentially violating the rights of the rightholder.

The attacker then claims the video, and claims revenue on the video rather than taking it down. There's an appeal process which seems to work about 0% of the time, and after that the only recourse the uploader has is to submit a copyright counter notification, which becomes the start of a potential lawsuit.

The attackers in question will then typically relent if challenged with a copyright counter claim, but most videos don't get counter claimed once this happened.

This pattern of behavior seems so common that it has led to two interesting patterns of behavior that I've seen.

1) Any video that uses popular public domain music will be claimed not once, but dozens or hundreds of times, because there are so many channels operating this way now, and

2) If you post a video and immediately get hit with a huge wave of claims, rather than going through Youtube's process, most uploaders just delete the video, remove whatever audio caused the problem, and reupload.

Re: Tainting the CSAM client-side scanning database

#79

Post by mcpackieh »

regularjack wrote: post #65
Hizonner wrote: post #43

It says: > This shows that the database can be tainted with non-CSAM material by an entity that can submit entries to it. Actually, it can easily be tainted by anybody . Take your massaged hash-colliding image, which remember is still visually child porn , and post it on some pedos-R-us forum. The people who maintain the database actively troll those forums. They'll see the image and add the hash to the database for…

You'd be publishing child porn, which I think is not the wisest thing to be doing.

A spy agency or secret police doing this to use the client-side scanning system as a surveillance/censorship system for political media would not care about that.

Re: Tainting the CSAM client-side scanning database

#80

Post by kypro »

This is basically what I suggested we do a few years ago if Apple added client-side content scanners.

Seems all need to here is obtain a finger print of a CSAM image then you can reverse engineer a non CSAM image to match that finger print. Distribute this image wide enough and you effectively render these algorithms useless.

Which is a shame because they could be used for good, but it seems kinda obvious this will expand out to scanning for terrorist materials. And again, while I might be in favour of this in theory I am frequently shocked by what fits this definition. In the EU if you promote the wrong ideas or say something "hateful" you can be arrested and sentenced 10+ years in prison.

I think what concerns me is that currently you really only put yourself at risk if you make your wrong think public, e.g tweet about your dislike of Islam on Twitter. These client-side scanners would now enable the authorities to not only go after those who say "racist" things online, but also those who view / save "racist" memes. If I were to guess at a motive, this would be it.

Legal note: I obviously don't agree with hate of any kind, but I do believe in the right to express views which may be considered hateful.

Edit: Fine with the downvotes – but curious what people are disagreeing with? Would you mind leaving a quick comment if you choose to downvote?

Post reply on HN