Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Perceptual hashing tools for detecting child sexual abuse material

github.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#41

Post by amitport »

I don't get it. Sound nice but what exactly does Thorn do? what is this spotlight services they're providing? Seriously I've just spend time on their site but I don't get it beyond "using data". How they use it?

Lets say I have friends in a local DA office and I want to sell them this idea of using data they have. They just know that giving access to their data is a big legal headache. what exactly to I tell them to get them on-board with this? (exactly how the data / what data is handled?)

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#42

Post by mayniac »

nieve wrote: post #27
faustomorales wrote: post #7

Earlier quoted context omitted.

Thanks for asking these important questions! * What are the challenges surrounding verification that your system functions properly, given that the test material is illicit? You're right that storing child sexual abuse material (CSAM) is illegal, unless you are the National Center for Missing and Exploited Children (NCMEC) or law enforcement. What is legal is to maintain a hash of known CSAM. NCMEC, Law Enforcement,…

A false positive rate of 1/1000 is hard to assess without actual prevalence stats, but with a decent-sized userbase it seems likely you're still going to get a significant number of false positives. Is it intended that users of your system would have employees manually vet all positives (with legal and mental health concerns) or just submit them without review? I'm coming from having built tools to support a large ma…

>I'm coming from having built tools to support a large manual sweep in the 2000s

Any chance you're one of the devs behind C4All?

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#43

Post by GistNoesis »

How do you defend against semi transparent overlaying ?

Take an image you want to censor, overlay with a very transparent offensive image. Also publish the original image (or a second image with a different transparency value), and an in-browser extension can reconstruct the offensive image.

Your hash database will be flooded with wrong values.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#44

Post by natch »

What are some of the ways you can improve on the n-squared nature of these distance calculations at scale?

In your use case you have a limited set of hashes you are comparing against, but I'm thinking of the more difficult use case of comparing every image against every other image (the context is ML training and weeding out overly similar examples so that overfitting doesn't happen, not related to CSAM). This quickly becomes untenable if you get, say, millions or billions of images.

"Don't do that" is one answer but are there any other ways you have come across to mitigate that?

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#45

Post by concordDance »

faustomorales wrote: post #2

Hi HN! I'm a data scientist at Thorn, a non-profit dedicated to defending children from sexual abuse. We're excited to open source some elements of our perceptual hashing tooling as a Python package. We've tried to make it very flexible both for ourselves and hopefully also for others. Our aim with is to provide tools that (1) help more people eliminate child sexual abuse material from the internet and (2) assist wit…

> help more people eliminate child sexual abuse material from the internet

Is that actually a good thing?

Less material seems like it would mean more motivation to produce more, which is the very thing we want to avoid.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#46

Post by lonelappde »

GistNoesis wrote: post #43

How do you defend against semi transparent overlaying ? Take an image you want to censor, overlay with a very transparent offensive image. Also publish the original image (or a second image with a different transparency value), and an in-browser extension can reconstruct the offensive image. Your hash database will be flooded with wrong values.

That seems like symmetric key encryption with extra steps.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#47

Post by GistNoesis »

lonelappde wrote: post #46
GistNoesis wrote: post #43

How do you defend against semi transparent overlaying ? Take an image you want to censor, overlay with a very transparent offensive image. Also publish the original image (or a second image with a different transparency value), and an in-browser extension can reconstruct the offensive image. Your hash database will be flooded with wrong values.

That seems like symmetric key encryption with extra steps.

This is a general attack against perceptual hashing. It tries to achieve multiple objectives. Using stenography you could add any data in the image, like you would with encryption as you suggest. But here it is deeper.

By superposing a very transparent bad image with the good one, the perceptual hash won't be affected but the image would be still labelled as offensive by the poor guy labeling the data because the transparent offensive image is still visible. This mean that the good data will be marked as potentially offensive.

This allows anyone to target any individual user, or website. You take some of their published content, overlay offensive image and republish. The image will be reported as offensive, yet have a similar hash (i.e. a collision) to the good image which automated systems will pick-up as the original image being offensive and blacklisting its user or tanking down the website on search engine results.

Because this is sensitive data, by law it should be deleted as soon as it is detected which means the proofs get deleted automatically.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#48

Post by nieve »

mayniac wrote: post #42
nieve wrote: post #27

Earlier quoted context omitted.

A false positive rate of 1/1000 is hard to assess without actual prevalence stats, but with a decent-sized userbase it seems likely you're still going to get a significant number of false positives. Is it intended that users of your system would have employees manually vet all positives (with legal and mental health concerns) or just submit them without review? I'm coming from having built tools to support a large ma…

>I'm coming from having built tools to support a large manual sweep in the 2000s Any chance you're one of the devs behind C4All?

No, thank god. I was hired as technical lead for the team put together when my new employers inherited a medium-sized low profile social network primarily popular in SA and SEA from their parent company. We got a report on something in the supposedly mostly unused photo sharing feature and discovered there were no moderation tools at all, so the first thing I had to do was build something to even verify the reports. At that point it turned out to be necessary to do a sweep and legal thought it should be done by hand, so my coworkers spent days going through it. It would have been grueling for them even if they hadn't run into the awful material.

(I escaped having to help because my sole minion was the kind of guy who decompresses a several gb bzipped log file as root in _/root_ and wanders away while it's running.)

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#49

Post by faustomorales »

natch wrote: post #44

What are some of the ways you can improve on the n-squared nature of these distance calculations at scale? In your use case you have a limited set of hashes you are comparing against, but I'm thinking of the more difficult use case of comparing every image against every other image (the context is ML training and weeding out overly similar examples so that overfitting doesn't happen, not related to CSAM). This quickl…

Hi there! Our example on deduplication [1] takes the case of deduplicating of the Caltech256 [2] dataset for the purpose I think you're describing, which I interpreted as avoiding duplicates in a dataset so that you don't have images that end up in both your training and test sets. Even though the dataset contains >30K images, you can still do this in memory (and find a handful of duplicates!) because each category is relatively small and you probably don't need to deduplicate images of trains with images of dogs.

On the same page, we mention two tools (specifically, FAISS [3] and Annoy [4]) to help with doing approximate search for scales where computing the distance matrix is impractical.

[1] https://perception.thorn.engineering/en/latest/examples/dedu...

[2] https://authors.library.caltech.edu/7694/

[3] https://github.com/facebookresearch/faiss

[4] https://github.com/spotify/annoy

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#50

Post by natch »

faustomorales wrote: post #49
natch wrote: post #44

What are some of the ways you can improve on the n-squared nature of these distance calculations at scale? In your use case you have a limited set of hashes you are comparing against, but I'm thinking of the more difficult use case of comparing every image against every other image (the context is ML training and weeding out overly similar examples so that overfitting doesn't happen, not related to CSAM). This quickl…

Hi there! Our example on deduplication [1] takes the case of deduplicating of the Caltech256 [2] dataset for the purpose I think you're describing, which I interpreted as avoiding duplicates in a dataset so that you don't have images that end up in both your training and test sets. Even though the dataset contains >30K images, you can still do this in memory (and find a handful of duplicates!) because each category i…

Yes looked at annoy previously and it is really impressive. Will probably revisit that as part of the solution here. You did interpret my use case exactly right. Thanks for the reply and especially for the links!
Post reply on HN