Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Perceptual hashing tools for detecting child sexual abuse material

github.com

Post reply on HN
Print view View original

21–30 of 52 posts

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#21

Post by RcouF1uZ4gsC »

faustomorales wrote: post #2

Hi HN! I'm a data scientist at Thorn, a non-profit dedicated to defending children from sexual abuse. We're excited to open source some elements of our perceptual hashing tooling as a Python package. We've tried to make it very flexible both for ourselves and hopefully also for others. Our aim with is to provide tools that (1) help more people eliminate child sexual abuse material from the internet and (2) assist wit…

Was there discussion about the downsides of open sourcing the implementation? With an open source implementation, it becomes easier to test transformations that will change the hash without changing the image in ways a human can’t notice?

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#22

Post by faustomorales »

RcouF1uZ4gsC wrote: post #21
faustomorales wrote: post #2

Hi HN! I'm a data scientist at Thorn, a non-profit dedicated to defending children from sexual abuse. We're excited to open source some elements of our perceptual hashing tooling as a Python package. We've tried to make it very flexible both for ourselves and hopefully also for others. Our aim with is to provide tools that (1) help more people eliminate child sexual abuse material from the internet and (2) assist wit…

Was there discussion about the downsides of open sourcing the implementation? With an open source implementation, it becomes easier to test transformations that will change the hash without changing the image in ways a human can’t notice?

Great question -- all of the hashes we included in the package have been public for years (except for PDQ, which was open-sourced this year). So this package doesn't reveal anything new with respect to the algorithms themselves. What we add is an easy path to using at least one of them for the CSAM hashing / matching use case. Non-public perceptual hashes for this use case exist and, naturally, are not available in the package.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#23

Post by Nasrudith »

httpsterio wrote: post #15

Without reading the link, can this be used to help find child porn? If yes, are there any steps you can or are going to take in order to prevent this?

It only filters images already there - child porn isn't already among the chaff only thing it can find by definition would be false positives. Which could be an interesting vetting exercise in itself - feed in tons of cat pictures until it dings on one as inappropriate.

Technically you can always try cycling through image combinations match the hash/looks like child pornography and nothing could technically prevent it except that it would take a very, very, long time.

Even given adversarial networks I would be very surprised to get anything more than vague figures and I don't even expect the result to look human. In which case it is really generating instead of finding.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#24

Post by tailaiw »

faustomorales wrote: post #2

Hi HN! I'm a data scientist at Thorn, a non-profit dedicated to defending children from sexual abuse. We're excited to open source some elements of our perceptual hashing tooling as a Python package. We've tried to make it very flexible both for ourselves and hopefully also for others. Our aim with is to provide tools that (1) help more people eliminate child sexual abuse material from the internet and (2) assist wit…

Great work faustomorales. Thank you and your colleagues so much for this. Greeting from your old buddies ;)

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#25

Post by [deleted] »

faustomorales wrote: post #7
akersten wrote: post #3

Earlier quoted context omitted.

If I may, I'm curious about your thoughts on few things, in the context of use case #1 (abuse), not #2 (reduplication): * What are the challenges surrounding verification that your system functions properly, given that the test material is illicit? * Can you speak to the reliability of the system in a sensitivity/specificity kind of way? In other words, what are the false positive and false negative rates? * Are you…

Thanks for asking these important questions! * What are the challenges surrounding verification that your system functions properly, given that the test material is illicit? You're right that storing child sexual abuse material (CSAM) is illegal, unless you are the National Center for Missing and Exploited Children (NCMEC) or law enforcement. What is legal is to maintain a hash of known CSAM. NCMEC, Law Enforcement,…

[deleted]

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#26

Post by probablypat »

faustomorales wrote: post #2

Hi HN! I'm a data scientist at Thorn, a non-profit dedicated to defending children from sexual abuse. We're excited to open source some elements of our perceptual hashing tooling as a Python package. We've tried to make it very flexible both for ourselves and hopefully also for others. Our aim with is to provide tools that (1) help more people eliminate child sexual abuse material from the internet and (2) assist wit…

Thank you for what you and your team do. Is there anyway I can help contribute to your organization?

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#27

Post by nieve »

faustomorales wrote: post #7
akersten wrote: post #3

Earlier quoted context omitted.

If I may, I'm curious about your thoughts on few things, in the context of use case #1 (abuse), not #2 (reduplication): * What are the challenges surrounding verification that your system functions properly, given that the test material is illicit? * Can you speak to the reliability of the system in a sensitivity/specificity kind of way? In other words, what are the false positive and false negative rates? * Are you…

Thanks for asking these important questions! * What are the challenges surrounding verification that your system functions properly, given that the test material is illicit? You're right that storing child sexual abuse material (CSAM) is illegal, unless you are the National Center for Missing and Exploited Children (NCMEC) or law enforcement. What is legal is to maintain a hash of known CSAM. NCMEC, Law Enforcement,…

A false positive rate of 1/1000 is hard to assess without actual prevalence stats, but with a decent-sized userbase it seems likely you're still going to get a significant number of false positives. Is it intended that users of your system would have employees manually vet all positives (with legal and mental health concerns) or just submit them without review? I'm coming from having built tools to support a large manual sweep in the 2000s and watching the toll it took on my coworkers.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#29

Post by catalogia »

pbalau wrote: post #13

From what I've read, this is supposed to catch the bad thing after it happened, not to prevent it. Then you say that 1 in 1000 is an acceptable error rate. 1 in 7 billions is not an acceptable error rate!

> "Then you say that 1 in 1000 is an acceptable error rate. 1 in 7 billions is not an acceptable error rate!"

When systems like this return a positive match, that doesn't result in a summary execution. Rather, it prompts a deeper investigation.

Re: Show HN: Perceptual hashing tools for detecting child sexual abuse material

#30

Post by faustomorales »

tailaiw wrote: post #24
faustomorales wrote: post #2

Hi HN! I'm a data scientist at Thorn, a non-profit dedicated to defending children from sexual abuse. We're excited to open source some elements of our perceptual hashing tooling as a Python package. We've tried to make it very flexible both for ourselves and hopefully also for others. Our aim with is to provide tools that (1) help more people eliminate child sexual abuse material from the internet and (2) assist wit…

Great work faustomorales. Thank you and your colleagues so much for this. Greeting from your old buddies ;)

Thanks, old friend! Hope you decide to give the package a peek and maybe lend us some of your Python chops. :)
Post reply on HN