Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is the best way to programatically detect porn images? (2009)

stackoverflow.com

Post reply on HN
Print view View original

61–70 of 110 posts

Re: What is the best way to programatically detect porn images? (2009)

#61

Post by rfergie »

ma2rten wrote: post #50
mixmax wrote: post #4

detecting all porn seems to be an almost impossible problem. Many kinds of advanced porn (BDSM, etc.) don't have much skin - often the actors are in latex, tied up, or whatever. It's obviously porn when you see it, but detecting it seems incredibly hard. Detecting smurf-porn(1) (yes that's a thing...) is even harder since all the actors are blue. http://pinporngifs.blogspot.dk/2012/09/smurfs-porn.html?zx=7... - obvio…

It is possible high accuracy if you use machine learning and a sufficiently large training set. That said said even humans sometimes don't agree is something is porn or not.

Just saying "machine learning" is not very useful here. What machine learning techniques work well in this case and what are the major pitfalls?

Then you can convince me that a "sufficiently large training set" exists and is smaller than "all the images on the internet".

Re: What is the best way to programatically detect porn images? (2009)

#62

Post by mhurron »

selmnoo wrote: post #9
phorese wrote: post #8

Earlier quoted context omitted.

So i can literally get paid for looking at porn. Huh, who knew... :)

At the cost of your mental health, sure. I recall reading an article about the human workers who had this job at Google... they had crap benefits, crap pay, and no mental healthcare. As a result a lot of the people in the field had depression and other mental issues.

Isn't Obamacare supposed to cover mental health care? Seems like a solved problem.

Re: What is the best way to programatically detect porn images? (2009)

#63

Post by clebio »

ma2rten wrote: post #47

I did this for my bachelor thesis for a company that shall remain unnamed. I am pretty confident that my approach works better than any of the answer posted on stackoverflow. I used the so called Bag of Visual Words approach. At that time the state of the art in image recognition (now it's neural networks). You can read about on Wikipeida. The only main change from the standard approach (SHIFT + k-means + histograms…

This sounds quite interesting. Is there any of the research of code base that you can share? Or otherwise any references about the standard approach which you would recommend?

Re: What is the best way to programatically detect porn images? (2009)

#64

Post by XorNot »

eksith wrote: post #11
RossM wrote: post #7

Earlier quoted context omitted.

I seem to remember reading an article about people doing this at Google(?) in a pretty poor state. Edit: I think it was this one: http://www.buzzfeed.com/reyhan/tech-confessional-the-googler...

Yep! That's the one. Thanks. A pretty unpleasant job no matter what angle you look at it. I remember this bit the most : "Google covered one session with a government-appointed therapist — and encouraged me to go out and get my own therapy after I left." This is the down side to 'abstracting away' the dirty end of filtering. I'm looking forward to a day when this can be properly automated, but, considering the ever-c…

Developing a strong AI which can do this without going insane itself is going to be the robo-pysche challenge of the future.

Re: What is the best way to programatically detect porn images? (2009)

#65

Post by XorNot »

hugofirth wrote: post #35
VLM wrote: post #21

Earlier quoted context omitted.

"score above a threshold number of positive results" How about social scoring? A normal (or even a weirdo) teenage boy would spend less than a second examining my ugly old profile pix, but after ten or so of your known teen male users are detected to spend 5 minutes at a time, a couple times a day, closely studying a suspected profile pix, I think you can safely conclude that pix is not a pix of me and then flag / ce…

The graph theorist in me is rubbing my hands in glee at the thought of seeing if you could extrapolate out that approach to catch a broader range of offensive imagery through relationships and usage patterns.

Key problem: profile pictures are everywhere on a website by definition. People look at them for over an hour at a time depending how your site is laid out.

If you wanted to spot shock imagery it's easier - study navigation aways or rapid scrolling.

Re: What is the best way to programatically detect porn images? (2009)

#66

Post by viraptor »

pa5tabear wrote: post #48
_mulder_ wrote: post #25

Here's an idea... Develop a bot to trawl NSFW sites and hash each image (combined with the 'skin detecting' algorithms detailed previously). Then compare the user uploaded image hash with those in the NSFW database. This technique relies on the assumption that NSFW images that are spammed onto social media sites will use images that already exist on NSFW sites (or are very similar to). Then it simply becomes a case o…

How do you program that sort of thing? Do you have to tell it what shapes/colors to look for? Or do a combination of overall image similar combined with localized image similarity and portion by portion image comparison?

Maybe recognising the furniture in the background would work too ;) I remember there was a website/catalog of IKEA furniture somewhere made using NSFW photos.

Re: What is the best way to programatically detect porn images? (2009)

#67

Post by ma2rten »

clebio wrote: post #63
ma2rten wrote: post #47

I did this for my bachelor thesis for a company that shall remain unnamed. I am pretty confident that my approach works better than any of the answer posted on stackoverflow. I used the so called Bag of Visual Words approach. At that time the state of the art in image recognition (now it's neural networks). You can read about on Wikipeida. The only main change from the standard approach (SHIFT + k-means + histograms…

This sounds quite interesting. Is there any of the research of code base that you can share? Or otherwise any references about the standard approach which you would recommend?

This software is free for non-commerical use: http://koen.me/research/colordescriptors/

You can find other implementation of varying quality if you Google for Bag of Visual Words. For the final classification, I would recommend scikit-learn.

Re: What is the best way to programatically detect porn images? (2009)

#68

Post by differentView »

sharkweek wrote: post #45
selmnoo wrote: post #9

Earlier quoted context omitted.

At the cost of your mental health, sure. I recall reading an article about the human workers who had this job at Google... they had crap benefits, crap pay, and no mental healthcare. As a result a lot of the people in the field had depression and other mental issues.

Friend was a contractor on the YouTube filter team -- they lasted about 9 months before it became too much and had to leave. There was absolutely no reward in showing up to work, and some of the things they saw have likely scarred their memories forever

Challenge accepted!

Re: What is the best way to programatically detect porn images? (2009)

#69

Post by ma2rten »

rfergie wrote: post #61
ma2rten wrote: post #50

Earlier quoted context omitted.

It is possible high accuracy if you use machine learning and a sufficiently large training set. That said said even humans sometimes don't agree is something is porn or not.

Just saying "machine learning" is not very useful here. What machine learning techniques work well in this case and what are the major pitfalls? Then you can convince me that a "sufficiently large training set" exists and is smaller than "all the images on the internet".

See my other post on this page.

I would argue that this less difficult than you average image classification problem. Just have a look at what kind of challenges image classification can tackle, picking the correct class out of 1000s of classes. Porn is normally well-lit, the subject is at the center of the image, etc.

The main difficulty is to define what is porn and what is not ... It's easy see the difference between porn and pictures of bicycles. But how about porn and artistic nudity? You see it's actually a scale, but you are trying to make a binary decision.

Another problem (at least with the method I explained below) is that portraits sometimes get misclassified. Maybe it could help to integrate face detection. I'd suspect that more recent models would not have this problem (e.g. ones that take not only local features into account). Other times it makes mistakes, where you think "why on earth would think this is porn". Again combing different method should help to eliminate those outliers.

Outliers are also a problem, e.g. black and white pictures. Again an ensemble of different models (e.g. one color independent one) might help. Niches are not really a big problem. BSDM porn is as far as I have seen the only niche of porn that really different visually.

Re: What is the best way to programatically detect porn images? (2009)

#70

Post by _mulder_ »

pa5tabear wrote: post #48
_mulder_ wrote: post #25

Here's an idea... Develop a bot to trawl NSFW sites and hash each image (combined with the 'skin detecting' algorithms detailed previously). Then compare the user uploaded image hash with those in the NSFW database. This technique relies on the assumption that NSFW images that are spammed onto social media sites will use images that already exist on NSFW sites (or are very similar to). Then it simply becomes a case o…

How do you program that sort of thing? Do you have to tell it what shapes/colors to look for? Or do a combination of overall image similar combined with localized image similarity and portion by portion image comparison?

Well Image Hashing is distinct from normal MD5 hashing as the hash does consider similarity of colour, etc. so it's not purely binary. A Google search produced a library called pHash.org that might do something similar.
Post reply on HN