Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

dailydot.com

Post reply on HN
Print view View original

51–60 of 155 posts

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#51

Post by Kye »

JulianMorrison wrote: post #33

My feeling is that this may be the breakthrough moment for Mastodon.

I don't think there will be one single moment. Every stumble from a big silo leads to a huge burst of growth and a return to a slightly higher norm. It's more likely this slow transfer of users will continue until the narrative is more "Mastodon finishes off [x]" for each one and less "users are fleeing [x] for Mastodon".

Tumblr/Twitter/Facebook/etc are going to die the same way malls do. One decade they're a big deal, the next people's slow shift in preferences make them irrelevant. At least they won't leave rotting husks this time.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#52

Post by BlackLotus89 »

benmmurphy wrote: post #10

I'm not sure how this proves the algorithm is doing a poor job. Any NSFW classifier is going to have a non-trivial error rate. Tumblr hosts LOTS of images. So if you have a non-trivial error rate and multiply it by LOTS of images you are going to get LOTS of images being misclassified incorrectly. It's not surprising someone was able to write a story about a bunch of images being misclassified. This doesn't seem to t…

Since we don't have any real stats we can't classify how well the algorithm is doing, one important thought though is how the algorithm is designed.

If you got a good filtering method you can either optimize for a false positive rate or a false negative rate. Let's compare it to a simple spam filter (because if you look at it objectively that's all it is) would you rather have spam in your inbox or loose mails to your spam filter? Spam filter (normally) prioritize a high false negative so you have less spam, but don't loose content. Having a filter that obviously has a high false positive rate is just bad for the content creators (on which tumblr depends)

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#53

Post by radiowave »

xfitm3 wrote: post #41
loudmax wrote: post #38

I read a lot of posts on Reddit bemoaning Tumblr's decision to stop carrying adult content. A lot of people there and here feel that this decision spells the end of the company. That may be the case, but I never understood how Tumblr was profiting from hosting all that adult content to begin with. I presume they're selling ads, in which case disassociating themselves from anything that could be considered pornographi…

Static content is extremely cheap to host. I would compare it to S3 costs.

Not sure how Tumblr is doing things these days, but for a long time it literally was S3 costs, as that's where all the media was being served from.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#54

Post by opencl »

benmmurphy wrote: post #10

I'm not sure how this proves the algorithm is doing a poor job. Any NSFW classifier is going to have a non-trivial error rate. Tumblr hosts LOTS of images. So if you have a non-trivial error rate and multiply it by LOTS of images you are going to get LOTS of images being misclassified incorrectly. It's not surprising someone was able to write a story about a bunch of images being misclassified. This doesn't seem to t…

Maybe it's a good classifier by some measure but the error rate is clearly way too high for indiscriminately deleting everything it detects to be a good idea, and that is exactly what Tumblr plans on doing in 2 weeks. Randomly deleting posts that in no way violate the rules is going to piss a lot of people off even if "only" 5% of posts are affected.

This is kind of an inherent problem with automated moderation tools. And not new to Tumblr either, there is some sort of mysterious spam filter applied to the ask system that results in a lot of legitimate messages not being sent that people have complained about for ages.

And maybe my dash is not a representative sample but there really does seem to be a huge number of misclassified posts.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#55

Post by jamesholden »

AJ007 wrote: post #30
keypusher wrote: post #5

Oath is desperate to put more ads on Tumblr, because they massively overpaid for it ($1 billion) and have been throwing money at it ever since (100s of millions). I guess because of the way the content is displayed they can't just segregate nsfw stuff into a demonetized stream, but if they think they can turn Tumblr into a mainstream site that shows family-centric ads, they clearly don't understand what they bought.

This is clearly Tumblr's Digg moment. They are going to have a 100% write down on this one. It is also worth adding Twitter has built a reasonable ad business and they are full of just as much porn as Tumblr. Google actually shows ads for porn and other adult services on their platform. If any of the higher ups at Verizon are reading this: - Fire everyone involved in the NSFW filtering. Clearly this wasn't even teste…

- Fire everyone who thought NSFW material had to be purged from your platform.

I mean, can't they see that a lot of traffic flows through tumblr via those interests? Do they not know how much of a loss of usage that this will cause? I legitimately have zero other reason to use tumblr.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#56

Post by newen »

dpwm wrote: post #48

It would be quite humorous, and also quite telling of the experience of those implementing this, if it turned out to be classic class imbalance during training. If 90% of the training dataset is nsfw, and you train a classifier naively on that dataset, and your test and validation data is 90% nsfw, then 90% accuracy is very easy.

Oh god...I remember my first project I just ran an ML algorithm against the dataset, got high accuracy, boasted to colleagues, and then explored the dataset only to realize it was highly unbalanced. So embarrassing.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#57

Post by tech2 »

whatshisface wrote: post #42
loudmax wrote: post #38

I read a lot of posts on Reddit bemoaning Tumblr's decision to stop carrying adult content. A lot of people there and here feel that this decision spells the end of the company. That may be the case, but I never understood how Tumblr was profiting from hosting all that adult content to begin with. I presume they're selling ads, in which case disassociating themselves from anything that could be considered pornographi…

AT&T carries way more of it than any individual website serves, but nobody complains because they're generally thought of as a pipe that is at worst absent. If a social platform can get the coveted "at worst absent," perception in place, nobody will blame them, and by extension nobody will blame the advertisers. That's why it was against the rules (so to speak) for the media to hop on YouTube for the "Nazi" videos: t…

"Amusingly" this is almost exactly what EU Article 11 & 13 are likely to result in (for slightly different reasons). One can but hope sense is seen soon.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#58

Post by mcintyre1994 »

keypusher wrote: post #5

Oath is desperate to put more ads on Tumblr, because they massively overpaid for it ($1 billion) and have been throwing money at it ever since (100s of millions). I guess because of the way the content is displayed they can't just segregate nsfw stuff into a demonetized stream, but if they think they can turn Tumblr into a mainstream site that shows family-centric ads, they clearly don't understand what they bought.

> have been throwing money at it ever since (100s of millions)

Does it actually cost 100s of millions to run? They don't seem to be actively developing it or really doing anything with it other than banning all its content.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#59

Post by newen »

xte wrote: post #9

I do never used Tumblr but AFAICT porn is the most common contents, so i bet the major source of revenue... BTW the neovittorian movement against sex in general (disguised in many form to health things, anti-pornography for pedagogic things, anti-prostitution because of racket, ...) it's a clear sign of our degradation as society.

Why? It's not exactly a stretch to day that pornography and prostitution are clear signs of the degradation of a society.

Re: Tumblr’s anti-porn algorithm is flagging basically everything as NSFW

#60

Post by AJ007 »

dpwm wrote: post #48

It would be quite humorous, and also quite telling of the experience of those implementing this, if it turned out to be classic class imbalance during training. If 90% of the training dataset is nsfw, and you train a classifier naively on that dataset, and your test and validation data is 90% nsfw, then 90% accuracy is very easy.

I would really, really like to know what they are doing. Was it an in-house solution, something from a startup, or something from an established enterprise tech company?

One of the things I’ve been saying repeatedly is that companies are going to implement machine learning incorrectly, and there will be obvious failures and blow ups, but the most dangerous and catastrophic failures will be a cascade of things that look like they work correctly. In this case, Tumblr implemented something that was blatantly obviously broken. That’s just pathetically poor management.

In the next 10 years, I would give high odds of least 1 company in the s&p 500 will fail due to bad machine learning implementation(s.)

Post reply on HN