Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The trauma of Facebook’s content moderators

restofworld.org

Post reply on HN
Print view View original

111–120 of 268 posts

Re: The trauma of Facebook’s content moderators

#111

Post by throwawaytime40 »

untog wrote: post #35
MattGaiser wrote: post #3

They should recruit people on Reddit for this. A large number of them seem perfectly fine with this type of content. There are dedicated communities to all these things. I do not mean this as a hit against Reddit users. It is just that there are small pockets of people who are psychologically capable of tolerating this or even enjoy it. It seems more ethical to hire people from /r/watchpeopledie.

> It is just that there are small pockets of people who are psychologically capable of tolerating this or even enjoy it. I wouldn't assume "watches this stuff" and "is psychologically capable of tolerating this stuff" are one and the same. Those people might be actively harming themselves by watching the content. That's one thing when they're volunteering to do so (though that's an interesting conversation in and of…

Honestly, this post speaks a lot of truth, at least in my experience. I use to be subbed to /r/watchpeopledie, really just out of morbid curiosity.

One day I had an anxiety attack because I had a dream that I had a heart attack and suddenly I had a deep fear of randomly dying.

All the random deaths on that sub started to really get to me, I even started to get depressed. It became harder and harder to watch content on it. Eventually I unsubbed from it and gradually bounced back to normal.

I don't really regret it because it gave me a new perspective on life and how valuable it is. But I would be lying if I said it didn't fuck me up in a way.

Re: The trauma of Facebook’s content moderators

#112

Post by exolymph »

teddyh wrote: post #83
kylecazar wrote: post #13

Earlier quoted context omitted.

I'd wonder if frequents in these subreddits are typically professional and healthy people to have on a team, though (honestly no idea, never been). It's a strange conundrum because ideally you want good people who are comfortable watching things good people rarely want to watch. Don't have much experience with this demographic -- maybe there are more sane, emotionally healthy people watching snuff films for fun than…

> maybe there are more sane, emotionally healthy people watching snuff films for fun than I instinctively expect. Probably not, since there are no documented instances of any snuff films ever found.

A snuff film is a video in which someone actually IRL dies. There are about a zillion of them.

Re: The trauma of Facebook’s content moderators

#113

Post by jbrennan »

dangus wrote: post #88
jbrennan wrote: post #77

In my opinion this is an argument in favour of shutting Facebook (and every other large social network) down. This sort of work is damaging and abusive, and nobody should have to endure it just so we can have social networks. I understand there are some jobs in the world that need to deal with dark stuff (like law enforcement), but social networks just aren’t worth the human cost.

That’s a non-solution. What you’re proposing is not just shutting down social networks, it’s shutting down any website that involves user content, anything that allows photo/video upload, comments, or any kind of user interaction. That’s impossible. You point out that public safety jobs are view more “worth it,” and certainly they are, but that logic brings up the question of who judges what job is worth undergoing t…

I’m actually not proposing shutting down any website that allows uploaded content, just large / public sites that require this sort of moderation. Not every site gets this stuff uploaded to it. The more private the network, the less need for this kind of company-led moderation.

As far as “worth it” goes, some people have to be exposed to it so long as we have law enforcement (but I’m certainly open to alternatives here). I’m not sure the train operator is a fair comparison, because seeing a suicide is an exceptional circumstance in their job, it’s not the norm. The content moderators, however, are sadly expected to be exposed to traumatizing content as part of their job description — it’s essentially the point of their job.

There are plenty of kinds of work we deem as hazardous to people’s health, and thus are either banned or regulated. I’m not sure if there’s a healthy way to expose people in these moderator jobs to the traumatizing content they face. It just doesn’t seem worth the tradeoff to endanger them like this.

Re: The trauma of Facebook’s content moderators

#114

Post by ElliotH »

msapaydin wrote: post #95

What I am wondering, and this is probably a dumb question, is why this has not been automatized? Can't content moderation be done with modern and strong machine learning based systems? There must be plenty of training data on this, and just like a spam filter which does not require humans in most cases, this should also be automatable. Why is it not?

What everyone else said, but also the presence of false positives means there'll be appeals. So you have a choice or ignoring the appeals and being accused of censoring people, or processing them - and that tends to need a person.

Re: The trauma of Facebook’s content moderators

#115

Post by remcob »

indigochill wrote: post #78
CM30 wrote: post #25

It's probably a bit controversial to say this, but haven't large sites had to deal with these problems for years by the time the likes of Facebook/Twitter/YouTube/Instagram/whatever came around? It's obviously horrible doing this as a job and it's obviously a bit more common on Facebook than say, a large old school internet forum, but... at least in this case people are paid to do this. Reddit mods and old school for…

>at least in this case people are paid to do this. I've done commercial moderation before and I would sooner go jobless than go back to exposing myself to the kinds of things people have to when doing this professionally. It's the kind of things you wish there was a brain cleaning solution for, but there isn't so it's stuck there basically forever (though I suspect the alcoholism rate is higher among people who have…

> wish there was a brain cleaning solution

'Drug-induced amnesia' to reduce long term memory generation is a thing and is sometimes used before traumatic surgical procedures. It needs to be taken in advance and is probably not very healthy. Plus I would have serious ethical concerns with jobs that would encourage that.

Re: The trauma of Facebook’s content moderators

#116

Post by teddyh »

exolymph wrote: post #112
teddyh wrote: post #83

Earlier quoted context omitted.

> maybe there are more sane, emotionally healthy people watching snuff films for fun than I instinctively expect. Probably not, since there are no documented instances of any snuff films ever found.

A snuff film is a video in which someone actually IRL dies. There are about a zillion of them.

A snuff film is defined by most as at least having been made for the purpose of entertainment of others. Some include the proviso that it has to have been made for profit, and yet others demand that it has at least been sold. But regardless, not even the first level of these criteria have ever been fulfulled.

https://www.snopes.com/fact-check/a-pinch-of-snuff/

Re: The trauma of Facebook’s content moderators

#117

Post by ericjang »

I read somewhere long ago that there have been incidents of FBI employees tasked with reviewing child pornography [1] who saw so much that they became desensitized or even aroused by it. Does anyone know if this is well-documented, or was this an exception rather than a common occurence?

[1] https://www.fbi.gov/history/famous-cases/operation-innocent-...

Re: The trauma of Facebook’s content moderators

#118

Post by pixelbath »

msapaydin wrote: post #109
pixelbath wrote: post #106

Earlier quoted context omitted.

>this is probably a dumb question Not dumb, but I do think it's not thought-through. You're proposing a simple solution to what is a hugely complex problem, and throwing ML at it just isn't going to work. To get "plenty of training data," a human still has to classify all of that, leading to the problem of viewing that much unpalatable content by a human. You also have to train your network, which requires humans to…

I am guessing that by now plenty of data has already been generated which could be used to train future systems..

I'm guessing you don't actually understand the complexity involved.

How does it differentiate between normal ranting and hate speech? How is hate speech classified in the US vs Saudi Arabia? How does it tell the difference between someone asking a child innocent questions and asking them sexually-related questions on camera? Does the algorithm get trained to flag videos about depression that might lead to suicide, or does it say they're supporting getting help FOR depression and leave the video?

What about subjects that aren't already in the corpus of "flag these naughty things"? You still have to get a human to look at those; most likely a data scientist who knows what the algorithm is doing and what needs to be done to correct the training. Machine learning as-is will not get us there, so in the meantime the only option is moderation by people described in the article. It can be outsourced elsewhere, but it's just shifting the responsibility to a different subset of people.

Re: The trauma of Facebook’s content moderators

#119

Post by pgwhalen »

jbrennan wrote: post #77

In my opinion this is an argument in favour of shutting Facebook (and every other large social network) down. This sort of work is damaging and abusive, and nobody should have to endure it just so we can have social networks. I understand there are some jobs in the world that need to deal with dark stuff (like law enforcement), but social networks just aren’t worth the human cost.

This is not just a facebook problem though. Taking your proposal to its logical conclusion suggests that the internet should have no user created content on it - or am I missing some possible middle ground?

Re: The trauma of Facebook’s content moderators

#120

Post by msapaydin »

anonu wrote: post #2

Can we build better AI off of the data these moderators have generated to make their lives easier? Can you crowdsource the moderation task to double check what the ai is flagging?

I also made a comment before I saw yours about creating a machine learning based system to automate this process, I think it must be fairly easy given the amount of data already available.
Post reply on HN