Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The trauma of Facebook’s content moderators

restofworld.org

Post reply on HN
Print view View original

21–30 of 268 posts

Re: The trauma of Facebook’s content moderators

#21

Post by munificent »

throwaway0a5e wrote: post #11
dmalvarado wrote: post #10

Earlier quoted context omitted.

Perhaps. I would argue that there is some difference between seeing someone fall off a cliff and watching someone being tortured to death.

Then they should run sidebar ads on the gore sites. The point is that there appears to be a subset of the population that is not traumatized by this stuff. We may as well give them an opportunity to put that difference to use. Edit: and saying things like "well clearly these people are already traumatized if they're seeking this out, it's not good for them, we shouldn't give these jobs to them" seems laughably patern…

There is a subset of people who seek this stuff out, yes. Whether or not it is traumatic for them does not strictly follow from that.

People come to their choices with all manner of existing problems and conflicts, and they don't always do things in their own best interest.

Re: The trauma of Facebook’s content moderators

#22

Post by filoleg »

ceejayoz wrote: post #14
throwaway0a5e wrote: post #11

Earlier quoted context omitted.

Then they should run sidebar ads on the gore sites. The point is that there appears to be a subset of the population that is not traumatized by this stuff. We may as well give them an opportunity to put that difference to use. Edit: and saying things like "well clearly these people are already traumatized if they're seeking this out, it's not good for them, we shouldn't give these jobs to them" seems laughably patern…

How do we know they're not traumatized by it? That they seek it out doesn't mean they're not messed up.

^this. I am not arguing one way or another, because I don't see strong evidence making me lean predominantly one way. But imo, it is at least just as possible that they are seeking out that kind of content voluntarily because they are already traumatized in the first place, so this is a coping mechanism.

Maybe it helps dealing with that trauma. Maybe it makes the trauma worse. Maybe there was no trauma to begin with, and the person was seeking out that content simply due to morbid curiosity, but then it evolved into something else. Maybe there was no trauma to begin with and no trauma appeared after the exposure. Who knows.

But just because there are communities on reddit with people seeking that kind of material and not going on crazy violent sprees doesn't mean that it doesn't negatively affect them. It could be the case, but we should probably look into that first before jumping to "obvious solution" claims.

tl;dr: we gotta do more research and weigh pros and cons before simply declaring "we have an obvious solution, let's just hire all those gore content enthusiasts to moderate violent content, because they already do it on their own and don't seem to be going crazy in obvious ways due to it".

Re: The trauma of Facebook’s content moderators

#23

Post by munificent »

Sebb767 wrote: post #4
anonu wrote: post #2

Can we build better AI off of the data these moderators have generated to make their lives easier? Can you crowdsource the moderation task to double check what the ai is flagging?

> Can you crowdsource the moderation task[...]? I highly doubt that. These reported posts might contain private data or highly shocking content; putting that on a public platform, especially associated with the name of Facebook, is very unlikely to happen.

> putting that on a public platform, especially associated with the name of Facebook

Is that not what the person who originally posted the content being moderated is doing?

Re: The trauma of Facebook’s content moderators

#24

Post by throwaway0a5e »

lowmemcpu wrote: post #17

I recall a statistic from about 10 years ago that computer forensic investigators in law enforcement burn out after two years due to the trauma of the images they are exposed to.

There's likely some confounding factors. Pressing "go" on the overpriced software tools and then entering into evidence what you find is the lowest level of work in that field so the churn is going to naturally be very high as people move up or out. The pay also isn't that great.

Re: The trauma of Facebook’s content moderators

#25

Post by CM30 »

It's probably a bit controversial to say this, but haven't large sites had to deal with these problems for years by the time the likes of Facebook/Twitter/YouTube/Instagram/whatever came around?

It's obviously horrible doing this as a job and it's obviously a bit more common on Facebook than say, a large old school internet forum, but... at least in this case people are paid to do this. Reddit mods and old school forum mods have to deal with this stuff for free.

Re: The trauma of Facebook’s content moderators

#26

Post by filoleg »

anonu wrote: post #2

Can we build better AI off of the data these moderators have generated to make their lives easier? Can you crowdsource the moderation task to double check what the ai is flagging?

>Can you crowdsource the moderation task to double check what the ai is flagging?

This would help with false positives, which is a noble goal. But it would not help with false negatives, which is imo a very disastrous consequence in this specific scenario.

Something that isn't CP got marked as CP? Sure, a moderator can manually doublecheck that flag and override if needed. Something that is CP didn't get marked as CP? You got a problem on your hands, because there is no way to manually doublecheck that, since that would mean manually checking every single post marked as "no issues here", which gets us back to square one of manual moderation of all posts.

Not even mentioning the exposure of FB users to CP that it would bring, which would mentally ruin quite a lot of people and bring very negative press to FB on the level of severity that we haven't seen before.

Re: The trauma of Facebook’s content moderators

#27

Post by ceejayoz »

throwaway0a5e wrote: post #16
kylecazar wrote: post #13

Earlier quoted context omitted.

I'd wonder if frequents in these subreddits are typically professional and healthy people to have on a team, though (honestly no idea, never been). It's a strange conundrum because ideally you want good people who are comfortable watching things good people rarely want to watch. Don't have much experience with this demographic -- maybe there are more sane, emotionally healthy people watching snuff films for fun than…

As long as they have a pulse and can sit in the chair long enough to do the job they'll likely be fine. Content moderation farms, call centers and the like tend to optimize their internal processes such that they don't need to restrict their hiring to "professional and healthy people". There are plenty of people who are insane and unhealthy that can hack this sort of work.

Do you want "insane and unhealthy" people who voluntarily look at horrifying stuff deciding whether that stuff should be allowed on the platform, though?

It'd be like having a jury pool made up of serial killers.

Re: The trauma of Facebook’s content moderators

#28

Post by tmpz22 »

CM30 wrote: post #25

It's probably a bit controversial to say this, but haven't large sites had to deal with these problems for years by the time the likes of Facebook/Twitter/YouTube/Instagram/whatever came around? It's obviously horrible doing this as a job and it's obviously a bit more common on Facebook than say, a large old school internet forum, but... at least in this case people are paid to do this. Reddit mods and old school for…

Right but the difference is the sheer amount of money involved and the politicization that comes with that. If Facebook was worth 100m nobody would care. If facebook only has 10m users nobody would care. It's the scale that completely breaks the legal and political frameworks the our society leans on.

Re: The trauma of Facebook’s content moderators

#29

Post by filoleg »

shadowgovt wrote: post #6

I can't help but wonder what the numbers look like on this content. One of the things Facebook should have enough data to know is how prevalent this sort of thing is. Is deeply offensive content generated by a handful of users frequently, or many users less often? What's the volume look like?

Agreed. I would definitely love to see some sort of a basic data analysis blog post on this topic from FB, similar to how match.com used to publish data analysis posts on their blog back in the day.

Re: The trauma of Facebook’s content moderators

#30

Post by throwaway0a5e »

ceejayoz wrote: post #27
throwaway0a5e wrote: post #16

Earlier quoted context omitted.

As long as they have a pulse and can sit in the chair long enough to do the job they'll likely be fine. Content moderation farms, call centers and the like tend to optimize their internal processes such that they don't need to restrict their hiring to "professional and healthy people". There are plenty of people who are insane and unhealthy that can hack this sort of work.

Do you want "insane and unhealthy" people who voluntarily look at horrifying stuff deciding whether that stuff should be allowed on the platform, though? It'd be like having a jury pool made up of serial killers.

>Do you want "insane and unhealthy" people who voluntarily look at horrifying stuff deciding whether that stuff should be allowed on the platform, though?

These kinds of bottom of the barrel places of employment tend to have their policies and procedures tuned to make sure employees do things the way the company wants them done. That's just the nature of doing an unskilled, non-laborious job for a BigCo. There will be automatic spot checks, audits, reviews, etc. It is no different than the cube farm on the next floor where they review images of documents that AI systems have flagged as possibly fraudulent. Such systems are likely in place already for their current moderators.

Post reply on HN