The trauma of Facebook’s content moderators • Hacker News Board

Re: The trauma of Facebook’s content moderators

#161

Post by msapaydin » Wed, Jul 22, 2020, 8:53 PM UTC

three_seagrass wrote: post #145

nitwit005 wrote: post #133

Earlier quoted context omitted.

It has been. Most of these sites catch a ton of images and video automatically when they are similar enough to prior known content. That doesn't matter from a staff point of view though. You have a queue to work through. You'll be putting in an 8 hour day dealing with the stuff the system doesn't catch. The automation just means they don't need as much staff.

Yep. One way to view automation with machine vision / perception is that it can cover ~85-95% of the true positives. You're still going to get false positives and false negatives that need human review, and at a scale of Facebook, that's a lot of humans.

I am just hoping that those currently uncovered cases will be "milder or more nuanced" cases that will be less damaging to the psyche of human moderators and will, once labeled correctly, improve the coverage rate of automated moderators.

Re: The trauma of Facebook’s content moderators

#162

Post by shadowgovt » Wed, Jul 22, 2020, 8:53 PM UTC

rosywoozlechan wrote: post #38

Post gore, rape and torture in places where people don't want to see that and you should lose the right to have internet access. Criminalize this behavior.

As a less final measure, we can allow organizations to choose to discontinue service to users who post content they find objectionable.

Re: The trauma of Facebook’s content moderators

#163

Post by shadowgovt » Wed, Jul 22, 2020, 8:55 PM UTC

uniqueid wrote: post #87

anonu wrote: post #2

Can we build better AI off of the data these moderators have generated to make their lives easier? Can you crowdsource the moderation task to double check what the ai is flagging?

> Can we build better AI Nope. That's what the people in charge of our social media companies keep trying, over and over, and it doesn't work at all. AI-moderation is like the Ring is to Golem for them. They can't accept that "cheap and fair" moderation isn't currently possible. > Can you crowdsource the moderation task This is the clear answer, imo, and it isn't obvious to me why these C-level execs avoid it. If I h…

When the goal is to prevent users from seeing this content, crowd-sourcing moderation defeats the purpose.

Users don't want the back-stop to be "a critical mass of community members object;" for content like this, they want to see zero of it, ever, and will choose to use another site that satisfies that hard constraint if FB cannot.

Re: The trauma of Facebook’s content moderators

#164

Post by TonyTrapp » Wed, Jul 22, 2020, 8:56 PM UTC

CM30 wrote: post #25

It's probably a bit controversial to say this, but haven't large sites had to deal with these problems for years by the time the likes of Facebook/Twitter/YouTube/Instagram/whatever came around? It's obviously horrible doing this as a job and it's obviously a bit more common on Facebook than say, a large old school internet forum, but... at least in this case people are paid to do this. Reddit mods and old school for…

I have moderated several forums over the course of 15+ years, and I have never come across this kind of content. But then again, all of them were very specialized in their topics. I still am convinced that we would be better off if we went back to small forums for people sharing common interests rather than having a huge platform. People can police their small group of peers much more easily. It works.

Re: The trauma of Facebook’s content moderators

#165

Post by Tiksi » Wed, Jul 22, 2020, 8:56 PM UTC

slg wrote: post #73

irrational wrote: post #66

I'm surprised there hasn't been a revenge movie where one of these moderators use their access to find out where these people live and start hunting them down like some sort of batman-like person. I could see a movie like this being a means of making the more general population aware of the terrible stuff content moderators have to watch.

This is totally not what you are talking about, but the closest thing I have seen to that is the Netflix docuseries Don't Fuck With Cats. It hits on a lot of the same topics. The primary differences are that the hunters are Facebook users and not moderators and they are hunting the person to get justice and not violent revenge.

There's a show called Darknet https://en.wikipedia.org/wiki/Darknet_(TV_series) that's somewhat along these lines too, but it's very much in the horror genre.

Re: The trauma of Facebook’s content moderators

#166

Post by Nasrudith » Wed, Jul 22, 2020, 8:58 PM UTC

praveen9920 wrote: post #9

The issue is as real as western countries dumping garbage in eastern countries just because they can pay them off. The local governments won't acknowledge this as problem because of the money flowing in and a lot of people's livelihood is dependent on this. But we should call it what it is, exploitation.

The awkward ethical question is would they be better off without "exploitation"? Part of the reason companies are even willing to go through the overhead is because the costs are lower.

Real life has always been filled with messed up situations where the punchline is "... and the best/worst part is it qualified as an improvement!".

Re: The trauma of Facebook’s content moderators

#167

Post by wdgz » Wed, Jul 22, 2020, 9:00 PM UTC

AdmiralAsshat wrote: post #160

victoriasun wrote: post #148

Earlier quoted context omitted.

But are people like that people you want as your co-workers? Serious question, I don't mean this facetiously. I think is an extremely complex question and I wonder if introducing someone who is openly comfortable with gore is dangerous to the psychological health of the company. For example, at a previous company a co-worker publicly shared, without a trigger warning and in great detail, a very gorey thing he enjoyed…

Well, I probably wouldn't invite him to my game night, but if it's his job to watch ISIS videos, it seems like he'd be well-suited, no? It seems like if you acknowledge that: a) Terrible job exists b) Terrible job is necessary You should also allow for a person to do terrible job without thinking that they must be a terrible person by extension. Otherwise you continue the well-trodden history of certain professions l…

I'm sorry if I implied that I think that people who would enjoy this work are bad people. I certainly don't feel this way; I mean, I myself have worked on a platform that plenty of Nazis use and i don't consider myself a Nazi, so it would be hypocritical of me to say as such.

I agree that I wouldn't personally spend time with said person, but my question was less about personal social responsibilities outside of work. My example I pointed out happened at work, during normal work hours, in a social space where people have inherently less control over who they surround themselves with. This is the part of it all that is fuzzy to me, and I don't have a clear answer for.

Re: The trauma of Facebook’s content moderators

#168

Post by maerF0x0 » Wed, Jul 22, 2020, 9:03 PM UTC

I cant help wonder about the economic(choices) side of this around 1) Should this victimization be taken into account when punishing offenders eventually tried and 2) Is the damage done to the content moderators marginally worth keeping the marginally caught content off the platform (recall there already is automation doing the bulk of the work)...

I do think we need to think about the side effects of laws where some US citizen demands that Facebook must employ content moderations else be liable for hosting it, it doesnt result in Facebook executives, engineers et al. bearing the burden. It results in some poor person in a 3rd world country bearing the burden.

Kind of like the environmental aspect of things I would think the best investment is in total prevention of the heinous acts in the first place. ie, it's much cheaper to prevent CO2 than it is to clean it up after the fact.

Re: The trauma of Facebook’s content moderators

#169

Post by SeymourGlass » Wed, Jul 22, 2020, 9:08 PM UTC

three_seagrass wrote: post #140

stickfigure wrote: post #126

Earlier quoted context omitted.

This seems like something specific to the individual. Maybe this isn't the job for you, but there's probably enough people "made of different stuff" that could handle it? I feel like there's a subtext in this discussion that "nobody can handle it" or "you have to be a monster to work this job" but that's just not true. Just out of college in the mid-90s, I worked for a few months at a small local software company in…

It doesn't seem to be specific to an individual when there are class actions lawsuits and many people reporting to suffer from it. There exist jobs, like slaughterhouse knockers, which are linked to causing PTSD and increased criminal behavior. It's not far fetched to believe these moderator jobs do too.

Yeah, and they made an interesting point about how these problems are compounded by the lack of mental health awareness and the difficulties of defending mental health trauma in a court of law in countries like India and the United States. I didn't think of that before.

Re: The trauma of Facebook’s content moderators

#170

Post by Nasrudith » Wed, Jul 22, 2020, 9:08 PM UTC

jbrennan wrote: post #77

In my opinion this is an argument in favour of shutting Facebook (and every other large social network) down. This sort of work is damaging and abusive, and nobody should have to endure it just so we can have social networks. I understand there are some jobs in the world that need to deal with dark stuff (like law enforcement), but social networks just aren’t worth the human cost.

How on earth does the size make any difference to the supposed problem? Even if we somehow had a stable and viable size capping system to whatever nebulous concept of "too big" there would still be people exposed to the same content. Not to mention in the US said restrictions would get a "haha no" from the courts on First Ammendment grounds against arbitrary limitations on speech by source.