Is content moderation a dead end? • Hacker News Board

Re: Is content moderation a dead end?

#281

Post by dredmorbius » Wed, Apr 14, 2021, 8:49 PM UTC

michaelmrose wrote: post #99

weird-eye-issue wrote: post #56

Earlier quoted context omitted.

It's funny that you comment that on HN

Which has a single front page which shows the same headlines to everyone where people who disagree can all see each others posts and we can disagree with each other so long as we can avoid being jerks to one another. At worst you lose imaginary internet points if you say something that the group doesn't agree with.

HN doesn't much create internal filter bubbles.

It is something of a collective filter bubble, and there is a pervasive criticism from those who largely don't participate here of typically HN behaviours. I agree with a fair bit of that. There are certainly topics HN doesn't seem to be able to reasonably discuss. (This seems to be a frustration of the mods as well. They're certainly aware of the issue.)

My own (in my view at least) largely contrarian voice seems to have been reasonably well tolerated here, though.

Re: Is content moderation a dead end?

#282

Post by dredmorbius » Wed, Apr 14, 2021, 8:52 PM UTC

cortesoft wrote: post #19

PicassoCTs wrote: post #10

Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat. You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bub…

Also, if there is a severe penalty for vouching for bad people, but not much gain for vouching for someone, will you end up with no one wanting to vouch for anyone else?

Bad behaviour of this nature tends strongly to operate at scale.

Bad actors form massive communities, and quickly. You can look at the rise of various networks, such as T_D on Reddit, or the rapid growth (and often fast decline) of other such forums elsewhere. I find it very difficult to believe that such growth is organic.

(Commercial and brand accounts / pages / platforms often follow similar trends, and for similar reasons. Advertising and propaganda are one and the same.)

Re: Is content moderation a dead end?

#283

Post by ArneBab » Wed, Apr 14, 2021, 9:30 PM UTC

crazygringo wrote: post #50

PicassoCTs wrote: post #10

Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat. You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bub…

Yes, I'm convinced at some point we're going to figure out an algorithm to solve content moderation with some version of crowdsourcing like this based on reputation, though I'd prefer a system based on building up trustworthiness through one's actions (consistently flagging similarly to already-trustworthy people). But the challenge is still the same one you describe -- what do you do with competing "groups" or subco…

In the Freenet Project we’re solving content moderation with propagating trust — and visibility slowly increasing with social interaction, which is free for honest users (who just want to communicate) but has a cost for spammers and disrupters.

The main tool is to drop the idea of a global definition of trustworthiness: You have seed IDs to find people who supply CAPTCHAs, but otherwise all trust is computed locally.

If you want to try this, there are two steps:

- First the current state in Freenet: https://github.com/xor-freenet/plugin-WebOfTrust/blob/master...

- Then the optimizations needed so this scales to arbitrary size: https://www.draketo.de/english/freenet/deterministic-load-de...

Here’s some data if you want to test algorithms: https://figshare.com/articles/dataset/The_Freenet_social_tru...

And some starting code of a more generic prototype for faster testing: https://hg.sr.ht/~arnebab/wispwot/

(if you prefer a slightly larger writeup that might expand as time moves: https://www.draketo.de/software/decentralized-moderation.htm... )

Re: Is content moderation a dead end?

#284

Post by johnnyanmac » Wed, Apr 14, 2021, 11:14 PM UTC

motohagiography wrote: post #104

It's hard to separate content moderation from the problem of Evil. Low entropy evil is easy to automate out, high entropy and sophisticated evil can convince you it doesn't exist. This is also the basic problem of growing communities, where you want to attract new people while still providing value to your core group, while still managing both attrition and predators. What content moderation problems have proven is t…

>There is a "be excellent to each other" rule which gives very reasonable blanket principle powers to moderators, and it's pretty easy to check.

easy to check, but most sites tend to have almost zero accountability for moderators "staying excellent" and most moderators are often unpaid, normal human beings with their own personal beliefs. They won't be let go unless they frustrate the site owners.

And to begin with, people's perceptions of if "laziness" and especially "stupidity" should be aligned with "Evil" is a question for discussion in and of itself. Some people may just want to throw around some jokes and memes and not be in some pristine forum full of essays for response. It's not my personal preference, but I also don't think it's worthy of a blanket internet ban, nor do I necessarily want to say forums for those people are of "lesser quality" (even if you can argue that it attracts more truly "Evil" people).

I guess it's a tiptoe as moderation as always. I've also see some forums lean too deeply on quality and end up feeling gatekeep-y as a result (and not even for sensitive topics where it may be needed, but some "casual" forums, as advertised). You turn a community into work and people will look elsewhere.

Re: Is content moderation a dead end?

#285

Post by johnnyanmac » Wed, Apr 14, 2021, 11:26 PM UTC

lifeisstillgood wrote: post #113

>>> One could also think of big European cities before modern policing - 18th century London or Paris were awash with sin and prone to mob violence, because they were cities. Is the solution in the article? Do we simply need to recognise that as all society is online we now need online police? Online Community support officers (UK police adjuncts (think teaching assistants). I suspect there is an overlap with the "de…

>Do we simply need to recognise that as all society is online we now need online police? Online Community support officers (UK police adjuncts (think teaching assistants).

International issues would get in the way. As much as it sucks, most police won't act on a death threat made from some 12 year old in Brazil towards a normal US citizen. It's a lot of resources for no benefit on their end.

it wouldn't be enforceable unless websites are enforced to only be used by citizens within that country (and worldwide punished for not abiding), with specific embassies for international conversation. Some cases may even divide it within states/providences.

I see that as the worst case of a medium that was created with the goal to bring the world together.

Re: Is content moderation a dead end?

#286

Post by crazygringo » Wed, Apr 14, 2021, 11:28 PM UTC

ArneBab wrote: post #283

crazygringo wrote: post #50

Earlier quoted context omitted.

Yes, I'm convinced at some point we're going to figure out an algorithm to solve content moderation with some version of crowdsourcing like this based on reputation, though I'd prefer a system based on building up trustworthiness through one's actions (consistently flagging similarly to already-trustworthy people). But the challenge is still the same one you describe -- what do you do with competing "groups" or subco…

In the Freenet Project we’re solving content moderation with propagating trust — and visibility slowly increasing with social interaction, which is free for honest users (who just want to communicate) but has a cost for spammers and disrupters. The main tool is to drop the idea of a global definition of trustworthiness: You have seed IDs to find people who supply CAPTCHAs, but otherwise all trust is computed locally.…

Thanks for that, I'll take a look!

Re: Is content moderation a dead end?

#287

Post by johnnyanmac » Wed, Apr 14, 2021, 11:30 PM UTC

dan-robertson wrote: post #175

I wonder how well somethingawful-style signup fees solve this problem: pay $10 to sign up and if the platform doesn’t like your behaviour they can ban you or set you back to the pre-payment stage. This then somewhat disincentivises bad behaviour and, in particular, repeated or automated bad behaviour. Surely this would harm growth and it doesn’t feel like you’d have success charging a smaller fee in poorer countries.…

If news sites are having trouble getting people to pay to read news, I don't feel much more optimistic about a forum paying for the privelege of commenting (I say this as someone with a SA account). Maybe in the 90's it could have been normalized, but nowadays there are dozens of "free" alternatives, so the idea is considered absurd. The last thing a site trying to compete with FB/Twitter needs is more barriers, and it won't fix the problem on the largest sites even if they end up moderately successful.

Re: Is content moderation a dead end?

#288

Post by johnnyanmac » Wed, Apr 14, 2021, 11:41 PM UTC

tunesmith wrote: post #128

I see it as kind of a funnel. First you decide how much participation you want to allow in the first place, and that's a big lever. Smaller niche communities are easier to moderate because a lot of policies are customs, meaning you don't need to make them explicit. Another lever is related - decide how much you want to limit the kind of content. A like button is easier to moderate than upvote/downvote, which is easie…

The problem with a "smart jerk" is that (from my experience) many mods are awful at separating tone from opinion. Because the discussion always seems to inevitably shift from the starting spirit of "don't be an asshole" to "don't have asshole opinions". However, depending on how invested one is, they may see certain opinions themselves as being unwelcoming. And so the filter bubbles occur, be they for something as impactful as politics or as inane as your favorite TV show.

Re: Is content moderation a dead end?

#289

Post by tunesmith » Wed, Apr 14, 2021, 11:44 PM UTC

dredmorbius wrote: post #276

tunesmith wrote: post #128

I see it as kind of a funnel. First you decide how much participation you want to allow in the first place, and that's a big lever. Smaller niche communities are easier to moderate because a lot of policies are customs, meaning you don't need to make them explicit. Another lever is related - decide how much you want to limit the kind of content. A like button is easier to moderate than upvote/downvote, which is easie…

Kuro5hin's moderation and reputation system had some novel-for-the time (and still comparatively rare) features, though it ultimately suffers much the same failure as most other crowdsourced moderation systems, and ultimately doesn't work under sustained attack or concerted effort. For those unfamiliar, Kuro5hin, inspired by Slashdot, was an online discussion site where individuals would submit items, along with a (h…

Ah, interesting. I could have sworn K5 had a vouching system, where the strength of a rating was partially dependent on the rating of the person doing the rating.

Re: Is content moderation a dead end?

#290

Post by johnnyanmac » Wed, Apr 14, 2021, 11:51 PM UTC

ericbarrett wrote: post #57

PragmaticPulp wrote: post #39

Earlier quoted context omitted.

I was part of solving a content moderation problem for a tech company forum once. The most troublesome users were often the most prolific posters. The people who had a lot of free time on their hands to post every single day were often the most disgruntled, seizing every issue as a chance to stir up more controversy. It was tough enough to reel in difficult users when they had no power. Giving extra power to people w…

I used to moderate a few message boards, and I fully agree. I think empowering the "power users" like this inevitably leads to Stack Overflow-style communities, where arrogant responses are the norm, the taste of a few regulates the many, and the culture of the community ossifies because new contributors are not welcomed.

The factor missing with a SO style "meritocracy" is that it needs to be hard to rise up, but very easy to fall down from misconduct: in other words, a built in accountability mechanism to ensure they don't abuse their power and become arrogant over it. Not necessarily ban worthy offenses (lest, they simply be banned), but ones that make a less welcome community.

I'm lost on how to do this within an internet community, however. You don't want burners being made to report over a disagreement in opinion and impact them. Or worse, power users tearing each other down in retaliation. I don't think "election periods" would work well either because few people really care to be a moderator to begin with. I can only see it being done by having professional moderators and having a manager review any potential user complaints. Which I'm sure has other problems.