Live data from Hacker News

Is content moderation a dead end?

ben-evans.com

271–280 of 309 posts

Re: Is content moderation a dead end?

#271
post #103

Earlier quoted context omitted.

>Putting the moderation burden on people I don't think GP is saying we should put the moderation burden on people. When you accept that there is no objective definition of right or wrong, you begin to see that perhaps there are ways for people to self-organize on the internet according to their values rather than being shoved into the same box like we often do today. Many people are looking for ways to efficiently an…

"Right or wrong" maybe not, but for well managed communities on the internet, there are objective definitions for appropriate and inappropriate, based on shared values and context. If you leave it up to each individual to decide what is appropriate or inappropriate, and provide them with the tools to block content they consider inappropriate, that's a burden on them, because you're not taking care of it at the commun…

>>"Right or wrong" maybe not, but for well managed communities on the internet, there are objective definitions for appropriate and inappropriate, based on shared values and context.

Objective in what sense? Consensus != Truth.

>>If you leave it up to each individual to decide what is appropriate or inappropriate, and provide them with the tools to block content they consider inappropriate, that's a burden on them, because you're not taking care of it at the community level.

You can't take care of anyone at the community level because a community as an indivisible unit does not exist. The word "community" is merely a label, an abstraction for the ideas and events that characters interactions between two or more different people.

>>And if the community's strength comes from shared values, and you leave that up to each individual to decide, what's shared, and what sort of "community" is actually offered?

Either this is a loaded question or you've really put the cart before the horse, but I'll bite. A community's value comes the achievements of its individual constituents. Shared values are not a required prerequisite. They naturally arise from personal observations and discovery, real or perceived. That's it. Every culture, superstition, law, religion was a consequence of personal examination by one person. What made these so-called shared values "shared" was war and trade.

Re: Is content moderation a dead end?

#272

Earlier quoted context omitted.

Does this apply to a phone call? To a zoom call with five people in it? Should trying to stop someone from doing something, even if abhorrent, in private, really be a priority? I can only see this ending badly. If there is a link to real world crimes, sure, intercepting a discussion is one method available to detect / deter / deny / disrupt, just like a wiretap. But (and maybe I misunderstand your comment) beginning…

Moderating private discussions (e.g. most Zoom calls) are very different from moderating public discussions.

Where's the distinction? It's like saying a phone call is functionally different from a conference call. Both Zoom and Clubhouse use private infrastructure, thus all discussions are private even if many people are involved in them.

Re: Is content moderation a dead end?

#273

I think we're looking at the issue in a completely wrong way. There's no objective definition of right or wrong in content moderation. Right and wrong is subjective, especially across cultures, and moderation should be subjective too. I believe end users should have the choice to adopt blocklists, Adblock style. Those lists could contain single posts, accounts, or even specific words. A lot of content (like flashing…

Where nobody is willing to stand in on the editorial side, the decision is being made, effectively, by the submitter pool with the largest content creation capacity, and its interaction with algorithmic and promoted content dissemination. That is, the decision is already being made, the only question is whether or not the results have strong truth valance.

The ship has, in a word, sailed.

If you create a vast, global, instant, high-fidelity, interactive content dissemination system with a strongly aspirational and appealing audience, then parasitic opportunists who seek only to serve their own rather than others' interests will flock to it. And have, in spades.

And I've seen this game play out repeatedly: in print, on radio, television, Usenet, email, and since approximately 1997, on the Web and mobile Internet.

There are parties already making truth determinations, and their doing a demonstrably horrible job at it from a common weal perspective. And that is the problem.

"Just don't use Facebook" doesn't work for two principle reasons:

1. Facebook is increasingly central to, or required for, numerous real-world interactions.

2. Even if, as I do, you don't use the service, you live in the world it creates. Facebook has massive negative externalities. Like, oh, say, civil war and genocide in Myanmar, to mention only one aspect.

(There are others closer to home for most readers here. I'm hoping HN won't lose its collective mind if I don't mention these.)

Another element that factors in is that Big Lie propaganda is reliant on disseminating the Big Lie. Scale is directly the problem, and offering unrestricted access to, choose the amplification metaphor of your choice, the printing press, microphone, camera, TV/Radio station, etc., has risks. Especially for those who would see the tools themselves burned down along with all else.

In which case, drastically curtailing the spread of any content from identified actors and their associated networks responsible for spreading obvious and notable disinformation ... is highly defensible.

I'm well aware of numerous arguments, predicated on or observing exceptions to free speech, which typically follow such statements. I find both the free-speech absolutist and the private property / private actor restriction privilege arguments tired and uninspired. The reality is complex, and I don't have either simple solutions or any which are conformant simultaneously with "free speech" or "property rights" positions.

Both, to my mind, exist in a nexus and network of overlapping interests and rights. I've suggested "information autonomy" as an alternative to "free speech", though the expansion of that notion quickly points out internal conflicts. Common weal might offer one path out.

Re: Is content moderation a dead end?

#274
post #199

Earlier quoted context omitted.

What is a social network? Fundamental its a graph structure of human relationships. Humans are the actors/nodes and edges are the interactions. Two people chatting fits this criteria. You can have a private chat on Clubhouse between two people.

> Clubhouse is not just private phone calls. Yes, you can have a private chat on Clubhouse. You can also have a private chat on Facebook or Twitter. They are all social networks. I don't think trying to find edge cases for what constitutes a social network is helpful to this conversation.

Versatility is not an edge case. By your standard, conference calls can't be considered phone calls because few people ever used the feature.

Re: Is content moderation a dead end?

#275
post #82

This is a weird take. You get lots of messages from Nigerian scammers, but the solution was not to prevent people from writing freeform emails. The solution was to build powerful spam detection algorithms, make it easy for people to classify emails to help strengthen the training set, and the problem is basically solved. There's no easy answer to content moderation. There's no one size fits all solution, nor is there…

The threat models from 419 (Nigerian / advance-fee fraud) spam are:

- Individuals are subject to a fraud risk.

- Trust in the medium as a whole is decreased.

- Risk of false-positive spam flagging increases, decreasing legitimate message transmission reliability.

- Fraud will continue so long as costs of operation are lower than rewards.

Economic spam, in a word, follows an economic model, and is subject to economic attacks. The notion of some form of "email postage" is one ultimate form (thus far both impractical and widely rejected), but other "increasing the costs of business" have had ... significant if not perfect ... results in limiting the activity.

The calculus for political propaganda is vastly different, with the rewards of even a marginal success being tremendous, and the resources of antagonists being prodigious. It is very much like war, that most costly of economic activities.

It is war.

It is information warfare, rather than physical warfare. The weapons are messages, messengers, channels, informants, and audiences, rather than missiles and knives and fire and water and chariots (or tanks or helicopters or bombers or ...).

But Sun Tzu wrote of both types 2,500 years ago.

Unfortunately, the fundamental architecture and premises of a major contingent of participants fundamentally enables the attackes presently being conducted. (This is of course something of a tautology: if a terrain is fundamentally favourable to a specific mode of attack, that attack will tend to be employed, successfully, on that terrain.)

Re: Is content moderation a dead end?

#276

I see it as kind of a funnel. First you decide how much participation you want to allow in the first place, and that's a big lever. Smaller niche communities are easier to moderate because a lot of policies are customs, meaning you don't need to make them explicit. Another lever is related - decide how much you want to limit the kind of content. A like button is easier to moderate than upvote/downvote, which is easie…

Kuro5hin's moderation and reputation system had some novel-for-the time (and still comparatively rare) features, though it ultimately suffers much the same failure as most other crowdsourced moderation systems, and ultimately doesn't work under sustained attack or concerted effort.

For those unfamiliar, Kuro5hin, inspired by Slashdot, was an online discussion site where individuals would submit items, along with a (hopefully) brief description, or an original essay. There was no editorial queue (as Slashdot had and has). Both posts and comments could be rated on a five-point scale by individuals, and the ultimate score landed in the bound 1-5 (later extended to 0 or -1 with flags). Moderation converged on a mean with time, at least in theory. Users "mojo" (karma) would follow similarly.

This avoids both the runaway endless votes problem of sites such as Reddit (or Hacker News), and the behaviour of Slashdot in which moderation would rise or fall but never converge and was artificially capped at 5 points. On Kuro5hin, the 100th moderation score was weighted as 1/100. On Slashdot, the 100th moderation score had a weight of 1.

If this sounds something like a star-based review system such as Rotten Tomatoes, or Amazon ... it's because it is. And ultimately the same problems apply:

- Sybil attacks in which sockpuppet or shill accounts are created to sway moderation / review scores are quite possible.

- If the site fails to attract (or motivate) consciencious moderators, the moderation quality still falls.

- Information on divergent (strongly bimodal) rating is lost. 10 moderations split between 5 '1's and 5 '5's look the same as 10 '3' ratings. The information contained is not the same. (Summarising complex data is, it turns out, complex.)

- There's still no distinction between popular and truthful content. Or between "this is true" and "I disagree with this" (or don't like it, or the submitter looks funny, or whatever).

The K5 system was interesting, but ultimately not a magic bullet. The site itself failed, though for complex reasons.

Re: Is content moderation a dead end?

#277
post #40

I'd love seeing an anonymous peer review approach, to post anything you need to review X other posts. Until there are N accepts posts are invisible. I think it could work, but I am sure HN can tell me how I am wrong :)

There's still the Sybil problem.

And good submitters != good reviewers.

That said, a variant in which less vetted content has to be reviewed by more vetted reviewers, before gaining wider disclosure, seems to me to have merits.

Re: Is content moderation a dead end?

#278

A system that limits the number of posts individuals can make on a per-article or per-diem basis would go a long way to silencing overly strident voices which, out of their sheer verbosity and pertinacity make it appear that their (often) extreme views are more prevalent and widely accepted than they in fact are. An alternative is to generate an in-forum currency that can be spent on comments, either on a per-post or…

Sybil problem.

Per-account limits don't apply to those with unlimited accounts.

Some form of payment (or hash cash) is an option, but hugely favours those with the ability of those who can pay (or hash). This correlates strongly with current sources of disinformation.

Re: Is content moderation a dead end?

#279

Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat. You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bub…

I was part of solving a content moderation problem for a tech company forum once. The most troublesome users were often the most prolific posters. The people who had a lot of free time on their hands to post every single day were often the most disgruntled, seizing every issue as a chance to stir up more controversy. It was tough enough to reel in difficult users when they had no power. Giving extra power to people w…

The fact that there's very little overlap between those with the inclination to contribute and those with value to contribute, is fundamental to virtually all communication channels / fora. Unless the channel is exceedingly aggressively managed, the former will outweigh the latter.

An early form of this appeared as an observation on Hal Varian's page at UC Berkeley (he's an economist, was with the School of Information Management Science, and now works for Google).

On HN it's interesting to look at what kind of riff-raff appear high on the leaderboard, vs. revered names in the field who may have posted only a few times.

Neither truth nor quality are popularity (or volume) contests.

Re: Is content moderation a dead end?

#280

Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat. You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bub…

Rather than use high engagement as a basis for vouching, arbitrarily selected communities, perhaps of about 50--150 active participants (posting or moderating) might be better. Think Mastodon, but in order to be federated, good posts must be "voted off the island", in the good sense.

I've been applying a somewhat similar notion to collecting and managing reading material and suggestions, what I call BOTI, or Best of the Interval. It's a round-robin style system (think "43 Folders" from GTD), where I compile a list of references over a period of time (monthly to annually seems to be most appropriate for me, though hour / day / week / month / year / decade / ... could be applied). At the end of an interval, some limited number of items is carried forward.

This is one way of addressing the "firehose of content" nature of information, recognising that in any given time period, you only have so much personal bandwidth to dedicate.

With the federated model, content federating is itself subject to assessment, and is effectively re-vetted. Note that different communities might favour different content: cooking, kittens, Kabbalah, canoodling, cypherpunk, core meltdowns, classic cars, concerthalls. Vetted / re-vetted streams themselves might be of interest.

Both community- and time-based elements of this could get interesting.

Post reply on HN