Live data from Hacker News

Is content moderation a dead end?

ben-evans.com

151–160 of 309 posts

Re: Is content moderation a dead end?

#151

Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat. You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bub…

Yes, I'm convinced at some point we're going to figure out an algorithm to solve content moderation with some version of crowdsourcing like this based on reputation, though I'd prefer a system based on building up trustworthiness through one's actions (consistently flagging similarly to already-trustworthy people). But the challenge is still the same one you describe -- what do you do with competing "groups" or subco…

Sounds like a network analysis problem to tackle.

Re: Is content moderation a dead end?

#152
> Microsoft made it much harder to do bad stuff, and wrote software to look for bad stuff.

Before MS there was no bad stuff on the commodore 64. It just didn't exist. Loading things from tapes, disks or the internet doesn't matter. You switch it off and on again then load the next thing. I see no reason why this cant scale. You would have problems if you allow unchecked io and remote code execution and you would have to deal with that but even then a simple reset would clean it up. There is no need to give strangers the keys to your home and offer them a place to hide where you cant find them.

> Virus scanners and content moderation are essentially the same thing - they look for people abusing the system

The problem is that it is not your page. This forces you to live up to someone else's standards (if not forein laws) It is like the PC architecture where the computer is not yours.

Facebook is really what web standards should have offered. I would probably have been against it myself but in hindsight it is what people really wanted.

>...content moderation is a Sisyphean task, where we can certainly reduce the problem, but almost by definition cannot solve it.

I don't know, perhaps we can. Should we want to?

> I wonder how differently newsfeeds and sharing will work in 5 years

Ill still be using RSS and Atom.

Re: Is content moderation a dead end?

#153

People with moderating experience: Why not just delete [most] offending comments, immediately, no questions asked (and ban repeat offenders)? For maybe 95% (as a wild guess), there's no question - it's clear that the comment is inflammatory or disinformation or whatever. It surprises me that I see so many of them permitted in so many forums, even here on HN. Why tolerate them? One click and move on. Tell people about…

> "Why not just delete offending comments..." How do you define offending? Some people are offended by a great many things. China is offended if you point out they have human rights abuses. The US is offended if you point out their interference in other country's affairs. Thailand literally makes it illegal (jail time) to offend some people that in the US is not only legal, but encouraged by our culture! It's not so…

IMHO this argument is potentially interesting philosophically, if someone has something new to say. It's an appeal to the logical extreme of post-modern relativism (and when we see logical extremes, I believe it's a good question to ask - is this a real problem or just philosophical). It also is misleading, IMHO, because it conflates morality, offensiveness, and power. Regardless, these kinds of philosophical arguments can be continued indefinitely, but so is the one that says the Internet is a figment of my imagination. I'm talking about reality.

In reality, the human mind doesn't need and very rarely uses the extreme of hard and fast algorithms; we are not computers. I can judge good from bad, constructive from destructive, etc. and it is easy to identify most of the problematic comments. When it's a forum with rules, it's easy to identify (again, a wild guess) 95% of them.

Re: Is content moderation a dead end?

#154

People with moderating experience: Why not just delete [most] offending comments, immediately, no questions asked (and ban repeat offenders)? For maybe 95% (as a wild guess), there's no question - it's clear that the comment is inflammatory or disinformation or whatever. It surprises me that I see so many of them permitted in so many forums, even here on HN. Why tolerate them? One click and move on. Tell people about…

Then people will get offended for getting censored out?

Some will, but is that a loss? If you aim at accommodating destructive behavior, you'll have it and attract more of it. If you aim at accommodating constructive behavior, you'll have it and attract more of it. I'd happily let some other sites have the entire market of destructive behavior.

But we are just speculating; can someone with actual experience and expertise say how it would work?

Re: Is content moderation a dead end?

#155

People with moderating experience: Why not just delete [most] offending comments, immediately, no questions asked (and ban repeat offenders)? For maybe 95% (as a wild guess), there's no question - it's clear that the comment is inflammatory or disinformation or whatever. It surprises me that I see so many of them permitted in so many forums, even here on HN. Why tolerate them? One click and move on. Tell people about…

I think a clever mod will realize that they are not the Supreme Court. They are certainly not the best judge out there. Just clicking accounts and comments out of existence wont solve misinformation - it will just make the mod a tyrant. I know for a fact if i were moderating, there would be few safe users.

Re: Is content moderation a dead end?

#156

Earlier quoted context omitted.

Like the credit bureaus? Seems like they've been doing this for decades already.

This is very different. Credit bureaus are amoral. They're just gathering data and doing math. GP is talking about a world where you can't shop on Amazon because you committed wrongthink on Twitter.

That's a naive view of credit bureaus. There are value judgments in there throughout the stack. You can't get a job because you didn't pay off a medical debt. You can't get a mortgage because you don't have the right history of past debt (for example, if you're too young).

Re: Is content moderation a dead end?

#157

>>> One could also think of big European cities before modern policing - 18th century London or Paris were awash with sin and prone to mob violence, because they were cities. Is the solution in the article? Do we simply need to recognise that as all society is online we now need online police? Online Community support officers (UK police adjuncts (think teaching assistants). I suspect there is an overlap with the "de…

Yes. And that brings up the issue of how, exactly, are we supposed to allow a literal “thought police”.

It's not a thought police firstly. It's a published statement police (pretty sure there are lots of laws like "disturbing the peace")

This will play out over time - call it 30 years - as we try to find out how to do lots of new things

- Handle mental health better. Partly we need medical breakthroughs, but social acceptance will be a huge improvement, as will standardised approaches, early years interventions and detection. It will take huge amounts of parent training - and the recognition that has genuine costs (how many start ups did not start because the parent decided to put their energy into supporting a child. And woe betide anyone suggesting that is another hurdle to be overcome with go getting attitude)

So we need huge investment in dealing with chronic mental health, not just medical, but social support, education etc.

- then sorting out acute. Look at any UK prison. It's basically young men with some mix of drug / mental health / abuse issues. Want to reduce the prison population - start 20 years ago. Don't define the police - simply introduce interventions so that in 20 years they won't need to do the social work job they do now

There is probably a dozen brilliant papers on this clearly showing what we should do and already modelled in a few enlightened communities globally. But it's going to take a decade of mistakes before those percolate up.

Let me know if you spot the me early.

After that it's social norms. We are trying to find a set of behaviours that are acceptable in the new online spaces. Public urination is frowned upon IRL - trolling is the online equivalent I guess. One can imagine things like loss of anonymity being the first part. Then slowly people develop tools to use humans inbuilt social mechanisms - so for example some asshat is intolerable, a record of their conversation is sent to the mother-in-law, 4 grandparents, all their wife's bridesmaids. (this may not work but you get the idea). We know this sort of thing works because every so often we all find that great viral thread where someone gets a comeuppance.

All of this does demand content moderation as ben says - and yes I do think all of this is too much. My take is social media will die back to manageable levels - if we are looking at a mass crowd situation, a swirling football crowd and asking, how can this crowd ever become a manageable city. Well crowds don't - crowds become cities, they disperse.

- There are too many forms of social media - each of us has one main form, and keep up with rest - just as we would have one friend in a crowd but interact with others. This will just die down. I mean how much value do we get from social media versus the constant worry / mind back that makes us check all the time. as humans develop social media innoculations this will die back

Secondly ad dollars will help die back social media - really it's amazing no one has noticed it's a scam and waste money f money.

At some point regulations and advertising will drive to a point where it is simply easier to dump the algorithm, stop driving for engagement and just supply the limited feed of friends posts, interspersed with billboard ads. Influencers will still influence, ads will still pop up, but as we are choosing who to follow we won't care. Just follow fewer people.

Agents - the other big silent one. I can run a refer over my emails but the facebook client won't let me. This is the other big change likely to occur - software agents acting in my best interest filtering shit for me.

Anyway bed time

Re: Is content moderation a dead end?

#158

Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat. You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bub…

That sounds like a pretty fantastic way to build an echo chamber.

it would become an echo chamber.

Re: Is content moderation a dead end?

#159

Earlier quoted context omitted.

Like the credit bureaus? Seems like they've been doing this for decades already.

How do credit bureaus "enforce morality"? I think the point is that the effect of a bad credit rating is much more limited than a universal blackout on the internet. In fact strictly more limited, to the extent that many creditors rely on social media in making their decisions.

I strongly disagree. Bad credit (sometimes even wrongly attributed) can block you from jobs, mobile plans, bank accounts, credit/debit cards, renting, etc. I'd rather be blocked from Facebook than be told I can't rent an apartment or be disqualified from a job.

Re: Is content moderation a dead end?

#160
post #22

Sigh. A "certain level of bad behaviour on the internet and on social might just be inevitable" has been an excuse since the beginning. I worked on the first web-based chat, bianca.com, and I heard it back then. More recently, I worked on anti-abuse at Twitter a few years back and hear the same talking point. Now I work on the problem at a not-for-profit, and it's still a talking point. Ignoring that the social media…

I did think I’d made it extremely explicit that I don’t think any of that at all, but perhaps not (although the way you throw in a rather childish ad hominem sneer suggests you’re not thinking very clearly). What I actually wrote is that though (of course!) there will always be some bad behaviour, we want to minimise it (I compare it to malware, for heaven’s sake), but moderation might not be the best way to minimise…

> ad hominem [...] you're not thinking very clearly

Huh. Not totally sure you understand what "ad hominem" means.

But moving on from that, I'm not objecting to the notion that we might need different models. The way we do anything today is unlikely to be the best way for all of time. Given that I've spent years trying to improve things, perhaps you can take it as read that I think we can improve things.

What I'm objecting to is your fatalism that bad shit is probably going to happen to somebody (a note you include in your closing paragraph) combined with your failure to examine exactly who's going to bear the brunt of it. Something you conspicuously didn't do in your reply here, instead suggesting it's some sort of shocking rudeness to point out that as a rich person, it's unlikely to be you.

Post reply on HN