One old story I remember is about how Lego have built their own virtual world platform and then spent enormous amount of developer time figuring out how to programmatically detect when a user-created content turned out to be shaped like a penis.
You either die an MVP or live long enough to build content moderation
401–410 of 429 posts
Re: You either die an MVP or live long enough to build content moderation
#402There is an untold story here which is how incredibly well HN is moderated. Don’t know how it can remain so good. I feel like the center cannot hold. The site seems to me hideously understaffed, yet they do a pretty much perfect job of moderating. Would love to know if it is all human, supplemented by ML, or what.
While there are rules and expectations on HN, I wouldn't call it heavy moderation. I think there are about 2 moderators? What I see is a reply when something crosses the line and users might down vote the post. Users adjust and carry on.
dang is the public face of what's apparently a multi-member moderation team.
They can be reached at hn@ycombinator.com (the best way to report any major issue or one requiring intervention, e.g., bad links / title typos, etc.).
sctb is frequently remembered as "the other" moderator, though he left HN some time back.
(I've had multiple interactions, mostly positive, with dang. And, I presume, others on the team.)
Re: You either die an MVP or live long enough to build content moderation
#403Just like Batman, "You either die a hero or you live long enough to see yourself become the villain": https://m.youtube.com/watch?v=D_aQupiaCSA
Re: You either die an MVP or live long enough to build content moderation
#404Hey there :). Author here. It was a fun experiment to play around with some different strategies for adding content moderation to https://stream.new Hive ended up being the one I landed on after trying Google Vision first ( https://cloud.google.com/vision ). The other one I was looking at is Clarity.ai but I didn't get a chance to try that one yet.
One thing that wasn't obvious to me is, why did you care about uploads of NSFW? As I understood it, you want to become Imgur of video. Imgur only became so big because they allowed NSFW stuff.
NSFW affects not only the site, but its users.
If a site is specifically intended as, or is incidentally used in the course of, professional work, then NSFW content can literally get your users fired if overseen, logged, or otherwise detected.
That turns out to shift your user demographics in the medium-to-longer term, and not in a direction that's generally compatible with high-quality and engaging content and interactions.
Where NSFW content is permitted, it should be specifically tagged as such, and not presented unless users specifically opt in to it.
Re: You either die an MVP or live long enough to build content moderation
#405Can't help but think back to W. Edwards Deming's distinction between after-the-fact efforts to "inspect" quality into the process -- as opposed to before-the-fact efforts to build quality into the process. OP offers a first-rate review (strategy + tactics!) for the inspection approach. But, the unspoken alternative is to rethink the on-ramp to content-creation privileges, so that only people with net-positive value t…
That's strongly parallel to Gall's Law:
A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.
https://en.wikipedia.org/wiki/John_Gall_(author)#Gall.27s_la...
Re: You either die an MVP or live long enough to build content moderation
#406Earlier quoted context omitted.
It sounds like Second Life lived long enough to build content moderation, pushed the work of content moderation onto its users, and in a hilarious psychological trick worthy of machiavelli, made the users think they own a piece of something (they don't) so that what other users do on "your" land is up to you. My job would also love if I paid them to work there instead of the other way around. The Internet must be hea…
made the users think they own a piece of something (they don't) True, land in Second Life is really a transferable lease. It is a asset, though. You can resell to someone else. Second Life makes most of their money from "tier charges", which work like property taxes and are a fixed amount per square meter. Land resales and rentals, though, are a free market. "Content moderation" is a minor part of using land in Secon…
In both your original comment and in this followup, which makes the point explicit, I find it ironic that SL have addressed one of the key challenges of "spaceless" cyberspace ... by reinventing the concept of space. Which, as you say, "keeps everything from being in the same place".
That's not a criticism. There's a considerable degree of respect. There's some irony in that spacelessness is one of the purported advantages of the online world. It also makes me wonder what, if any, other options might exist, because effective tools for combatting abuse seem scarce, and those that do exist either brittle or capricious.
Re: You either die an MVP or live long enough to build content moderation
#407Noodling at some thoughts on how technology works --- is fundamental mechanisms, not what it is --- I'd come up with a list of eight factors that seemed central. And then thought for a few days and came up with a ninth: hygiene.
In building any technology, what comes to dominate after a time are the unintended or unwanted consequences. This is particularly true of any networked system, in which as the network grows, various cost functions emerge and begin to overwhelm the value prospects.
I suspect that those cost functions are both relatively constant per node, but also tend to increase over time. The result is that small networks can grow quickly (the large positive per-node values dominante the costs), but as the node-count rises, marginal value decreases (see Tilly-Odlyzko's refutation of Metcalfe's law: v = nlog(n)), and* the cost functions increase both as bad actors are attracted to the system and as interactions between nominally good-faith actors become increasingly high-friction.
The consequence is that a "no-moderation" or "light-moderation" policy works really well, until it suddenly doesn't.
This has parallels to other network-based structures, most notably cities. Without sewage, or crime, or public health, or waste-removal, or pollution-control systems, cities can grow fairly rapidly, up to some maximum viable size. After that point, not only do already-extant issues become intolerable (or more specifically: unsurvivable), but new harms co-evolve with the cities themselves to emerge within them. (See Kyle Harper's The Fate of Rome for a wonderful exploration of this concept.)
For online UCG platforms, this presents as various forms of activity that directly attack trust and safety of the platform itself. Ultimately, growth of the platform is often limited by these effects.
Incidentally, a new professional organisation formed last year to address these issues specifically, the TSPA (Trust and Safety Professional Association). It's still finding its feet, though it's begun organising both a training curriculum and a library of practices and references on the issues, processes, considerations, and legal factors involved.
https://www.tspa.info/2020/06/17/new-organizations-dedicated...
Re: You either die an MVP or live long enough to build content moderation
#408Earlier quoted context omitted.
Yeah, there's an entire category of "idea guys" who don't get this. They repeatedly try to crack the code on a truly moderation-free or purely crowd-moderated platform, and it never, ever, ever works. It almost always boils down to a poor understanding of how humans work (usually some sort of "homo economicus") or how computers work (usually some sort of "AI magic wand").
It depends on the scale. I can vouch that crowd-moderation works fine for a small forum (~ 1000 members) which I am part of. And there is no karma system. You get to report posts (3 reports mean that the post is deleted), and warn users (24-hour ban after 3 "active" warnings, and then it scales up to a permanent ban after 15 "active" warnings). Warnings become "inactive" after a month. It also depends on the threat m…
Yeah, I will agree with you that there can be some smaller-scale systems that work fine. That said, in these cases in my experience it's always few key "hero" mods who are just very committed to volunteering to keep things cleaned up.
Without actually hiring people, it's hard to get that level of commitment, and just as you are saying, as soon as the work gets hard enough (eg clever trolls that turn users against each other, or paranoid political crusaders who think the mods are in league with unseen forces), even the best volunteers end up quitting at the worst times.
In the long run I think the solution is just hiring moderators. It costs money, but if you want a job done well and consistently ya gotta pay.
Re: You either die an MVP or live long enough to build content moderation
#409Earlier quoted context omitted.
Yeah, there's an entire category of "idea guys" who don't get this. They repeatedly try to crack the code on a truly moderation-free or purely crowd-moderated platform, and it never, ever, ever works. It almost always boils down to a poor understanding of how humans work (usually some sort of "homo economicus") or how computers work (usually some sort of "AI magic wand").
In the email space this has happened so much that there’s a well-known term for it: FUSSP (Final Ultimate Solution to the Spam Problem).
No wonder the email space is so behind. I'm already working on the next problem, NRTOIATFUSSP
(No, Really, This One Is Actually The Final Ultimate Solution to the Spam Problem)
Re: You either die an MVP or live long enough to build content moderation
#410Here's a thought. Have a platform where identity is verified. Users can post publicly or within their circle. Any illegal or fraudulent content can be handled by the legal system due to the lack of anonymity. Beyond that, let users form groups for topics like reddit. Where does this fall down?
1. Assholes will be assholes under their real names.
2. The politically, economically, and/or socially disadvantaged are further disadvantaged under such regimes. Yonatan Zunger, chief architect of Google+, put that quite clearly: "In practice, the forced revelation of information makes individual privilege and power more important. When everyone has to play with their cards on the table, so to speak, then people who feel like they can be themselves without consequence do so freely -- these generally being people with support groups of like-minded people, and who are neither economically nor physically vulnerable. People who are more vulnerable to consequences use concealment as a method of protection: it makes it possible to speak freely about controversial subjects, or even about any subjects, without fear of harassment."
https://web.archive.org/web/20180903205908/https://plus.goog...
3. Laws and regimes may change. Ask the inhabitants of Germany circa 1933, or those of Afghanistan circa 2021, among numerous other examples.
4. Groups themselves can lead to and engage in destructive activities. See the case of Reddit's increasing problems with brigading and other hostile activity amongst subreddits, resulting in the banning of numerous of these.
5. The consequences of a toxic media landscape at scale are felt by those considerably removed from the platform(s) themselves. Twitter cheered the Arab Spring protests. Facebook are attempting to distance themselves from their very active role in the Myanmar genocide. History shows that changes to media landscapes are often very disruptive socially and politically, often at a horrific cost of innocent lives.